Road surface gradient detection device

• US 9,771,080 B2
• Filed: 04/07/2016
• Issued: 09/26/2017
• Est. Priority Date: 04/17/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Ito, Hiroaki, Sugaya, Fumio
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US15/092,852
• Publication Number:
  US 20160304098A1
• Time in Patent Office: 537 Days
• CPC Class Codes: B60W 40/076 Slope angle of the road
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597

Abstract

A road surface gradient detection device is provided. The road surface gradient detection device includes an image area setting unit configured to divide a captured image to set a plurality of image areas, a weighting setting unit configured to set the weighting of the pixel range, a representative height calculation unit configured to calculate the representative parallax of each image area and the representative height of each image area based on the parallax of each pixel range, the coordinates of each pixel range, and a magnitude of the weighting of each pixel range, and a road surface gradient detection unit configured to detect the road surface gradient from the captured image based on the representative parallax of each image area and the representative height of each image area.

Claims

1. A road surface gradient detection device, the road surface gradient detection device comprising:

an image area setting unit configured to divide a captured image to set a plurality of image areas;

a coordinate recognition unit configured to recognize coordinates of a pixel range corresponding to an image area of the plurality of image areas;

a parallax calculation unit configured to calculate a parallax of the pixel range based on the parallax information of the captured image;

a luminance calculation unit configured to calculate a luminance of the pixel range based on luminance information of the captured image;

a weighting setting unit configured, based on the luminance of the pixel range and the coordinates of the pixel range, to set a weight of the pixel range to a first value when the luminance difference between the pixel range and an adjacent pixel range is less than a first threshold value, and configured, based on the luminance of the pixel range and the coordinates of the pixel range, to set the weight of the pixel range to a second value greater than the first value when the luminance difference between the pixel range and the adjacent pixel range is equal to or greater than the first threshold value;

a representative height calculation unit configured to calculate a representative parallax of the image area and a representative height of the image area based on the parallax of the pixel range, the coordinates of the pixel range, and a magnitude of the weight of the pixel range; and

a road surface gradient detection unit configured to detect a road surface gradient from the captured image based on the representative parallax of the image area and the representative height of the image area.

2. The road surface gradient detection device according to claim 1, further comprising:

a stereoscopic object detection unit configured to detect a stereoscopic object included in the captured image,

wherein the image area setting unit is configured to set the plurality of image areas so as not to include pixels constituting the stereoscopic object.

3. The road surface gradient detection device according to claim 1,

wherein the representative height calculation unit is configured to calculate a height of the pixel range based on the parallax of the pixel range and the coordinates of the pixel range, add the magnitude of the weight of the pixel range to a pixel range having a same height in the image area, and calculate the height having a greatest addition value of the weight as the representative height of the image area.

4. The road surface gradient detection device according to claim 1,

wherein the representative height calculation unit is configured to add the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculate a parallax having a greatest addition value of the weight as the representative parallax of the image area.

5. The road surface gradient detection device according to claim 2,

wherein the representative height calculation unit is configured to calculate a height of the pixel range based on the parallax of the pixel range and the coordinates of the pixel range, add the magnitude of the weight of the pixel range to a pixel range having a same height in the image area, and calculate the height having a greatest addition value of the weight as the representative height of the image area.

6. The road surface gradient detection device according to claim 2,

wherein the representative height calculation unit is configured to add the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculate a parallax having a greatest addition value of the weight as the representative parallax of the image area.

7. The road surface gradient detection device according to claim 3,

wherein the representative height calculation unit is configured to add the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculate a parallax having a greatest addition value of the weight as the representative parallax of the image area.

8. The road surface gradient detection device according to claim 5,

wherein the representative height calculation unit is configured to add the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculate a parallax having a greatest addition value of the weight as the representative parallax of the image area.

9. A road surface gradient detection device, the road surface gradient detection device comprising:

at least one memory comprising computer executable instructions; and

at least one processor configured to read and execute the computer executable instructions, the computer executable instructions causing the at least one processor to perform;

determining coordinates of a pixel range corresponding to an image area from among a plurality of image areas of a captured image;

calculating a parallax of the pixel range based on the parallax information of the captured image;

calculating a luminance of the pixel range based on luminance information of the captured image;

based on the luminance of the pixel range and the coordinates of the pixel range, setting a weight of the pixel range to a first value when the luminance difference between the pixel range and an adjacent pixel range is less than a first threshold value and setting the weight of the pixel range to a second value greater than the first value when the luminance difference between the pixel range and the adjacent pixel range is equal to or greater than the first threshold value;

calculating a representative parallax of the image area and a representative height of the image area based on the parallax of the pixel range, the coordinates of the pixel range, and a magnitude of the weight of the pixel range; and

detecting a road surface gradient from the captured image based on the representative parallax of the image area and the representative height of the image area.

10. The road surface gradient detection device according to claim 9, wherein the computer executable instructions further cause the at least one processor to perform:

detecting a stereoscopic object included in the captured image; and

setting the plurality of image areas so as not to include pixels constituting the stereoscopic object.

11. The road surface gradient detection device according to claim 9,

wherein the calculating the representative height of the image area comprises calculating a height of the pixel range based on the parallax of the pixel range and the coordinates of the pixel range, adding the magnitude of the weight of the pixel range to a pixel range having a same height in the image area, and calculating the height having a greatest addition value of the weight as the representative height of the image area.

12. The road surface gradient detection device according to claim 9,

wherein the calculating the representative parallax of the image area comprises adding the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculating a parallax having a greatest addition value of the weight as the representative parallax of the image area.

13. A road surface gradient detection method, the method comprising:

determining coordinates of a pixel range corresponding to an image area from among a plurality of image areas of a captured image;

calculating a parallax of the pixel range based on the parallax information of the captured image;

calculating a luminance of the pixel range based on luminance information of the captured image;

based on the luminance of the pixel range and the coordinates of the pixel range, setting a weight of the pixel range to a first value when the luminance difference between the pixel range and an adjacent pixel range is less than a first threshold value and setting the weight of the pixel range to a second value greater than the first value when the luminance difference between the pixel range and the adjacent pixel range is equal to or greater than the first threshold value;

calculating a representative parallax of the image area and a representative height of the image area based on the parallax of the pixel range, the coordinates of the pixel range, and a magnitude of the weight of the pixel range; and

detecting a road surface gradient from the captured image based on the representative parallax of the image area and the representative height of the image area.

14. The road surface gradient detection method according to claim 13, further comprising:

detecting a stereoscopic object included in the captured image; and

setting the plurality of image areas so as not to include pixels constituting the stereoscopic object.

15. The road surface gradient detection device according to claim 13,

wherein the calculating the representative height of the image area comprises calculating a height of the pixel range based on the parallax of the pixel range and the coordinates of the pixel range, adding the magnitude of the weight of the pixel range to a pixel range having a same height in the image area, and calculating the height having a greatest addition value of the weight as the representative height of the image area.

16. The road surface gradient detection device according to claim 13,

wherein the calculating the representative parallax of the image area comprises adding the magnitude of the weight of the pixel range to a pixel range having a same parallax in the image area and calculating a parallax having a greatest addition value of the weight as the representative parallax of the image area.