Three-dimensional object detection device

• US 9,349,059 B2
• Filed: 07/24/2013
• Issued: 05/24/2016
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Hayakawa, Yasuhisa, Fukata, Osamu
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US14/406,817
• Publication Number:
  US 20150125031A1
• Time in Patent Office: 1,035 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A three-dimensional object detection has an image capturing device, a three-dimensional object detection unit, a high-luminance area assessment unit and a controller. The image capturing device captures images of an area including a right-side detection area or a left-side detection area rearward of a vehicle. The three-dimensional object detection unit detects a three-dimensional object based on the images acquired by the image capturing device. The high-luminance area assessment unit accesses a first detection area including a high-luminance area complying with a predetermined reference on either the right-side detection area or the left-side detection area. The controller suppresses detection of the three-dimensional object based on image information of the first detection area that was detected, and maintains or promotes detection of the three-dimensional object based on image information of a second detection area other than the first detection area within the right-side detection area or the left-side detection area.

Claims

1. A three-dimensional object detection device comprising:

an image capturing unit configured to capture images of an area including a right-side detection area or a left-side detection area rearward of a vehicle;

a three-dimensional object detection unit programmed to detect a three-dimensional object based on the images acquired by the image capturing unit;

a high-luminance area assessment unit programmed to assess whether one of the right-side detection area and the left-side detection area includes a high-luminance area complying with a predetermined reference, and—

if one of the right-side detection area and the left-side detection area includes the high-luminance area—

to selectively designate the one of the right-side detection area and the left-side detection area as a first detection area and the other of the right-side detection area and the left-side detection area as a second detection area, the high-luminance area corresponding to a bright environment illuminated by light; and

a controller programmed to selectively suppress detection of the three-dimensional object based on image information of the first detection area, and to maintain or promote detection of the three-dimensional object based on image information of the second detection area.

2. The three-dimensional object detection device according to claim 1, wherein

the high-luminance area assessment unit is programmed to assess whether one of the right-side detection area and the left-side detection area will be designated as the first detection area by determining whether one of the right-side detection area and the left-side detection area in which pixels at or above a predetermined luminance exist at or above a predetermined density within the right-side detection area or the left-side detection area.

3. The three-dimensional object detection device according to claim 1, wherein

the controller is programmed to suppress assessment that the three-dimensional object that was detected is another vehicle based on image information corresponding to the one of the right-side detection area and the left-side detection area that was designated as the-first detection area.

4. A three-dimensional object detection device comprising:

an image capturing unit configured to capture images of an area including a right-side detection area or a left-side detection area rearward of a vehicle;

a three-dimensional object detection unit programmed to detect a three-dimensional object based on the images acquired by the image capturing unit;

a high-luminance area assessment unit programmed to assess a first detection area including a high-luminance area complying with a predetermined reference on either the right-side detection area or the left-side detection area;

an image conversion unit programmed to perform viewpoint conversion of the images obtained by the image capturing unit to bird'"'"'s-eye view images;

a three-dimensional object assessment unit programmed to access whether the three-dimensional object detected by the three-dimensional object detection unit is another vehicle, the three-dimensional object detection unit being further programmed to detect a presence of the three-dimensional object in a detection area set on each of a right side and a left side rearward of the vehicle based on differential waveform information in which the differential waveform information is generated by aligning bird'"'"'s-eye view positions of bird'"'"'s-eye view images obtained at different points in time by the image conversion unit, and counting a number of pixels indicating a predetermined difference in a differential image on the bird'"'"'s-eye view images that were aligned in which the predetermined difference is indicated by a luminance difference in the differential image along a direction of collapsing of the three-dimensional object when viewpoint conversion of the bird'"'"'s-eye view images is performed to create a frequency distribution; and

a controller programmed to suppress detection of the three-dimensional object or assessment that the three-dimensional object is another vehicle based on image information of the detected first detection area, and to maintain or promote detection of the three-dimensional object or assessment that the three-dimensional object is another vehicle based on image information of a second detection area other than the first detection area within the right-side detection area or the left-side detection area.

5. The three-dimensional object detection device according to claim 4, wherein

the three-dimensional object detection unit is further programmed to detect the three-dimensional object based on the differential waveform information and a first threshold value α

; and

when the first detection area is assessed by the high-luminance area assessment unit, the controller is programmed to output to the three-dimensional object detection unit a control command for upwardly adjusting the first threshold value α

when performing three-dimensional object detection processing based on image information corresponding to the first detection area so that the three-dimensional object is difficult to detect, and downwardly adjusting or maintaining at the same value the first threshold value a when performing three-dimensional object detection processing based on image information corresponding to the second detection area so that the three-dimensional object is easy to detect.

6. The three-dimensional object detection device according to claim 4, wherein

when the first detection area was assessed by the high-luminance area assessment unit, the controller is programmed to output to the three-dimensional object detection unit a control command for outputting a downwardly adjusted value obtained by counting the number of the pixels indicating a predetermined difference on the differential image of the bird'"'"'s-eye view image and forming the frequency distribution in a three-dimensional object detection process based on image information corresponding to the first detection area, and to output an upwardly adjusted, or the same value, obtained by counting the number of the pixels indicating the predetermined difference in the differential image of the bird'"'"'s-eye view images and forming the frequency distribution in the three-dimensional object detection process based on image information corresponding to the second detection area.

7. A three-dimensional object detection device comprising:

an image capturing unit configured to capture images of an area including a right-side detection area or a left-side detection area rearward of a vehicle;

a three-dimensional object detection unit programmed to detect a three-dimensional object based on the images acquired by the image capturing unit;

a high-luminance area assessment unit programmed to assess a first detection area including a high-luminance area complying with a predetermined reference on either the right-side detection area or the left-side detection area;

an image conversion unit programmed to perform viewpoint conversion of the images obtained by the image capturing unit to bird'"'"'s-eye view images;

a three-dimensional object assessment unit programmed to access whether a three-dimensional object detected by the three-dimensional object detection unit is another vehicle, the three-dimensional object detection unit being further programmed to detect a presence of the three-dimensional object based on edge information derived from an edge component that is detected, on the bird'"'"'s-eye view images obtained by the image conversion unit, the edge component including pixels in which a luminance difference between mutually adjacent image areas is greater than or equal to a predetermined difference along a direction of collapsing of the three-dimensional object is used, when viewpoint conversion of the bird'"'"'s-eye view images is performed, to detect the presence of the three-dimensional object based on the edge information derived from the edge component; and

a controller programmed to suppress detection of the three-dimensional object or assessment that the three-dimensional object is another vehicle based on image information of the detected first detection area, and to maintain or promote detection of the three-dimensional object or assessment that the three-dimensional object is another vehicle based on image information of a second detection area other than the first detection area within the right-side detection area or the left-side detection area.

8. The three-dimensional object detection device according to claim 7, wherein

the three-dimensional object detection unit is further programmed to detect the three-dimensional object based on the edge information and a second threshold value β

; and

when the first detection area is assessed by the high-luminance area assessment unit, the controller is programmed to output to the three-dimensional object detection unit a control command for upwardly adjusting a second threshold value β

when performing three-dimensional object detection processing based on image information corresponding to the first detection area so that the three-dimensional object is difficult to detect, and downwardly adjusting or not changing a second threshold value β

when performing three-dimensional object detection processing based on image information corresponding to the second detection area so that the three-dimensional object is easy to detect.

9. The three-dimensional object detection device according to claim 7, wherein

when the first detection area is assessed by the high-luminance area assessment unit, the controller is programmed to output to the three-dimensional object detection unit a control command to output a downwardly adjusted amount of edge information detected in three-dimensional object detection processing based on image information corresponding to the first detection area, and an upwardly adjusted, or the same, amount of edge information detected in three-dimensional object detection processing based on image information corresponding to the second detection area.

10. A three-dimensional object detection device comprising:

an image capturing unit configured to capture images of an area including a right-side detection area or a left-side detection area rearward of a vehicle;

a three-dimensional object detection unit programmed to detect a three-dimensional object based on the images acquired by the image capturing unit;

a high-luminance area assessment unit programmed to assess a first detection area including a high-luminance area complying with a predetermined reference on either the right-side detection area or the left-side detection area, the high-luminance area assessment unit is further programmed to execute;

acquiring a traveling direction and a travel location of the vehicle;

referencing a correlation obtained by associating left and right identifiers of a first detection area predicted to include a high-luminance area in a relationship between the traveling direction of the vehicle and a direction in which the sun is present; and

assessing that the first detection area includes the high-luminance area within the right-side detection area or the left-side detection area based on a direction in which the sun is present at the travel location that was acquired as assessed from the travel location that was acquired and calendar information that associates a direction in which the sun is present at individual travel locations and a point in time, and based on the acquired traveling direction at the acquired travel location of the vehicle; and

a controller programmed to suppress detection of the three-dimensional object based on image information of the first detection area that was detected, and to maintain or promote detection of the three-dimensional object based on image information of a second detection area other than the first detection area within the right-side detection area or the left-side detection area.

11. The three-dimensional object detection device according to claim 10, wherein

the correlation includes a first correlation for when a three-dimensional object has not been detected by the three-dimensional object detection unit and a second correlation for when a three-dimensional object has been detected by the three-dimensional object detection unit;

and

the high-luminance area assessment unit is further programmed to access the first detection area by referencing the first correlation upon acquiring a detection result that the three-dimensional object detection unit has not detected a three-dimensional object, and referencing the second correlation upon acquiring a detection result that the three-dimensional object detection unit detected a three-dimensional object.

12. The three-dimensional object detection device according to claim 10, wherein

the high-luminance area assessment unit is further programmed to detect a travel time point of the vehicle, reference the calendar information associating a height of the sun at individual sites with a point in time, and reference the correlation to assess a high-luminance area when the height of the sun at the travel location of the vehicle detected is less than a predetermined height.