Three-dimensional object detection device, and three-dimensional object detection method

• US 9,558,546 B2
• Filed: 07/26/2013
• Issued: 01/31/2017
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Clarion Co., Ltd. (Faurecia SA), Nissan Motor Co., Ltd.
• Original Assignee: Clarion Co., Ltd. (Faurecia SA), Nissan Motor Co., Ltd.
• Inventors: Utagawa, Akira, Hayakawa, Yasuhisa, Fukata, Osamu, Takemura, Masayuki, Muramatsu, Shoji, Irie, Kota
• Primary Examiner(s): Zhao, Daquan
• Application Number:
  US14/410,730
• Publication Number:
  US 20150324972A1
• Time in Patent Office: 1,285 Days
• Field of Search: 386/300, 386/307, 386/308, 386/311, 386/312, 386/291, 386/326, 386/353, 348/254, 348/255, 348/276, 348/222.1
• US Class Current: 1/1
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A three-dimensional object detection device has an image capturing unit, an object detection unit, a nighttime assessment unit, a luminance detection unit, a luminance peak detection unit and a controller. The image capturing unit captures images rearward of a vehicle. The object detection unit detects a presence of an object from the captured images. The nighttime assessment unit assesses if nighttime has fallen. The luminance detection unit detects a luminance of image areas from the captured image. The luminance peak detection unit detects a peak in the luminance having a luminance gradient that is greater than or equal to a predetermined reference value from among the detected peaks in the luminance as a target luminance peak. The controller controls detection of the object in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen by the nighttime assessment unit.

Claims

1. A three-dimensional object detection device comprising:

an image capturing device mounted on a host vehicle, and comprising a lens for capturing images behind the host vehicle;

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object in a detection area based on captured images acquired by the image capturing unit; and

a nighttime assessment unit programmed to assess whether or not nighttime has fallen;

a luminance detection unit programmed to detect a luminance of a plurality of image areas based on the captured image;

a luminance peak detection unit programmed to detect a peak in the luminance having a luminance gradient that is greater than or equal to a predetermined reference value from among the peaks in the luminance detected by the luminance detection unit as a target luminance peak; and

a controller programmed to control detection of the three-dimensional object by the three-dimensional object detection unit in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen by the nighttime assessment unit.

2. The three-dimensional object detection device according to claim 1, wherein

the controller is further programmed to suppress the detection of the three-dimensional object by the three-dimensional object detection unit in an image area in which the target luminance peak is detected by prohibiting the detection of the three-dimensional object by the three-dimensional object detection unit.

3. The three-dimensional object detection device according to claim 1, wherein

the controller is further programmed to suppress the detection of the three-dimensional object by the three-dimensional object detection unit based on at least one of the number of the target luminance peaks and the magnitude of the luminance values of the target luminance peaks.

4. The three-dimensional object detection device according to claim 1, further comprising:

a lens state assessment unit programmed to assess a degree of pollution of the lens based on the captured images,

the controller being further programmed to set the suppression degree to be higher when suppressing the detection of the three-dimensional object with higher amounts of pollution by promoting the detection of the target luminance peak by the luminance peak detection unit.

5. The three-dimensional object detection device according to claim 1, further comprising:

a traveling speed acquisition unit programmed to acquire the traveling speed of the three-dimensional object and the traveling speed of a host vehicle; and

a three-dimensional object assessment unit programmed to determine whether or not the three-dimensional object is another vehicle based on the traveling speed of the three-dimensional object and the traveling speed of the host vehicle;

wherein

the controller being further programmed to decrease the suppression degree when suppressing the detection of the three-dimensional object when the three-dimensional object is determined to be another vehicle VX by the three-dimensional object assessment unit.

6. The three-dimensional object detection device according to claim 1, further comprising:

a light source detection unit programmed to detect an image area having a luminance difference with a surrounding of equal to or greater than a predetermined luminance difference and having a size that is equal to or greater than a predetermined area as a light source corresponding to the headlight of another vehicle;

the controller being further programmed to decrease the suppression degree when suppressing the detection of the three-dimensional object by the three-dimensional object detection unit when the light source is detected.

7. The three-dimensional object detection device according to claim 6, further comprising:

a road surface state estimation unit programmed to estimate whether or not a road surface state is such that a water film has formed on a road surface;

wherein

the controller being further programmed to suppress the detection of a light source corresponding to a headlight of another vehicle by the light source detection unit when a speculation has been made by the road surface state estimation unit that the road surface state is such that the water film has formed on the road surface.

8. The three-dimensional object detection device according to claim 1, further comprising:

a road surface state estimation unit programmed to estimate whether or not a water film has formed on a road surface;

the controller being further programmed to increase the suppression degree when suppressing the detection of the three-dimensional object when a speculation has been made by the road surface state estimation unit that a road surface state is such that the water film has formed on the road surface by promoting the detection of the target luminance peak by the luminance peak detection unit.

9. The three-dimensional object detection device according to claim 8, wherein

the road surface state estimation unit is further programmed to estimate whether or not the road surface state is such that the water film has formed on the road surface by assessing whether or not rain is falling in the host vehicle'"'"'s surroundings based on an operating state of a wiper.

10. The three-dimensional object detection device according to claim 8, wherein

the road surface state estimation unit is further programmed to estimate that the road surface state is such that the water film has formed on the road surface when an operating state of a wiper is changed from a high operation frequency state to a low operation frequency state.

11. The three-dimensional object detection device according to claim 8, wherein

the road surface state estimation unit is further programmed to specify one assessment line out of a plurality of assessment lines along a direction in which a three-dimensional object collapses when the viewpoint of the captured images are converted to the bird'"'"'s-eye view images as the reference assessment line;

when the luminance difference between the luminance of an image area on the reference line and the luminance of an image area on one or a plurality of comparative assessment lines, including the assessment lines that are adjacent to the reference assessment line, is less than a predetermined value, the road surface state estimation unit estimates that the road surface state is such that the water film has formed on the road surface.

12. A three-dimensional object detection device comprising:

an image capturing device mounted on a host vehicle, and comprising a lens for capturing images behind the host vehicle;

an image conversion unit programmed to perform viewpoint conversion of the captured images obtained by the image capturing device into bird'"'"'s-eye view images;

the three-dimensional object detection unit being programmed to detect a presence of a three-dimensional object based on differential waveform information in which the differential waveform information is generated by aligning in bird'"'"'s-eye view positions of the bird'"'"'s-eye view images obtained at different times by the image conversion device, and counting a number of pixels that indicate a predetermined difference in a difference image of aligned bird'"'"'s-eye view images to form a frequency distribution;

a nighttime assessment unit programmed to assess whether or not nighttime has fallen;

a luminance detection unit programmed to detect a luminance of a plurality of image areas based on the captured images;

a luminance peak detection unit programmed to detect a peak in the luminance having a luminance gradient that is greater than or equal to a predetermined reference value from among the peaks in the luminance detected by the luminance detection unit as a target luminance peak; and

a controller programmed to control detection of the three-dimensional object by the three-dimensional object detection unit in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen by the nighttime assessment unit.

13. The three-dimensional object detection device according to claim 12, wherein

the three-dimensional object detection unit is further programmed to generate the differential waveform information by counting a number of pixels that indicate a difference of equal to or greater than a predetermined first threshold value to form the frequency distribution, and three-dimensional object detection unit detects the three-dimensional object on the difference image of the bird'"'"'s-eye view image based on the differential waveform when the peak value of the differential waveform information is equal to or greater than a predetermined second threshold value;

and

the controller is further programmed to suppress the detection of the three-dimensional object by the three-dimensional object detection unit based on a result of detecting the target luminance peak by changing one of the first threshold value and the second threshold value to a higher value.

14. The three-dimensional object detection device according to claim 12, wherein

when the three-dimensional object detection unit generates the differential waveform information, the controller is further programmed to cause the three-dimensional object detection unit to calculate the value obtained by counting a number of pixels that indicate the predetermined difference in the difference image of the bird'"'"'s-eye view images to form the frequency distribution to lower, thereby suppressing the detection of the three-dimensional object by the three-dimensional object detection unit.

15. A three-dimensional object detection device comprising:

an image capturing device mounted on a host vehicle, and comprising a lens for capturing images behind the vehicle;

an image conversion unit programmed to perform viewpoint conversion of the captured images obtained by the image capturing device into bird'"'"'s-eye view images;

the three-dimensional object detection unit programmed to detect a presence of a three-dimensional object based on edge information that is detected from the bird'"'"'s-eye view images obtained by the image conversion unit;

a nighttime assessment unit programmed to assess whether or not nighttime has fallen;

a luminance detection unit programmed to detect a luminance of a plurality of image areas based on the captured images;

a luminance peak detection unit programmed to detect a peak in the luminance having a luminance gradient that is greater than or equal to a predetermined reference value from among the peaks in the luminance detected by the luminance detection unit as a target luminance peak; and

a controller programmed to control detection of the three-dimensional object by the three-dimensional object detection unit in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen by the nighttime assessment unit.

16. The three-dimensional object detection device according to claim 15, wherein

the three-dimensional object detection unit is further programmed to detect edge components that have a luminance difference between adjacent pixel areas of equal to or greater than a predetermined first threshold value based on the bird'"'"'s-eye view images and to detect the three-dimensional object based on the edge information when a quantity of the edge information based on the edge components is equal to or greater than a predetermined second threshold value;

and

the controller is further programmed to suppress the detection of the three-dimensional object by the three-dimensional object detection unit based on a result of detecting the target luminance peak by changing one of the first threshold value and the second threshold value to a higher value.

17. The three-dimensional object detection device according to claim 15, wherein

the controller is further programmed to suppress the detection of the three-dimensional object by the three-dimensional object detection unit when the three-dimensional object detection unit detects the edge information by causing the three-dimensional object detection unit to output the edge information as low.

18. A three-dimensional object detection method comprising:

converting viewpoints of captured images into bird'"'"'s-eye view images;

aligning positions of the bird'"'"'s-eye view images that are obtained at different times in a bird'"'"'s-eye view;

generating differential waveform information by counting a number of pixels that indicate a predetermined difference in a difference image of aligned bird'"'"'s-eye view images to form a frequency distribution;

detecting a presence of a three-dimensional object based on the differential waveform information;

determining whether or not nighttime has fallen;

detecting a luminance in a plurality of image areas based on the captured images;

detecting a peak in the luminance having a luminance gradient of a predetermined value or more, from among the detected peaks in the luminance, as a target luminance peak; and

suppressing the detection of the three-dimensional object in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen.

19. A three-dimensional object detection method comprising:

converting viewpoints of captured images into bird'"'"'s-eye view images;

detecting edge information from the bird'"'"'s-eye view images; and

detecting a presence of a three-dimensional object based on the edge information;

determining whether or not nighttime has fallen;

detecting a luminance in a plurality of image areas based on the captured images;

detecting a peak in the luminance having a luminance gradient of a predetermined value or more, from among the detected peaks in the luminance, as a target luminance peak; and

suppressing the detection of the three-dimensional object in an image area in which the target luminance peak is detected when an assessment has been made that nighttime has fallen.