THREE-DIMENSIONAL OBJECT DETECTION DEVICE AND THREE-DIMENSIONAL OBJECT DETECTION METHOD

• US 20150324651A1
• Filed: 07/25/2013
• Published: 11/12/2015
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: FUKATA, Osamu, HAYAKAWA, Yasuhisa
• Granted Patent: US 9,589,193 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A three-dimensional object detection device has an image capturing unit, a first object detection unit, a light source detection unit, a second object detection unit, a degree-of-clouding calculation unit and a controller. The image capturing unit captures images rearward of a vehicle. The first object detection unit detects a presence of a three-dimensional object from the captured images. The second object detection unit detects a presence of a three-dimensional object from a detection result of the light source detection unit. The degree-of-clouding calculation unit calculates a degree of lens clouding from the captured images. The controller assess the object as an adjacent vehicle in an adjacent lane from the degree of lens clouding when the degree of lens clouding is equal to or greater than a predetermined value and from the presence of the object in the captured images when the degree of lens clouding is less than the value.

Claims

1. A three-dimensional object detection device comprising:

an image capturing unit having a lens for forming captured images of an area rearward of a host vehicle;

a first three-dimensional object detection unit programmed to detect a presence of a three-dimensional object-based on the captured images captured by the image capturing unit;

a light source detection unit programmed to detect a light source which is present behind the host vehicle based on the captured images captured by the image capturing unit;

a second three-dimensional object detection unit programmed to detect a presence of a three-dimensional object based on a detection result of the light source detected by the light source detection unit;

a degree-of-clouding calculation unit programmed to calculate a degree to which the lens is clouded as a degree of lens clouding based on the captured images; and

a controller programmed to;

assess whether the three-dimensional object is an adjacent vehicle present in an adjacent lane adjacent to a lane of travel of the host vehicle based on at least a detection result from the second three-dimensional object detection unit when the degree of lens clouding is equal to or greater than a predetermined determination value; and

assess whether the three-dimensional object is the adjacent vehicle based on at least a detection result from the first three-dimensional object detection unit when the degree of lens clouding is less than the determination value.

2. The three-dimensional object detection device according to claim 1, wherein

the first three-dimensional object detection unit includes an image conversion unit programmed to perform viewpoint conversion of the captured images captured by the image capturing unit to bird'"'"'s-eye view images;

the first three-dimensional object detection is further programmed to detect the presence of the three-dimensional object based on differential waveform information in which the differential waveform information is generated by aligning in bird'"'"'s-eye view positions of the bird'"'"'s-eye view images obtained at different times by the image conversion unit, and counting a number of pixels indicating a predetermined difference in a differential image of aligned bird'"'"'s-eye view images along a direction in which the three-dimensional object collapses when the viewpoint conversion of the images that were captured to the bird'"'"'s-eye view images to form a frequency distribution; and

the degree-of-clouding calculation unit is further programmed to calculate the degree of lens clouding based on a number of peaks having a value equal to or greater than a predetermined threshold value among peaks in the differential waveform information generated within a predetermined time interval.

3. The three-dimensional object detection device according to claim 2, wherein

the controller is programmed to weight the detection result of the first three-dimensional object detection unit relatively smaller and to weight the detection result of the second three-dimensional object detection unit relatively larger as the degree of lens clouding becomes higher, and

the controller is programmed to assess whether the three-dimensional object is the adjacent vehicle based on a weighted detection result from the first three-dimensional object detection unit and a weighted detection result from the second three-dimensional object detection unit.

4. The three-dimensional object detection device according to claim 2, wherein

the degree-of-clouding calculation unit is further programmed to initiate calculation of the degree of lens clouding when the light source detection unit detects the light source.

5. The three-dimensional object detection device according to claim 4, wherein

when initiating calculation of the degree of lens clouding, the degree-of-clouding calculation unit is further programmed to set the predetermined time interval using a time that the light source was detected as a reference, and to calculate the degree of lens clouding based on a number of peaks in the differential waveform information having a value equal to or greater than the predetermined threshold value among the peaks in the differential waveform information generated within the predetermined time interval.

6. The three-dimensional object detection device according to claim 4, wherein

when initiating calculation of the degree of lens clouding, the degree-of-clouding calculation unit is further programmed to set an amount of time before the time at which the light source was detected in the predetermined time interval for calculating the degree of lens clouding.

7. The three-dimensional object detection device according to claim 2, wherein

the degree-of-clouding calculation unit is further programmed to calculate a first score based on a number of peaks in the differential waveform information having a value equal to or greater than the predetermined threshold value in the differential waveform information, to calculate a second score based on a total value of the first scores calculated within the predetermined time interval, and to calculate an integrated value of the second score that was calculated as the degree of lens clouding.

8. The three-dimensional object detection device according to claim 7, wherein

the degree-of-clouding calculation unit is further programmed to set an upper limit to at least one of the first and second scores.

9. The three-dimensional object detection device according to claim 1, wherein

the first three-dimensional object detection unit includes an image conversion unit programmed to perform viewpoint conversion of the captured image s captured by the image capturing unit to bird'"'"'s-eye view images;

the first three-dimensional object detection is further programmed to detect the presence of the three-dimensional object based on edge information detected from the bird'"'"'s-eye view images obtained by the image conversion unit, in which the edge information is detected along the direction in which the three-dimensional object collapses when the viewpoint conversion of the images that were captured to the bird'"'"'s-eye view images; and

the degree-of-clouding calculation means unit is further programmed to calculate the degree of lens clouding based on a quantity of edge lines included in the edge information detected within a predetermined time interval.

10. The three-dimensional object detection device according to claim 9, wherein

the controller is programmed to weight the detection result of the first three-dimensional object detection unit relatively smaller and to weight the detection result of the second three-dimensional object detection unit relatively larger as the degree of lens clouding becomes higher, and

the controller is programmed to assess whether the three-dimensional object is the adjacent vehicle based on a weighted detection result from the first three-dimensional object detection unit and a weighted detection result from the second three-dimensional object detection unit.

11. The three-dimensional object detection device according to claim 9, wherein

the degree-of-clouding calculation unit is further programmed to initiate calculation of the degree of lens clouding when the light source detection unit detects the light source.

12. The three-dimensional object detection device according to claim 11, wherein

when initiating calculation of the degree of lens clouding, the degree-of-clouding calculation unit is further programmed to set the predetermined time interval using a time that the light source was detected as a reference, and to calculate the degree of lens clouding based on a quantity of edge lines included in the edge information detected within the predetermined time interval.

13. The three-dimensional object detection device according to claim 11, wherein

when initiating calculation of the degree of lens clouding, the degree-of-clouding calculation unit is further programmed to set an amount of time before the time at which the light source was detected in the predetermined time interval in calculating the degree of lens clouding.

14. The three-dimensional object detection device according to claim 9, wherein

the degree-of-clouding calculation unit is further programmed to calculate a first score based on a quantity of the edge lines included in the edge information, to calculate a second score based on a total value of the first scores calculated within the predetermined time interval, and calculates a integrated value of the calculated second score as the degree of lens clouding.

15. The three-dimensional object detection device according to claim 14, wherein

the degree-of-clouding calculation unit is further programmed to set an upper limit to at least one of the first and second scores.

16. A three-dimensional object detection method comprising:

converting viewpoints of captured images of an area rearward of a host vehicle to bird'"'"'s-eye view images;

aligning positions of the bird'"'"'s-eye view images obtained at different times in bird'"'"'s-eye view;

generating differential waveform information by counting a number of pixels indicating a predetermined difference in a differential image of aligned bird'"'"'s-eye view images along direction in which a three-dimensional object collapses when the viewpoints of the captured images are converted to the bird'"'"'s-eye view images to form a frequency distribution;

detecting a presence of a three-dimensional object based on the differential waveform information and outputting a first detection result;

detecting a light source which is present behind the host vehicle based on the captured images;

detecting a presence of the three-dimensional object based on a detection result of the light source being present behind the host vehicle and outputting a second detection result; and

assessing whether the three-dimensional object is an adjacent vehicle which is present in an adjacent lane adjacent to a lane of travel of the host vehicle based on the first detection result and the second detection result;

calculating a degree to which a lens is clouded as a degree of lens clouding based on a number of peaks in the differential waveform information having a value equal to or greater than a predetermined threshold value among peaks in the differential waveform information generated within a predetermined time interval;

assessing whether the three-dimensional object is the adjacent vehicle based on at least the second detection result from among the first detection result and the second detection result when the degree of lens clouding is equal to or greater than a predetermined determination value; and

assessing whether the three-dimensional object is the adjacent vehicle based on at least the first detection result from among the first detection result and the second detection result when the degree of lens clouding is less than the determination value.

17. A three-dimensional object detection method comprising:

converting viewpoints of captured images of an area rearward of a host vehicle to bird'"'"'s-eye view images;

detecting edge information along a direction in which a three-dimensional object collapses when the viewpoints of the captured images are converted to the the bird'"'"'s-eye view images;

detecting a presence of the three-dimensional object based on the edge information and outputting a first detection result;

detecting a light source which is present behind the host vehicle based on the captured image;

detecting a presence of the three-dimensional object based on a detection result of the light source being present behind the host vehicle and outputting a second detection result; and

assessing whether the three-dimensional object is an adjacent vehicle which is present in an adjacent lane adjacent to a lane of travel of the host vehicle based on the first detection result and the second detection result;

calculating a degree to which a lens is clouded as a degree of lens clouding based on a quantity of edge lines included in the edge information detected within a predetermined time interval;

assessing whether the three-dimensional object is the adjacent vehicle based on at least the second detection result from among the first detection result and the second detection result when the degree of lens clouding is equal to or greater than a predetermined determination value; and

assessing whether the three-dimensional object is the adjacent vehicle based on at least the first detection result from among the first detection result and the second detection result when the degree of lens clouding is less than the determination value.