Three-dimensional object detection device

• US 9,830,519 B2
• Filed: 02/26/2013
• Issued: 11/28/2017
• Est. Priority Date: 03/01/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Hayakawa, Yasuhisa, Fukata, Osamu
• Primary Examiner(s): Hallenbeck-Huber, Jeremaiah C
• Assistant Examiner(s): Shahnami, Amir
• Application Number:
  US14/373,049
• Publication Number:
  US 20150002671A1
• Time in Patent Office: 1,736 Days
• Field of Search: 348148
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A three-dimensional object detection device includes an image capturing unit, an image conversion unit, a three-dimensional object detection unit, a light source detection unit and a control unit. The image conversion unit converts a viewpoint of the images obtained by the image capturing unit to create bird'"'"'s-eye view images. The three-dimensional object detection unit detects a presence of a three-dimensional object within the adjacent lane. The three-dimensional object detection unit determines the presence of the three-dimensional object within the adjacent lane-when the difference waveform information is at a threshold value or higher. The control unit set a threshold value higher so that the three-dimensional object is more difficult to detect in a forward area than rearward area with respect to a line connecting the light source and the image capturing unit when the light source has been detected.

Claims

1. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture images of a predetermined area relative to an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device; and

a computer including

an image conversion unit programmed to convert a viewpoint of the images obtained by the image capturing unit to create bird'"'"'s-eye view images,

an alignment unit programmed to generate difference image data by obtaining pixel value differences between bird'"'"'s-eye view images obtained at different times by the image conversion unit,

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane based on the difference image data, the three-dimensional object detection unit generating difference waveform information by counting and creating a frequency distribution of a number of pixels in the difference image data that indicate a pixel value difference having a predetermined first threshold value or higher, the three-dimensional object detecting unit detecting the presence of the three-dimensional object within the adjacent lane upon determining the difference waveform information is at a predetermined second threshold value or higher,

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the images obtained by the image capturing unit, and

a control unit programmed to set at least one of the first threshold value and the second threshold value higher in a forward area than in a rearward area of the bird'"'"'s-eye view image when the light source has been detected by the light source detection unit, the forward area being separated from the rearward area by a line connecting the light source and the image capturing unit in the predetermined area.

2. The three-dimensional object detection device according to claim 1, wherein

the control unit has a first control map indicating a relationship between the first threshold value and luminance, and a second control map indicating a relationship between the first threshold value and luminance in which the first threshold value is set higher than in the first control map, the first threshold value obtained using the first control map and the first threshold value obtained using the second control map being weighted to thereby decide an ultimate first threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the first threshold value obtained using the second control map more than the first threshold value obtained in the first control map in the forward area of the predetermined area, and weights the first threshold value obtained using the first control map more than the first threshold value obtained using the second control map in the rearward area of the predetermined area.

3. The three-dimensional object detection device according to claim 1, wherein

the control unit has a first control map indicating a relationship between the second threshold value and luminance, and a second control map indicating a relationship between the second threshold value and luminance in which the second threshold value is set higher than in the first control map, the second threshold value obtained using the first control map and the second threshold value obtained using the second control map being weighted to thereby decide an ultimate second threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the second threshold value obtained using the second control map more than the second threshold value obtained using the first control map in the forward area of the predetermined area, and weights the second threshold value obtained using the first control map more than the second threshold value obtained using the second control map in the rearward area of the predetermined area.

4. The three-dimensional object detection device according to claim 1, wherein

the control unit sets the first threshold value and the second threshold value higher in the forward area of the predetermined area when the light source has been detected by the light source detection unit.

5. The three-dimensional object detection device according to claim 4, wherein

the control unit has a first control map indicating a relationship between the first threshold value and luminance, and a second control map indicating a relationship between the first threshold value and luminance in which the first threshold value is set higher than in the first control map, the first threshold value obtained using the first control map and the first threshold value obtained using the second control map being weighted to thereby decide an ultimate first threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the first threshold value obtained in the second control map more than the first threshold value obtained using the first control map in the forward area of the predetermined area, and weights of the first threshold value obtained in the first control map more than the first threshold value obtained in the second control map in the rearward area of the predetermined area.

6. The three-dimensional object detection device according to claim 4, wherein

the control unit has a first control map indicating a relationship between the second threshold value and luminance, and a second control map indicating a relationship between the second threshold value and luminance in which the second threshold value is set higher than in the first control map, the second threshold value obtained using the first control map and the second threshold value obtained using the second control map being weighted to thereby decide an ultimate second threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the second threshold value obtained using the second control map more than the second threshold value obtained using the first control map in the forward area of the predetermined area, and weights the second threshold value obtained in the first control map more than the second threshold value obtained using the second control map in the rearward area of the predetermined area.

7. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture an image of a predetermined area including an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device; and

a computer including

an image conversion unit programmed to convert a viewpoint of the image obtained by the image capturing unit to create a bird'"'"'s-eye view image,

an edge line detection unit programmed to detect edge information in the bird'"'"'s-eye view image, the edge information indicating if one or more edge lines exist in the bird'"'"'s-eye view image,

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane based on the edge information, the three-dimensional object detection unit being programmed to determine a quantity of the edge lines in the bird'"'"'s-eye view image for which a change in luminance on the edge line is smaller than a predetermined first threshold value, and the three-dimensional object detection unit determining the presence of the three-dimensional object within the adjacent lane upon determining the quantity of the edge lines is at a predetermined second threshold value or higher,

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the images obtained by the image capturing unit, and

a control unit programmed to set at least one of the first threshold value and the second threshold value higher in a forward area than in a rearward area within the bird'"'"'s-eye view image when the light source has been detected by the light source detection unit, the forward area being separated from the rearward area by a line connecting the light source and the image capturing unit in the predetermined area.

8. The three-dimensional object detection device according to claim 7, wherein

the control unit sets the first threshold value and the second threshold value higher in the forward area of the predetermined area when the light source has been detected by the light source detection unit.

9. The three-dimensional object detection device according to claim 7, wherein

the control unit has a first control map indicating a relationship between the first threshold value and luminance, and a second control map indicating a relationship between the first threshold value and luminance in which the first threshold value is set higher than in the first control map, the first threshold value obtained using the first control map and the first threshold value obtained using the second control map being weighted to thereby decide an ultimate first threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the first threshold value obtained using the second control map more than the first threshold value obtained using the first control map in the forward area of the predetermined area, and weights the first threshold value obtained using the first control map more than the first threshold value obtained in the second control map using the rearward area of the predetermined area.

10. The three-dimensional object detection device according to claim 7, wherein

the control unit has a first control map indicating a relationship between the second threshold value and luminance, and a second control map indicating a relationship between the second threshold value luminance and in which the second threshold value is set higher than in the first control map, the second threshold value obtained using the first control map and the second threshold value obtained using the second control map being weighted to thereby decide an ultimate second threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the second threshold value obtained using the second control map more than the second threshold value obtained using the first control map in the forward area of the predetermined area, and weights the second threshold value obtained using the first control map more than the second threshold value obtained using the second control map in the rearward area of the predetermined area.

11. The three-dimensional object detection device according to claim 8, wherein

the control unit has a first control map indicating a relationship between the first threshold value and luminance, and a second control map indicating a relationship between the first threshold value and luminance in which the first threshold value is set higher than in the first control map, the first threshold value obtained using the first control map and the first threshold value obtained using the second control map being weighted to thereby decide an ultimate first threshold value to be used when the three-dimensional object is to be detected, and

the control unit weights the first threshold value obtained using the second control map more than the first threshold value obtained using the first control map in the forward area of the predetermined area, and weights the first threshold value obtained using the first control map more than the first threshold value obtained using the second control map in the rearward area of the predetermined area.

12. The three-dimensional object detection device according to claim 8, wherein

the control unit has a first control map indicating a relationship between the second threshold value and luminance, and a second control map indicating a relationship between the second threshold value and luminance and in which the second threshold value is set higher than in the first control map, the second threshold value obtained using the first control map and the second threshold value obtained using the second control map being weighted to thereby decide an ultimate second threshold value to be used when the three-dimensional object is to be detected, and

the control unit weighted the second threshold value obtained using the second control map more than the second threshold value obtained using the first control map in the forward area of the predetermined area, and weights the second threshold value obtained using the first control map more than the second threshold value obtained using the second control map in the rearward area of the predetermined area.

13. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture an image of a predetermined area relative to an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device; and

a computer including

an image conversion unit programmed to convert a viewpoint of the image obtained by the image capturing unit to create a bird'"'"'s-eye view image,

a luminance difference calculation unit programmed to calculate luminance differences between pairs of pixels arranged along a direction in which the three-dimensional object collapses when the viewpoint is converted to the bird'"'"'s-eye view image,

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane based on the luminance differences calculated by the luminance difference calculation unit, the of pixels in the bird'"'"'s-eye view image obtained by the image conversion unit, the three-dimensional object detection unit determining a quantity of the calculated luminance differences at a predetermined first threshold value or greater, and the three-dimensional object detection unit determining the presence of the three-dimensional object upon determining that the quantity is at a predetermined second threshold value or greater,

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the image obtained by the image capturing unit, and

a control unit programmed to set at least one of the first threshold value and the second threshold value higher in a forward area than a rearward area of the bird'"'"'s-eye view image when the light source has been detected by the light source detection unit, the forward area being separated from the rearward area by a line connecting the light source and the image capturing unit in the predetermined area.