Three-dimensional object detection device

• US 9,135,511 B2
• Filed: 02/26/2013
• Issued: 09/15/2015
• Est. Priority Date: 03/01/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Hayakawa, Yasuhisa, Fukata, Osamu
• Primary Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US14/373,057
• Publication Number:
  US 20140363050A1
• Time in Patent Office: 931 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01S 17/04 Systems determining the presence of a target
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A three-dimensional object detection device includes an image capturing unit, an image conversion unit, a three-dimensional object detection unit, a light source detection unit a degree-of-certainty assessment unit and a control unit. The degree-of-certainty assessment unit assesses a degree of certainty that a light source is headlights of another vehicle in two lanes over. The control unit sets a threshold value so that the three-dimensional object is more difficult to detect in a forward area of a line connecting the light source and the image capturing unit in the detection frame when the degree of certainty is at a predetermined value or higher, and sets a threshold value so that the three-dimensional object is more difficult to detect in progression from a center side toward front or rear ends of the detection frame when the degree of certainty is less than a predetermined value.

Claims

1. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture images of a predetermined area relative to an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device;

an image conversion unit programmed to convert a viewpoint of the images obtained by the image capturing unit to create bird'"'"'s-eye view images;

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane in which the bird'"'"'s-eye view images obtained at different times by the image conversion unit are aligned, and difference waveform information is generated by counting and creating a frequency distribution of a number of pixels that indicate a difference having a predetermined first threshold value or higher in a difference image of the bird'"'"'s-eye view images that were aligned to detect the presence of the three-dimensional object within the adjacent lane upon determining the difference waveform information is at a predetermined second threshold value or higher in a detection frame set inside the predetermined area;

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the images obtained by the image capturing unit;

a degree-of-certainty assessment unit programmed to assess a degree of certainty that the light source is headlights of another vehicle traveling in an adjacent-adjacent lane next to the adjacent lane, based on a mode of the light source detected by the light source detection unit; and

a control unit programmed to set at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in a forward area than a rearward area with respect to a line connecting the light source and the image capturing unit in the detection frame upon determining the degree of certainty is at a predetermined value or higher, and to set the first and second threshold values so that the three-dimensional object is more difficult to detect in progression from a center side toward one of the front or rear ends of the detection frame upon determining the degree of certainty is less than a predetermined value.

2. The three-dimensional object detection device according to claim 1, wherein

the degree-of-certainty assessment unit specifies a number of the light sources detected by the light source detection unit as a mode of the light source, and assesses the degree of certainty based on a specified number of the light sources.

3. The three-dimensional object detection device according to claim 2, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

4. The three-dimensional object detection device according to claim 1, wherein

the degree-of-certainty assessment unit specifies a position of the light source detected by the light source detection unit as a mode of the light source, and assesses the degree of certainty based on a specified position of the light sources.

5. The three-dimensional object detection device according to claim 4, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

6. The three-dimensional object detection device according to claim 1, wherein

the degree-of-certainty assessment unit specifies a size of the light source detected by the light source detection unit as the mode of the light source, and assesses the degree of certainty based on a specified size of the light source.

7. The three-dimensional object detection device according to claim 6, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

8. The three-dimensional object detection device according to claim 1, wherein

the degree-of-certainty assessment unit specifies a change in time of a state of the light source detected by the light source detection unit as the mode of the light source, and assesses the degree of certainty based on a specified change in time of the state of the light source.

9. The three-dimensional object detection device according to claim 8, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

10. The three-dimensional object detection device according to claim 1, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

11. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture images of a predetermined area including an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device;

an image conversion unit programmed to convert a viewpoint of the images obtained by the image capturing unit to create bird'"'"'s-eye view images;

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane based on edge information having a predetermined first threshold value or higher from the bird'"'"'s-eye view images obtained by the image conversion unit, and the three-dimensional object detection unit determining the presence of the three-dimensional object within the adjacent lane upon determining the edge information is at a predetermined second threshold value or higher in a detection frame set inside the predetermined area; and

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the images obtained by the image capturing unit,

a degree-of-certainty assessment unit programmed to assess a degree of certainty that the light source is headlights of another vehicle traveling in an adjacent-adjacent lane next to the adjacent lane, based on a mode of the light source detected by the light source detection unit; and

a control unit programmed to set at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in a forward area than a rearward area with respect to a line connecting the light source and the image capturing unit in the detection frame upon determining the degree of certainty is at a predetermined value or higher, and to set at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in progression from a center side toward one of the front or rear ends of the detection frame upon determining the degree of certainty is less than a predetermined value.

12. The three-dimensional object detection device according to claim 11, wherein

the degree-of-certainty assessment unit specifies a number of the light sources detected by the light source detection unit as a mode of the light source, and assesses the degree of certainty based on a specified number of the light sources.

13. The three-dimensional object detection device according to claim 12, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

14. The three-dimensional object detection device according to claim 11, wherein

the degree-of-certainty assessment unit specifies a position of the light source detected by the light source detection unit as a mode of the light source, and assesses the degree of certainty based on a specified position of the light sources.

15. The three-dimensional object detection device according to claim 14, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

16. The three-dimensional object detection device according to claim 11, wherein

the degree-of-certainty assessment unit specifies a size of the light source detected by the light source detection unit as the mode of the light source, and assesses the degree of certainty based on a specified size of the light source.

17. The three-dimensional object detection device according to claim 16, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

18. The three-dimensional object detection device according to claim 11, wherein

the degree-of-certainty assessment unit specifies a change in time of a state of the light source detected by the light source detection unit as the mode of the light source, and assesses the degree of certainty based on a specified change in time of the state of the light source.

19. The three-dimensional object detection device according to claim 18, wherein

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

20. The three-dimensional object detection device according to claim 11, wherein,

when the degree of certainty is less than a predetermined value, the control unit specifies:

a front end part in which a rearward distance from the image capturing unit, including the front end of the detection frame, is less than a predetermined first distance;

a rear end part in which a rearward distance from the image capturing unit, including the rear end of the detection frame, is a second distance or greater, which is greater than the first distance; and

a center part positioned between the front end part and the rear end part and in which a rearward distance from the image capturing unit is equal to or greater than the first distance and less than the second distance, and

the control unit sets at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in the front end part and in the rear end part than in the center part.

21. A three-dimensional object detection device comprising:

an image capturing unit arranged to capture an image of a predetermined area relative to an adjacent lane rearward of a host vehicle equipped with the three-dimensional object detection device;

an image conversion unit programmed to convert a viewpoint of the image obtained by the image capturing unit to create bird'"'"'s-eye view image;

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object within the adjacent lane based on distribution information of pixels in the bird'"'"'s-eye view image obtained by the image conversion unit in which a luminance difference is at a predetermined threshold value or greater along a direction in which the three-dimensional object collapses when converted in viewpoint to the bird'"'"'s-eye view image, and the three-dimensional object detection unit determining the presence of the the three-dimensional object upon determining an amount of distribution of the pixels in the direction in which the three-dimensional object collapses is at a predetermined second threshold value or greater in the detection frame set in the predetermined area;

a light source detection unit programmed to detect a light source present rearward of the host vehicle based on the image obtained by the image capturing unit;

a degree-of-certainty assessment unit programmed to assess a degree of certainty that the light source is headlights of another vehicle traveling in an adjacent-adjacent lane next to the adjacent lane, based on a mode of the light source detected by the light source detection unit; and

a control unit programmed to set at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in a forward area than a rearward area with respect to a line connecting the light source and the image capturing unit in the detection frame when the degree of certainty is at a predetermined value or higher, and to set at least one of the first and second threshold values so that the three-dimensional object is more difficult to detect in progression from a center side toward one of the front or rear ends of the detection frame upon determining the degree of certainty is less than a predetermined value.