Three-dimensional object detection device, and three-dimensional object detection method

• US 9,862,320 B2
• Filed: 07/24/2013
• Issued: 01/09/2018
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Hayakawa, Yasuhisa, Fukata, Osamu
• Primary Examiner(s): Philippe, Gims
• Assistant Examiner(s): Mung, On S
• Application Number:
  US14/406,794
• Publication Number:
  US 20150145956A1
• Time in Patent Office: 1,630 Days
• Field of Search: 348 46, 348148
• CPC Class Codes: B60R 1/002 specially adapted for covering the peripheral part of the vehicle, e.g. for viewing tyres, bumpers or the like integrated in the windows B60R1/001; combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60S 1/46 using liquid; Windscreen washers
• CPC Class Codes: B60S 1/56 specially adapted for cleaning other parts or devices than front windows or windscreens rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A three-dimensional object detection device has a camera, a three-dimensional object detection unit, a lens cleaning device, a lens state assessment unit and a controller. The camera has a lens for forming an image of an area rearward of a vehicle. The three-dimensional object detection unit detects a three-dimensional object rearward of the vehicle based on the captured images. The lens cleaning device sprays cleaning fluid to clean the lens of the camera. The lens state assessment unit accesses whether the lens is in a predetermined state subject to control based on a timing at which cleaning fluid is sprayed on the lens. The controller suppresses detection of the three-dimensional object by retaining detection or assessment results for a predetermined length of time that were obtained immediately before the lens was assessed to be in the state subject to control, upon assessment that the lens state is subject to control.

Claims

1. A three-dimensional object detection device comprising:

a camera provided with a lens for forming a video image of an area rearward of a vehicle;

a three-dimensional object detection unit programmed to detect a three-dimensional object rearward of the vehicle based on images captured by the camera;

a lens cleaning device arranged to spray cleaning fluid to clean the lens of the camera;

a lens state assessment unit programmed to assess whether the lens is in a predetermined wetness state during a lens cleaning process executed by the lens cleaning unit; and

a controller programmed to suspend detection of the three-dimensional object for a predetermined length of time upon the lens state assessment unit assessing that the lens is in the predetermined wetness state,

during the suspension of detection of the three-dimensional object, the controller uses a detection result obtained by the three-dimensional object detection unit immediately before the lens was assessed to be in the predetermined wetness state.

2. The three-dimensional object detection device according to claim 1, further comprising:

an image conversion unit programmed to perform viewpoint conversion of the images obtained by the camera to bird'"'"'s-eye view images; and

a three-dimensional object assessment unit programmed to assess whether a three-dimensional object detected by the three-dimensional object detection unit is another vehicle;

the three-dimensional object detection unit being further programmed to detect a presence of the three-dimensional object in a detection area set on each of a right side and a left side rearward of the vehicle based on differential waveform information in which the differential waveform information is generated by aligning positions of bird'"'"'s-eye view images obtained at different points in time by the image conversion unit, and counting a number of pixels indicating a predetermined difference in a differential image of an aligned bird'"'"'s-eye view image along a direction in which the three-dimensional object collapses when viewpoint conversion of the bird'"'"'s-eye view images is performed to create a frequency distribution;

the lens cleaning device being provided with a pump for supplying cleaning fluid and a nozzle for spraying the supplied cleaning fluid onto the lens, and the lens cleaning device cleaning the lens of the camera by spraying cleaning fluid onto the lens at a predetermined timing according to a predetermined lens cleaning step; and

the controller is further programmed to use a detection result obtained by the three-dimensional object detection unit or an assessment result obtained by the three-dimensional object assessment unit immediately before the lens state was assessed to be the predetermined wetness state for a predetermined length of time, thereby suppressing detection of the three-dimensional object or assessment of the three-dimensional object as being the other vehicle upon the lens state assessment unit assessing that the lens state is the predetermined wetness state.

3. The three-dimensional object detection device according to claim 1, wherein

the controller is further programmed to use a detection result obtained by the three-dimensional object detection unit immediately before the lens state was assessed as being the predetermined wetness state for a predetermined length of time, upon

the lens state assessment unit assessing that the lens state is the predetermined wetness state, and

the controller identifying a detection state in which the three-dimensional object is being detected by the three-dimensional object detection unit immediately before the lens state is assessed as being the predetermined wetness state.

4. The three-dimensional object detection device according to claim 1, wherein

the lens cleaning device is configured to clean the lens using cleaning fluid according to predetermined lens cleaning steps including a lens wetting step, a first air blowing step, a cleaning step, and a second air blowing step;

the lens state assessment unit is further programmed to assess that the lens state is a middle wetness state in which a middle amount of cleaning fluid adheres to the lens during a period of time from when the first air blowing step is begun to when the first air blowing step is complete; and

the controller is further programmed to use a detection obtained by the three-dimensional object detection unit immediately before the lens state was assessed as being the middle wetness state for a predetermined length of time, upon the lens state being assessed to be the middle wetness state.

5. The three-dimensional object detection device according to claim 4, wherein

the lens state assessment unit is further programmed to identify a number of times air is blown during the first air blowing step;

and

the controller is further programmed to set the predetermined length of time for using the detection results from immediately before the lens state was assessed as being in the middle wetness state to be shorter as the identified number of times that air is blown becomes higher.

6. The three-dimensional object detection device according to claim 1, wherein

the three-dimensional object detection unit detects a three-dimensional object present in the area rearward of the vehicle based on a differential waveform information, and calculates a relative speed of the three-dimensional object based on a change over time in a waveform of a differential waveform information within the predetermined length of time;

the three-dimensional object assessment unit is further programmed to assess that the three-dimensional object is another vehicle, when the relative speed of the three-dimensional object is continuously within a predetermined value range for a predetermined length of time or longer;

the lens cleaning device is configured to clean the lens using cleaning fluid according to predetermined lens cleaning steps including a lens wetting step, a first air blowing step, a cleaning step, and a second air blowing step;

the lens state assessment unit is further programmed to assess that the lens state is a small wetness state in which a small amount of cleaning fluid adheres to the lens from when the air blowing process performed last in the first air blowing step is begun to when the process is ended, and from when the second air blowing step is begun to when the second air blowing step is completed; and

the controller is further programmed to use, for a predetermined length of time, the relative speed of the three-dimensional object detected by the three-dimensional object detecting unit or the assessment result obtained by the three-dimensional object assessment unit immediately before the lens state was assessed as being the small wetness state, upon the lens state being assessed to be the small wetness state.

7. The three-dimensional object detection device according to claim 1, further comprising:

an image conversion unit programmed to perform viewpoint conversion of the images obtained by the camera to bird'"'"'s-eye view images; and

a three-dimensional object assessment unit programmed to assess whether a three-dimensional object detected by the three-dimensional object detection unit is another vehicle;

the three-dimensional object detection unit being further programmed to detect edge information indicating that there is a luminance difference between adjacent image areas of equal to or greater than a predetermined threshold value along a direction of collapse of a three-dimensional object when viewpoint conversion to the bird'"'"'s-eye view images is performed in the bird'"'"'s-eye view images obtained by the image conversion unit, and the three-dimensional object detection unit being programmed to detect a presence of the three-dimensional object a three-dimensional object based on the edge information;

the lens cleaning device being provided with a pump for supplying cleaning fluid and a nozzle for spraying the supplied cleaning fluid onto the lens, the lens cleaning device cleaning the lens of the camera by spraying cleaning fluid onto the lens at a predetermined timing according to a predetermined lens cleaning step; and

the controller is further programmed to use, for a predetermined length of time, a detection result obtained by the three-dimensional object detection unit or an assessment result obtained from three-dimensional object assessment unit immediately before the lens state was assessed to be the predetermined wetness state, thereby suppressing detection of the three-dimensional object or assessment of the three-dimensional object as being the other vehicle upon the lens state assessment unit assessing that the lens state is the predetermined wetness state.

8. The three-dimensional object detection device according to claim 7, wherein

the controller is further programmed to use the detection result from immediately before the lens state was assessed as being the predetermined wetness state for a predetermined length of time, upon

the lens state assessment unit assessing that the lens state is the predetermined wetness state, and

the control identifying a detection state in which the three-dimensional object is being detected by the three-dimensional object detection unit immediately before the lens state is assessed as being the predetermined wetness state.

9. The three-dimensional object detection device according to claim 1, wherein

the lens cleaning device is configured to clean the lens using cleaning fluid according to predetermined lens cleaning steps including a lens wetting step, a first air blowing step, a cleaning step, and a second air blowing step;

the lens state assessment unit is further programmed to assess that the lens state is a large wetness state in which there is a large amount of cleaning fluid adhering to the lens during a period of time from when the lens cleaning steps are begun to when the lens wetting step is completed, or from when the cleaning step is begun to when the cleaning step is finished; and

the controller is further programmed to use a detection result obtained by the three-dimensional object detection unit immediately before the lens state was assessed as being the large wetness state for a predetermined length of time, upon the lens state being assessed to be the large wetness state.

10. The three-dimensional object detection device according to claim 7, wherein

the three-dimensional object detection unit detects a three-dimensional object present in the area rearward of the vehicle based on edge information, and calculates a relative speed of the three-dimensional object based on a change over time in the edge information within the predetermined length of time;

the three-dimensional object assessment unit is further programmed to assess that the three-dimensional object is another vehicle when the relative speed of the three-dimensional object is continuously within a predetermined value range for a predetermined length of time or longer;

the lens cleaning device is configured to clean the lens using cleaning fluid according to predetermined lens cleaning steps including a lens wetting step, a first air blowing step, a cleaning step, and a second air blowing step;

the lens state assessment unit is further programmed to assess that the lens state is a small wetness state subject to control in which a small amount of cleaning fluid adheres to the lens during a period of time from when the air blowing process performed last in the first air blowing step is begun to when the process is ended, and from when the second air blowing step is begun to when the second air blowing step is completed; and

the controller is further programmed to use, for a predetermined length of time, the relative speed of the three-dimensional object detected by the three-dimensional object detection unit or an assessment result obtained by the three-dimensional object assessment unit immediately before the lens state was assessed as being the small wetness state, upon the lens state being assessed to be the small wetness state.

11. A three-dimensional object detection method in which a computing device of a three-dimensional object detection device executes:

a step of detecting a three-dimensional object rearward of a vehicle based on images acquired by a camera that is installed in the vehicle and provided with a lens for forming a video image of an area rearward of the vehicle;

a step of assessing whether the state of the lens is a predetermined wetness state based on a timing at which cleaning fluid is sprayed onto the lens during a lens cleaning step in which the lens of the camera is cleaned by spraying cleaning fluid, the lens cleaning being executed using a lens cleaning device; and

a step of suspending, for a predetermined length of time, the step of detecting a three-dimensional object rearward of the vehicle upon assessing that the lens state is the predetermined wetness state, the suspension being accomplished by using a detection result obtained by the step of detecting a three-dimensional object rearward of the vehicle immediately before the lens was assessed to be in the predetermined wetness state.