Camera device, three-dimensional object detection device, and lens cleaning method

• US 9,804,386 B2
• Filed: 07/26/2013
• Issued: 10/31/2017
• Est. Priority Date: 07/27/2012
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Clarion Co., Ltd. (Faurecia SA), Nissan Motor Co., Ltd.
• Inventors: Hayakawa, Yasuhisa, Fukata, Osamu, Takemura, Masayuki, Kiyohara, Masahiro, Muramatsu, Shoji, Irie, Kota
• Primary Examiner(s): Aynalem, Nathnael
• Application Number:
  US14/416,650
• Publication Number:
  US 20150177512A1
• Time in Patent Office: 1,558 Days
• Field of Search: 348148
• CPC Class Codes: B60S 1/0848 Cleaning devices for cameras on vehicle
• CPC Class Codes: B60S 1/56 specially adapted for cleaning other parts or devices than front windows or windscreens rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602
• CPC Class Codes: G02B 27/0006 with means to keep optical surfaces clean, e.g. by preventing or removing dirt, stains, contamination, condensation G02B1/18 takes precedence; cleaning in general B08B
• CPC Class Codes: G03B 17/02 Bodies
• CPC Class Codes: G03B 17/08 Waterproof bodies or housings
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

A camera device includes image capturing device, a lens cleaning device, an adhesion state assessment unit and a controller. The image capturing device is installed on a vehicle and has a lens for forming an image of the vehicle surroundings. The lens cleaning device cleans the lens by spraying a cleaning fluid on the lens in accordance with a predetermined lens cleaning step, in which at least a supply time for supplying cleaning fluid to the lens surface is predetermined. The adhesion state assessment unit is programmed to assess an adhesion state of contamination from a distribution of pixels corresponding to foreign matter adhered to the lens based on a captured image. The controller is programmed to extend the time until a time to start supplying the cleaning fluid in the lens cleaning step as the number of pixels corresponding to foreign matter adhered to the lens increases.

Claims

1. A camera device comprising:

an image capturing device installed on a vehicle and comprising a lens for forming an image of the vehicle surroundings;

a lens cleaning device configured to clean the lens by spraying a cleaning fluid on the lens at a predetermined timing that is set in advance, in which at least a supply time for supplying cleaning fluid to the lens surface is predetermined;

an adhesion state assessment unit programmed to assess an adhesion state of contamination adhered to the lens based on an amount of change of a degree of adhesion by calculating the degree of adhesion of foreign matter adhered to a predetermined area of the lens based on a captured image captured by the image capturing device; and

a lens cleaning controller programmed to extend the time until a time to start supplying the cleaning fluid, the larger the amount of change in the adhesion degree of foreign matter adhered to the lens determined by the adhesion state assessment unit, the longer the lens cleaning controller extends the time until the cleaning fluid is supplied.

2. The camera device as recited in claim 1, wherein

the adhesion state assessment unit is programmed to calculate an adhesion amount of the foreign matter as the adhesion degree, based on pixel values of pixels included in the obtained captured image, and the number of pixels corresponding to foreign matter adhered to the surface of the lens, and to assess a state of adhered lens contamination based on the adhesion degree.

3. The camera device as recited in claim 1, wherein

the adhesion state assessment unit is programmed to calculate a variation in the adhesion amount of foreign matter as a variation in the adhesion degree, based on pixel values of pixels included in the obtained captured image, and a variation in the number of pixels corresponding to foreign matter adhered to the surface of the lens in a predetermined time, and to assess a state of adhered lens contamination based on the variation in the adhesion degree.

4. The camera device as recited in claim 1, wherein

the adhesion state assessment unit is programmed to calculate an adhesion ratio of the foreign matter as the adhesion degree, based on pixel values of the pixels included in the obtained captured image, and the distribution of the pixels corresponding to foreign matter adhered to the surface of the lens, and to assess a state of adhered lens contamination based on the adhesion degree.

5. The camera device as recited in claim 1, wherein

the adhesion state assessment unit is programmed to calculate a variation in the adhesion ratio of foreign matter as a variation in an adhesion degree, based on pixel values of the pixels included in the obtained captured image, and a variation in distribution of the pixels corresponding to foreign matter adhered to the surface of the lens in a predetermined time, and to assess a state of adhered lens contamination based on the variation in the adhesion degree.

6. The camera device as recited in claim 2, wherein

the adhesion state assessment unit is programmed to assess that the state of adhered lens contamination is a predetermined state when the adhesion degree of foreign matter is in a predetermined threshold value of equal to or greater than a first adhesion threshold and less than a second adhesion threshold, which are preset;

and

the lens cleaning controller is programmed to execute a control to extend the time until a time to start supplying the cleaning fluid when the state of adhered lens contamination is assessed as being the predetermined state.

7. The camera device as recited in claim 6, wherein

the first adhesion threshold is an adhesion degree of foreign matter when the vehicle is traveling on a paved road without patterns on the road surface or a paved road without irregularities on the road surface.

8. The camera device as recited in claim 6, wherein

the second adhesion threshold is an adhesion degree of foreign matter when the vehicle is traveling on one of a paved road with patterns on the road surface and a paved road with irregularities formed on the road surface.

9. The camera device as recited in claim 6, wherein

the lens cleaning controller is programmed to cause the lens cleaning device to clean the lens following the predetermined lens cleaning step when the adhesion degree of foreign matter is assessed as being less than the first adhesion threshold or is equal to or greater than the second adhesion threshold, which is greater than the first adhesion threshold, and when the state of adhered lens contamination is assessed as not being the predetermined state.

10. The camera device as recited in claim 6, wherein

the predetermined state is a state in which particles with a small average particle size have adhered to the lens.

11. The camera device as recited in claim 1, further comprising:

a traveling path state assessment unit programmed to assess whether a traveling path on which the vehicle is traveling is one of a paved road and an unpaved road based on traveling information acquired by the vehicle;

wherein

the lens cleaning controller supplies the cleaning fluid to the lens at a time at which the traveling path on which the host vehicle is traveling is assessed as being the paved road by the traveling path state assessment unit.

12. The camera device as recited in claim 11, wherein

the traveling path state assessment unit

acquires each wheel speed of a plurality of wheels provided to the vehicle, and when the variation of the plurality of acquired wheel speeds is less than a predetermined value, the traveling path on which the vehicle is traveling is assessed as being a paved road.

13. A three-dimensional object detection device including the camera device as recited in claim 2, the three-dimensional object detection device comprising:

an image conversion unit programmed to perform viewpoint conversion of captured images obtained by the camera device into bird'"'"'s-eye view images;

a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object, based on distribution information of pixels that is generated in the bird'"'"'s-eye view images obtained by the image conversion unit, when a luminance difference is at a predetermined threshold value or greater along a collapsing direction of a three-dimensional object when the viewpoint is converted to the bird'"'"'s-eye view image;

a three-dimensional object assessment unit programmed to assess whether the three-dimensional object detected by the three-dimensional object detection unit is another vehicle; and

a controller programmed to prevent a detection of the three-dimensional object and the three-dimensional object from being assessed as being another vehicle when the adhesion state assessment unit has assessed that the adhesion degree of foreign matter adhered to the lens is equal to or greater than a predetermined value.

14. A lens cleaning method that is executed by a camera device comprising:

a lens; and

a lens cleaning device for cleaning the lens by supplying cleaning fluid on the lens;

wherein

the camera device executes a step assessing an adhesion state of contamination adhered to the lens based on an amount of change of a degree of adhesion by calculating the degree of adhesion of foreign matter adhered to a predetermined area of the lens based on a captured image; and

extending the time until a time to start supplying the cleaning fluid, the larger the amount of change in the adhesion degree of foreign matter adhered to the lens, the longer the time is extended until the cleaning fluid is supplied.