VEHICLE SENSOR CLEANING

• US 20200094784A1
• Filed: 09/20/2018
• Published: 03/26/2020
• Est. Priority Date: 09/20/2018
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Herman, David Michael, Patil, Sunil, Krishnan, Venkatesh
• Granted Patent: US 10,829,091 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: B60S 1/0848 Cleaning devices for cameras on vehicle
• CPC Class Codes: B60S 1/56 specially adapted for cleaning other parts or devices than front windows or windscreens rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602
• CPC Class Codes: G02B 27/0006 with means to keep optical surfaces clean, e.g. by preventing or removing dirt, stains, contamination, condensation G02B1/18 takes precedence; cleaning in general B08B
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

A system includes a processor and a memory. The memory stores instructions executable by the processor to detect an occlusion on a surface in a vehicle sensor optical path based on a segmentation of sensor image data, and to select a cleaning plan for the surface based on the detected occlusion, map data, and vehicle route data.

Claims

1. A system, comprising a processor and a memory, the memory storing instructions executable by the processor to:

detect an occlusion on a surface in a vehicle sensor optical path based on a segmentation of sensor image data; and

select a cleaning plan for the surface based on the detected occlusion, map data, and vehicle route data.

2. The system of claim 1, wherein the instructions further include instructions to perform the segmentation of the sensor image data based on output of a neural network trained to detect a plurality of feature classes including at least an occlusion class.

3. The system of claim 2, wherein the plurality of feature classes further includes at least one of a road, a pedestrian, a car, a building, a vegetation, and a sky class.

4. The system of claim 1, wherein the surface includes at least one of a lens, a transparent cover, and a part of a vehicle windshield.

5. The system of claim 1, wherein the instructions further include instructions to:

determine a predicted segmentation class of a location including one or more points on an exterior of the surface based on the sensor field of view, the vehicle route data, and the map data; and

determine a priority of the point based at least in part on the predicted segmentation class.

6. The system of claim 1, wherein the selected cleaning plan further includes a time of actuation of a vehicle cleaning actuator and a predicted duration of cleaning.

7. The system of claim 6, wherein the instructions further include instructions to:

determine a location of the occlusion on an exterior of the surface;

predict a change of priority of the location on the exterior surface based on the vehicle route; and

plan the time of cleaning based on the predicted change of the priority and a vehicle speed.

8. The system of claim 7, wherein the instructions further include instructions to select a priority threshold for cleaning based on the vehicle speed;

and

upon determining that the priority of the location on an exterior of the sensor exceeds the selected priority threshold, actuate a vehicle cleaning actuator to clean the exterior of the surface.

9. The system of claim 1, wherein the instructions further include instructions to:

determine a cleaning score based on a priority of a location of the occlusion, a vehicle speed, a number of occluded pixels in the image data, and a predicted duration of a cleaning; and

select the cleaning plan based on the determined score.

10. The system of claim 1, wherein the instructions further include instructions to perform the selected cleaning plan.

11. The system of claim 10, wherein the selected cleaning plan further includes at least one of a time of actuating a cleaning actuator, a duration of actuation, and a mode of operating the cleaning actuator.

12. The system of claim 11, wherein the instructions to perform the selected cleaning plan further include instructions to activate the cleaning actuator at the time of actuating the cleaning actuator for the duration of actuation.

13. The system of claim 11, wherein the instructions further include instructions to:

determine normalized scores from start to end of a vehicle route;

determine a time of maximum normalized score; and

determine the time of actuating the cleaning actuator based on the time of a maximum normalized score.

14. A system, comprising:

a vehicle camera sensor having an optical path; and

a processor programmed to detect an occlusion on a surface in the optical path of the vehicle camera sensor based on a segmentation of sensor image data, and to select a cleaning plan for the surface based on the detected occlusion, map data, and vehicle route data.

15. The system of claim 14, wherein the processor is further programmed to perform the segmentation of the sensor image data based on output of a neural network trained to detect a plurality of feature classes including at least an occlusion class.

16. The system of claim 14, wherein the processor is further programmed to:

determine a predicted segmentation class of a location including one or more points on an exterior of the surface based on the sensor field of view, the vehicle route data, and the map data; and

determine a priority of the point based at least in part on the predicted segmentation class.

17. A method, comprising:

detecting an occlusion on a surface in the optical path of a vehicle camera sensor based on a segmentation of sensor image data, and

selecting a cleaning plan for the surface based on the detected occlusion, map data, and vehicle route data.

18. The method of claim 17, further comprising performing the segmentation of the sensor image data based on output of a neural network trained to detect a plurality of feature classes including at least an occlusion class.

19. The method of claim 17, further comprising:

determining a predicted segmentation class of a location including one or more points on an exterior of the surface based on the sensor field of view, the vehicle route data, and the map data; and

determining a priority of the point based at least in part on the predicted segmentation class.

20. The method of claim 17, further comprising performing the selected cleaning plan.