Stain and trash detection systems and methods

• US 10,509,974 B2
• Filed: 04/21/2017
• Issued: 12/17/2019
• Est. Priority Date: 04/21/2017
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Tokatyan, Shant
• Primary Examiner(s): Vazquez Colon, Maria E
• Application Number:
  US15/494,158
• Publication Number:
  US 20180307926A1
• Time in Patent Office: 970 Days
• Field of Search: None
• CPC Class Codes: B60S 1/64 for cleaning vehicle interiors, e.g. built-in vacuum cleaners
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions

Abstract

Example stain and trash detection systems and methods are described. In one implementation, a method receives a clean image of a vehicle interior and a second image of the vehicle interior after a passenger has occupied the vehicle. The method quantizes the clean image and the second image to reduce noise and identifies differences between the clean image and the second image. Additionally, the method identifies at least one contour in the identified differences and classifies an area defined by the contour as one of a stain, dirt, trash, or another item.

Claims

1. A method comprising:

receiving a clean image of a vehicle interior of a vehicle;

receiving a second image of the vehicle interior after a passenger has occupied the vehicle;

quantizing the clean image and the second image to reduce noise by reducing pixels of the clean image and the second image to three levels including one black level, one gray level, and one white level according to intensity of the pixels to obtain a quantized clean image and a quantized second image;

identifying differences between the quantized clean image and the quantized second image;

identifying a contour in the identified differences; and

classifying an area defined by the contour as one of a stain, dirt, trash, or another item;

wherein the vehicle is an autonomous vehicle;

wherein the intensities of the pixels of the clean image and the second image have a first range of values; and

wherein quantizing the clean image and the second image to reduce noise comprises;

replacing pixels in the clean image and the second image having intensities in a second range of values with the one white level;

replacing pixels in the clean image and the second image having intensities in a third range of values with the one gray level; and

replacing pixels in the clean image and the second image having intensities in a fourth range of values with the one black level; and

wherein the second range of values is higher than the third range of values and the third range of values is higher than the fourth range of values.

2. The method of claim 1, further comprising associating a bounding box with the contour.

3. The method of claim 1 wherein identifying differences between the quantized clean image and the quantized second image includes subtracting the quantized second image from the quantized clean image.

4. The method of claim 1, wherein classifying the area defined by the contour includes determining a shape of the contour.

5. The method of claim 1, wherein classifying the area defined by the contour includes determining a smoothness of edges of the contour.

6. The method of claim 1, further comprising determining whether the vehicle interior needs to be cleaned based on the classification of the area defined by the contour.

7. The method of claim 6, further comprising taking the vehicle out of service if the vehicle interior needs to be cleaned.

8. The method of claim 6, further comprising, responsive to determining that the vehicle interior needs to be cleaned, arranging cleaning of the vehicle interior.

9. The method of claim 6, further comprising awaiting instructions to pick up a new passenger if the vehicle interior does not need to be cleaned.

10. An apparatus comprising:

a communication manager configured to receive a clean image of a vehicle interior of a vehicle and configured to receive a second image of the vehicle interior after a passenger has occupied the vehicle;

an image quantization module configured to quantize the clean image and the second image by reducing pixels of the clean image and the second image to three levels including one black level, one gray level, and one white level according to intensity of the pixels to obtained a quantized clean image and a quantized second image;

an image processing module configured to identify differences between the quantized clean image and the quantized second image;

a contour identification module configured to identify a contour in the identified differences; and

a classification module configured to classify an area defined by the contour as one of a stain, dirt, trash, or another item

wherein the image quantization module is configured to quantize the clean image and the second image by;

replacing pixels in the clean image and the second image having intensities in a first range of values with the one white level;

replacing pixels in the clean image and the second image having intensities in a second range of values with the one gray level; and

replacing pixels in the clean image and the second image having intensities in a third range of values with the one black level; and

wherein the first range of values is higher than the second range of values and the second range of values is higher than the third range of values.

11. The apparatus of claim 10, further comprising a vehicle maintenance manager configured to determine whether the vehicle interior needs to be cleaned based on the classification of the area defined by the contour.

12. The apparatus of claim 10, wherein the apparatus is coupled to an automated driving system in the vehicle.