Method, system, imaging device, movable device and program product for detecting static elements in video and image sources

• US 9,454,703 B2
• Filed: 01/27/2015
• Issued: 09/27/2016
• Est. Priority Date: 01/28/2014
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Deigmller, Jrg, Einecke, Nils
• Primary Examiner(s): Strege, John
• Application Number:
  US14/606,540
• Publication Number:
  US 20150213318A1
• Time in Patent Office: 609 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: B60S 1/0818 including control systems responsive to external conditions, e.g. by detection of moisture, dirt or the like
• CPC Class Codes: B60S 1/56 specially adapted for cleaning other parts or devices than front windows or windscreens rear-view mirror arrangements mounted on vehicle exterior including cleaning devices B60R1/0602
• CPC Class Codes: G01N 2021/945 Liquid or solid deposits of macroscopic size on surfaces, e.g. drops, films, or clustered contaminants
• CPC Class Codes: G01N 21/94 Investigating contamination, e.g. dust G01N21/85 takes precedence
• CPC Class Codes: G01N 21/958 Inspecting transparent materials or objects, e.g. windscreens for conveyed flat sheet or rod G01N21/896
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor

Abstract

The invention relates to a method for determining a static element in images captured by an imaging means mounted on a movable device. The method comprises steps of acquiring a first image and a second image captured by the imaging means. The first image and the second image are captured at capture times separated by a time difference and the method is characterized by the time difference being selected depending on motion parameters of the movable device. A determination measure for corresponding regions of the first and the second image for representing a similarity of the corresponding regions is calculated and a static element of the first and the second image is determined based on the calculated determination measure. An output signal comprising information on the determined static element is generated.

Claims

1. Method for determining a static element in images captured by an imaging means mounted on a movable device, the method comprising steps of:

acquiring a first image and a second image captured by the imaging means, wherein the first image and the second image are captured at capture times separated by a time difference,

calculating a determination measure for corresponding regions of the first image and the second image for representing a similarity of the corresponding regions,

determining a static element of the first image and the second image based on the calculated determination measure,

generating an output signal comprising information on the determined static element, and wherein

the capture times of the first image and the second image that are acquired depends on at least one motion parameter of the movable device.

2. The method according to claim 1, wherein

the time difference Δ

t is dependent on at least one motion parameter of the movable device.

3. The method according to claim 1, wherein

the determination measure is calculated on the corresponding regions each of the first image and the second image for each pixel.

4. The method according to claim 1, wherein

the determination measure is calculated as a normalized determination measure.

5. The method according to claim 1, wherein

the determination measure is one of a normalized cross correlation, a summed normalized cross correlation, summed or squared difference measure for rank transformed images and a summed hamming distance of census transformed images.

6. The method according to claim 4, wherein

for determining the static element an element in the first image and the second image is determined for which the calculated determination measure exceeds a threshold.

7. The method according to claim 1, wherein

the determination measure is based on calculated optical flows between the first image and the second image.

8. The method according to claim 7, wherein

for determining the static element an element in the first image and the second image is determined for which the optical flow is smaller than a second threshold or for which the optical flow is zero.

9. The method according to claim 1, wherein

the imaging means is configured to capture a sequence of images.

10. The method according to claim 9, wherein

the first image and the second image are selected from the sequence of images generated by the imaging device with a frame rate, and

that the frame rate differs from the time difference between the first image and the at least one second image.

11. The method according to claim 9, wherein

plural determination measures are calculated for a plurality of pairs of images of the sequence of images.

12. The method according to claim 11, wherein

an average determination measure for determining the static element is calculated from the calculated plural determination measures.

13. The method according to claim 1, wherein

the motion parameters of the imaging means correspond to drive parameters of the movable device, the drive parameters comprising at least one of a speed and a turning angle of the movable device.

14. The method according to claim 1, wherein

the first image and the second image are selected having the capturing times during a turning motion of the imaging means or of the movable device on which the imaging means is mounted.

15. The method according to claim 1, wherein

the method for determining a static element is started when a turning movement of the imaging means or of the movable device is determined.

16. The method according to claim 1, wherein

the method comprises a step of determining underexposed or overexposed areas in the first image and the second image,

wherein the underexposed or overexposed areas in the first image and the second image are excluded from determining static elements.

17. The method according to claim 1, wherein

when a number of determined static elements exceeds a third threshold, the output signal is generated.

18. The method according to claim 17, wherein

the output signal initiates performing a lens-cleaning procedure or reporting to a user of a determined imaging means state or stopping moving the movable device on which the imaging means is mounted or moving the movable device on which the imaging means is mounted to a cleaning position.

19. An image processing system configured to perform the method according to claim 1.

20. An imaging device comprising the image processing system according to claim 19.

21. A movable device equipped with an imaging device according to claim 20, wherein

the movable device is one of an automobile device, an autonomous movable device, an autonomous lawn mower, a land vehicle, a car, a motorcycle, an aerial vehicle, a sea vehicle, an orbital vehicle, a spacecraft.

22. A computer program product, the computer program product comprising a non-transitory computer readable medium for performing the method according to claim 1 when executed on a computer or signal processor.