Systems and methods for detecting obstructions in a camera field of view

• US 10,632,916 B2
• Filed: 06/13/2018
• Issued: 04/28/2020
• Est. Priority Date: 11/23/2005
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Hadassi, Ofer
• Primary Examiner(s): Philippe, Gims S
• Application Number:
  US16/007,448
• Publication Number:
  US 20180290592A1
• Time in Patent Office: 685 Days
• Field of Search: 348148
• CPC Class Codes: B60Q 1/0023 the element being a sensor, e.g. distance sensor, camera
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 1/20 Fog lights
• CPC Class Codes: B60Q 2300/146 Abnormalities, e.g. fail-safe
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60S 1/023 including defroster or demisting means
• CPC Class Codes: B60S 1/04 Wipers or the like, e.g. scrapers
• CPC Class Codes: B60S 1/0822 characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system mounted on a vehicle for detecting an obstruction on a surface of a window of the vehicle, a primary camera is mounted inside the vehicle behind the window. The primary camera is configured to acquire images of the environment through the window. A secondary camera is focused on an external surface of the window, and operates to image the obstruction. A portion of the window, i.e. window region is subtended respectively by the field of view of the primary camera and the field of view of the secondary camera. A processor processes respective sequences of image data from both the primary camera and the secondary camera.

Claims

1. A system for detecting an obstruction on a window of a vehicle, comprising:

a primary camera configured to acquire a first image through the window of the vehicle, wherein the first image is an image of an environment outside the vehicle;

a secondary camera configured to acquire a second image; and

a processor configured to detect at least one suspected obstruction on a surface of the window based on the second image,

wherein a first region of the window is subtended by a field of view of the primary camera and a second region of the window is subtended by a field of view of the secondary camera, and the first region and the second region of the window overlap at least in part.

2. The system according to claim 1, wherein the primary camera is configured to acquire at least two images, and wherein the processor is further configured to:

detect a deterioration in quality between the at least two images; and

trigger processing of the second image in response to detection of the deterioration in quality between the at least two images.

3. The system according to claim 2, wherein the processor is further configured to determine an image coordinate of the secondary camera associated with the detected deterioration in quality between the at least two images.

4. The system according to claim 2, wherein the processor is further configured to determine whether the deterioration in quality between the at least two images is not related to the surface of the window, and when the processor determines that the deterioration in quality is not related to the surface of the window, the processor does not trigger processing of the second image.

5. The system according to claim 1, wherein responsive to detection of the at least one suspected obstruction on the surface of the window in the second image, the processor is further configured to determine, based on the second image, one or more regions in image space of the primary camera, wherein the one or more regions are affected by the at least one suspected obstruction.

6. The system according to claim 1, wherein the processor is further configured to classify the at least one suspected obstruction based on at least one of an image from the primary camera or an image from the secondary camera.

7. The system according to claim 1, wherein the processor is further configured to:

classify the at least one suspected obstruction based on an image from the primary camera or based on an image from the secondary camera; and

enhance the classification of the at least one suspected obstruction or determine that the detection of the at least one suspected obstruction is false, based on an image from the other one of the primary camera or the secondary camera.

8. The system according to claim 1, wherein the processor is further configured to:

process the second image to determine a presence of high spatial frequencies responsive to an edge in the second image; and

process a corresponding image coordinate of the edge in the first image.

9. The system according to claim 1, wherein the processor is further configured to:

process the second image to determine that there are no high spatial frequencies; and

activate, in response to the determination, a low visibility mode of the primary camera.

10. The system according to claim 8, wherein the processor is further configured to:

determine that there are high spatial frequencies at the corresponding image coordinate in the first image; and

trigger a classification of the at least one suspected obstruction into at least one of;

fog, a crack, a raindrop, thick mud, frost, or smoke.

11. A method for detecting an obstruction on a window of a vehicle, the method comprising:

acquiring, by a primary camera, a first image through the window of the vehicle, wherein the first image is an image of an environment outside the vehicle;

acquiring by a secondary camera a second image;

wherein a first region of the window is subtended by a field of view of the primary camera and a second region of the window is subtended by a field of view of the secondary camera, wherein the first region and the second region of the window overlap at least in part; and

detecting at least one suspected obstruction on a surface of the window based on the second image.

12. The method according to claim 11, further comprising:

acquiring at least two images by the primary camera;

detecting a deterioration in quality between the at least two images; and

triggering processing of the second image in response to detecting the deterioration in quality between the at least two images.

13. The method according to claim 12, further comprising:

determining an image coordinate of the secondary camera associated with the detected deterioration in quality between the at least two images.

14. The method according to claim 12, further comprising:

determining whether the deterioration in quality between the at least two images is not related to the surface of the window; and

after determining that the deterioration in quality is not related to the surface of the window, disabling or avoiding processing of the second image.

15. The method according to claim 11, further comprising:

responsive to detecting the at least one suspected obstruction on the surface of the window in the second image, determining, based on the second image, one or more regions in image space of the primary camera, wherein the one or more regions are affected by the at least one suspected obstruction.

16. The method according to claim 11, further comprising:

classifying the at least one suspected obstruction based on at least one of an image from the primary camera or an image from the secondary camera.

17. The method according to claim 11, further comprising:

classifying the at least one suspected obstruction based on an image from the primary camera or based on an image from the secondary camera;

based on an image from the other one of the primary camera or the secondary camera, enhancing the classification of the at least one suspected obstruction or determining that the detection of the at least one suspected obstruction is false.

18. The method according to claim 11, further comprising:

determining, based on the second image, a presence of high spatial frequencies responsive to an edge imaged in the second image; and

processing a respective image coordinate of the edge in the first image.

19. The method according to claim 11, further comprising:

determining that there are no high spatial frequencies in the second image; and

responsive to the determination, activating a low visibility mode of the primary camera.

20. The method according to claim 18, further comprising:

determining that there are high spatial frequencies in the corresponding image coordinate in the first image; and

triggering a classification of the at least one suspected obstruction in the second image into at least one of;

fog, a crack, a raindrop, thick mud, frost, or smoke.