Systems and methods for detecting obstructions in a camera field of view

• US 8,553,088 B2
• Filed: 11/23/2006
• Issued: 10/08/2013
• Est. Priority Date: 11/23/2005
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: Mobileye Technologies Limited (Intel Corporation)
• Inventors: Stein, Gideon, Hadassi, Ofer
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US11/562,998
• Publication Number:
  US 20070115357A1
• Time in Patent Office: 2,511 Days
• Field of Search: 348/148, 340/435, 340/438, 382/104, 250/208.1, 352/132, 701/36
• US Class Current: 348/148
• CPC Class Codes: B60Q 1/0023 the element being a sensor, e.g. distance sensor, camera
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 1/20 Fog lights
• CPC Class Codes: B60Q 2300/146 Abnormalities, e.g. fail-safe
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60S 1/023 including defroster or demisting means
• CPC Class Codes: B60S 1/04 Wipers or the like, e.g. scrapers
• CPC Class Codes: B60S 1/0822 characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system mounted on a vehicle for detecting an obstruction on a surface of a window of the vehicle, a primary camera is mounted inside the vehicle behind the window. The primary camera is configured to acquire images of the environment through the window. A secondary camera is focused on an external surface of the window, and operates to image the obstruction. A portion of the window, i.e. window region is subtended respectively by the field of view of the primary camera and the field of view of the secondary camera. A processor processes respective sequences of image data from both the primary camera and the secondary camera.

Claims

1. A system mounted on a vehicle for classifying at least one obstruction on a surface of a window of the vehicle, the system comprising:

a secondary camera adapted for mounting inside the vehicle, wherein said secondary camera is adapted to focus on at least a portion of the surface of the window and acquire a secondary image of the at least one obstruction when present on the surface; and

a processor adapted to process said secondary image and for classification of the at least one obstruction; and

a primary camera adapted for mounting inside the vehicle,

wherein the primary camera is configured to acquire a plurality of primary images of the environment through the window,

wherein a region of the window is subtended by respective fields of view of said secondary camera and said primary camera, and

wherein said processor is configured to process at least one of said primary images and to trigger said classification of the at least one obstruction responsive to deterioration of image quality in at least one of said primary images.

2. The system of claim 1, wherein information is extracted from said at least one primary image to selectably either enhance said classification of said at least one obstruction or eliminating false identification of an obstruction.

3. The system of claim 1, wherein said deterioration of image quality is caused by said at least one obstruction.

4. The system of claim 1, further comprising:

an activation mechanism operatively attached to said processor, wherein said activation mechanism causes cleaning fluid to be sprayed on the window and windshield wipers to be activated when said processor identifies the at least one obstruction as including dirt on an external surface of the window while the external surface is dry.

5. The system of claim 1, further comprising:

an activation mechanism operatively attached to said processor, said activation mechanism activating windshield wipers when said processor identifies the at least one obstruction on an external surface of the window.

6. The system of claim 1, wherein said processor is adapted to initiate a task to remove said at least one obstruction.

7. The system of claim 6, wherein said task is selected from the group of tasks consisting of:

notification of the driver of the vehicle, disabling the system, activating wipers with or without cleaning fluid activating defogging, and turning on fog lights.

8. The system of claim 1, wherein said secondary camera is adapted to also focus on at least a portion of an internal surface of the window and wherein said processor identifies the at least one obstruction when present on said at least a portion of the internal surface.

9. The system of claim 1, wherein said secondary camera is further adapted to perform a task selected from the group of applications consisting of:

ambient light sensing, and gain control sensing.

10. The system of claim 1, further comprising a pulsing light source, wherein said pulsing light source is capable of illuminating said region wherein secondary image acquisition by said secondary camera is synchronized with activating said pulsing light source.

11. A method performable by a system mountable inside a vehicle, the system including a primary camera and a secondary camera both connectible to a processor, the method comprising the steps of:

enabling focusing the secondary camera on a surface of the window, to provide at least one secondary image of said surface;

secondary processing said at least one secondary image thereby identifying when present on said surface at least one obstruction;

enabling acquiring a plurality of primary images of the environment through the window, wherein a region of the window is subtended by respective fields of view of the secondary camera and the primary camera; and

primary processing said at least one primary image thereby detecting at least one obstruction, when present on said surface, said at least one obstruction causing a deterioration of image quality in said at least one primary image, wherein said deterioration of image quality in said at least one primary image triggers said secondary processing of said at least one secondary image and said classifying thereby the at least one obstruction.

12. The method of claim 11, wherein said secondary processing includes edge detection of said at least one obstruction.

13. The method of claim 11, wherein said secondary processing of said at least one secondary image uses information extracted from said at least one primary image to selectably either enhance said identifying said at least one obstruction or eliminating false identification of an obstruction.

14. The method, according to claim 11, further comprising a step of:

upon determining a level of ambient light, setting a mode of the system to selectably either day mode, dusk mode or night mode.

15. The method, according to claim 14, when determining said level of ambient light as being of dusk mode or night mode, turning on the headlights.

16. The method of claim 11, further comprising the step of:

initiating at least one task to remove said at least one obstruction.

17. The method of claim 16, wherein said at least one task is selected from the group of tasks consisting of:

notification of the driver of the vehicle, disabling the system, activating windshield wipers, spraying fluid on windshield and activating defogging of a window.

18. The method of claim 11, wherein said primary processing of said at least one primary image includes at least one other vehicular vision processing step selected from the group consisting of:

detecting lane markings in a road, detecting pedestrians, detecting vehicles, detecting obstacles, detecting road signs, lane keeping, headway keeping and headlights control; and

wherein said at least one other vehicular vision processing step is turned off.

19. The method of claim 11, wherein said identifying of said at least one obstruction includes classifying said at least one obstruction, using a Support Vector Machine (SVM) technique.

20. The method of claim 19, wherein said identifying of said at least one obstruction includes classifying use of support vector machine (SVM) with Radial Basis Function kernel techniques.

21. The method of claim 11, further comprising the step of:

illuminating said window with a pulsing light source, wherein said secondary camera is operated synchronously with said pulsing light source.