VEHICULAR IMAGE SENSING SYSTEM

• US 20110285849A1
• Filed: 08/05/2011
• Published: 11/24/2011
• Est. Priority Date: 03/25/1996
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Vadas, Keith J., Larson, Mark L.
• Granted Patent: US 8,222,588 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60Q 1/14 having dimming means
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: G05D 1/0244 using reflecting strips
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An image sensing system for a vehicle includes an imaging sensor and a control. The imaging sensor has a two-dimensional array of light sensing pixels, and has a forward field of view through the windshield of a vehicle equipped with the image sensing system to the exterior of the equipped vehicle. The imaging sensor is operable to capture image data and the control includes an image processor. The image sensing system determines an object of interest present in the forward field of view of the imaging sensor via processing of the captured image data by the image processor. The image processing includes spatial filtering. The spatial filtering may, at least in part, identify atmospheric conditions. The spatial filtering may include analysis of a spectral signature representative of at least one detected light source present in the forward field of view of the imaging sensor.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing pixels;

said imaging sensor having a forward field of view through the windshield of a vehicle equipped with said image sensing system to the exterior of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system determines an object of interest present in said forward field of view of said imaging sensor via processing of said captured image data by said image processor; and

wherein said image processing comprises spatial filtering.

2. The image sensing system of claim 1, wherein said spatial filtering enhances determination of an object of interest present in said forward field of view of said imaging sensor by comparison of data from at least one of (a) adjacent pixels of said imaging sensor and (b) adjacent pixel groups of said imaging sensor.

3. The image sensing system of claim 2, wherein said spatial filtering enhances determination of at least one light source present in said forward field of view of said imaging sensor.

4. The image sensing system of claim 1, wherein said spatial filtering enhances determination of at least one of a headlamp of an approaching vehicle ahead of the equipped vehicle and a taillight of a leading vehicle ahead of the equipped vehicle.

5. The image sensing system of claim 1, wherein said spatial filtering comprises analysis of a spectral signature representative of at least one detected light source present in said forward field of view of said imaging sensor.

6. The image sensing system of claim 1, wherein said spatial filtering evaluates adjacent pixel groups to distinguish more closely proximate detected light sources from more spaced apart detected light sources.

7. The image sensing system of claim 6, wherein said spatial filtering enhances determination of a streetlight by determining the proximity of detected red light and detected white light present in said forward field of view of said imaging sensor.

8. The image sensing system of claim 1, wherein said spatial filtering, at least in part, identifies atmospheric conditions.

9. The image sensing system of claim 8, wherein said spatial filtering, at least in part, identifies at least one of (i) fog, (ii) rain and (iii) snow.

10. The image sensing system of claim 8, wherein said spatial filtering, at least in part, identifies atmospheric conditions by detecting at least one effect on a light source present in said forward field of view of said imaging sensor caused by different types of atmospheric conditions.

11. The image sensing system of claim 8, wherein said spatial filtering, at least in part, identifies fog by detecting a series of transition regions that extend from a detected light source.

12. The image sensing system of claim 8, wherein at least one forward facing light of the equipped vehicle is adjusted at least in part responsive to said identification of atmospheric conditions.

13. The image sensing system of claim 1, wherein said spatial filtering is enhanced by comparing image data over successive frames of said captured image data.

14. The image sensing system of claim 1, wherein said image processing comprises pattern recognition and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic.

15. The image sensing system of claim 1, wherein at least one of (a) said imaging sensor is at or proximate to an interior rearview mirror assembly of the equipped vehicle, and (b) said imaging sensor is at or proximate to the in-cabin surface of the windshield of the equipped vehicle.

16. The image sensing system of claim 1, wherein said image sensing system determines objects of interest based, at least in part, on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

17. The image sensing system of claim 1, wherein detected objects are qualified as objects of interest based, at least in part, on object motion in said field of view of said imaging sensor.

18. The image sensing system of claim 17, wherein detected objects are disqualified based, at least in part, on object motion in said field of view of said imaging sensor.

19. The image sensing system of claim 1, wherein said image sensing system determines an environment in which the equipped vehicle is being driven, and wherein said image sensing system controls a headlamp of the equipped vehicle at least in part responsive to said determination of the environment in which the equipped vehicle is driven.

20. The image sensing system of claim 1, wherein said image sensing system, at least in part, detects lane markers on a road being traveled by the equipped vehicle and present in the field of view of said imaging sensor in order to at least one of (a) assist the driver in steering the equipped vehicle and (b) provide a warning to the driver of the equipped vehicle.

21. The image sensing system of claim 20, wherein said detection of lane markers comprises identification of lane markers by spectral signature.

22. The image sensing system of claim 1, wherein said image sensing system, at least in part, identifies traffic signs.

23. The image sensing system of claim 22, wherein said image sensing system, at least in part, identifies traffic signs by at least one of (a) a spectral signature of the traffic signs and (b) a geometric organization of the traffic signs.

24. The image sensing system of claim 1, wherein said array of light sensing photosensor elements is formed on a semiconductor substrate, and wherein said array of light sensing photosensor elements and associated circuitry are formed on said semiconductor substrate and wherein said associated circuitry comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter, and wherein said array of light sensing photosensor elements and said associated circuitry are formed on said semiconductor substrate as a CMOS device.

25. The image sensing system of claim 1, wherein at least one of (a) at least a portion of said control is commonly formed with said array of light sensing photosensor elements on a semiconductor substrate as an integrated circuit, (b) said control comprises a logic circuit and at least a portion of said logic circuit comprises a configuration of digital logic elements formed on a semiconductor substrate, and (c) said control comprises a logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

26. The image sensing system of claim 1, wherein said control at least one of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, and (d) generates an alert to the driver of the equipped vehicle in response to said image processing.

27. The image sensing system of claim 1, wherein said image sensing system determines headlamps of approaching vehicles and taillights of leading vehicles.

28. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing pixels;

wherein said array of light sensing photosensor elements and associated circuitry are formed on a semiconductor substrate and wherein said associated circuitry comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter, and wherein said array of light sensing photosensor elements and said associated circuitry are formed on said semiconductor substrate as a CMOS device;

said imaging sensor having a forward field of view through the windshield of a vehicle equipped with said image sensing system to the exterior of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system determines an object of interest present in said forward field of view of said imaging sensor via processing of said captured image data by said image processor;

wherein said image processing comprises spatial filtering; and

wherein said spatial filtering, at least in part, identifies atmospheric conditions.

29. The image sensing system of claim 28, wherein said spatial filtering, at least in part, identifies at least one of (i) fog, (ii) rain and (iii) snow.

30. The image sensing system of claim 28, wherein said spatial filtering, at least in part, identifies atmospheric conditions by detecting at least one effect on a light source present in said forward field of view of said imaging sensor caused by different types of atmospheric conditions.

31. The image sensing system of claim 28, wherein at least one forward facing light of the equipped vehicle is adjusted at least in part responsive to said identification of atmospheric conditions.

32. The image sensing system of claim 28, wherein said spatial filtering enhances determination of an object of interest present in said forward field of view of said imaging sensor by comparison of data from at least one of (a) adjacent pixels of said imaging sensor and (b) adjacent pixel groups of said imaging sensor.

33. The image sensing system of claim 28, wherein said spatial filtering enhances determination of at least one light source present in said forward field of view of said imaging sensor.

34. The image sensing system of claim 28, wherein said image processing comprises pattern recognition and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic.

35. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing pixels;

said imaging sensor having a forward field of view through the windshield of a vehicle equipped with said image sensing system to the exterior of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system determines an object of interest present in said forward field of view of said imaging sensor via processing of said captured image data by said image processor;

wherein said image processing comprises spatial filtering;

wherein said spatial filtering enhances determination of at least one of a headlamp of an approaching vehicle ahead of the equipped vehicle and a taillight of a leading vehicle ahead of the equipped vehicle; and

wherein said spatial filtering comprises analysis of a spectral signature representative of at least one detected light source present in said forward field of view of said imaging sensor.

36. The image sensing system of claim 35, wherein said spatial filtering, at least in part, identifies atmospheric conditions, and wherein at least one forward facing light of the equipped vehicle is adjusted at least in part responsive to said identification of atmospheric conditions.

37. The image sensing system of claim 35, wherein said spatial filtering enhances determination of an object of interest present in said forward field of view of said imaging sensor by comparison of data from at least one of (a) adjacent pixels of said imaging sensor and (b) adjacent pixel groups of said imaging sensor.

38. The image sensing system of claim 35, wherein said spatial filtering enhances determination of at least one light source present in said forward field of view of said imaging sensor.