Vehicular image sensing system

• US 7,994,462 B2
• Filed: 12/17/2009
• Issued: 08/09/2011
• Est. Priority Date: 03/25/1996
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Vadas, Keith J., Larson, Mark L., Schofield, Kenneth
• Primary Examiner(s): Sohn; Seung C
• Application Number:
  US12/640,425
• Publication Number:
  US 20100090603A1
• Time in Patent Office: 600 Days
• Field of Search: 250/208.1, 382/104, 701/200, 701/224, 701/23, 701/27, 701/28, 702/92, 340/901, 348/118
• US Class Current: 250/208.1
• CPC Class Codes: B60Q 1/14 having dimming means
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: G05D 1/0244 using reflecting strips
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A vehicular image sensing system includes an imaging sensor having a forward field of view to the exterior of and through the windshield of an equipped vehicle. The imaging sensor is operable to capture image data and the image sensing system includes a control having an image processor. The image sensing system identifies objects of interest in the forward field of view of the image sensor via processing of the captured image data by the image processor. Identification of objects of interest may be enhanced by comparing objects over successive frames of captured image data. Objects of interest may be qualified and/or disqualified based at least in part on object motion in the field of view of the imaging sensor. The image processing may include pattern recognition.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor having a forward field of view to the exterior of a vehicle equipped with said image sensing system and through the windshield of the equipped vehicle;

wherein said imaging sensor is operable to capture frames of image data;

a control comprising an image processor;

wherein said image sensing system identifies objects in said forward field of view of said image sensor via processing of said captured image data by said image processor;

wherein said image processing comprises pattern recognition, and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic; and

wherein said pattern recognition is enhanced by comparing image data of objects over successive frames of said captured image data.

2. The image sensing system of claim 1, wherein said imaging sensor is at or proximate to an interior rearview mirror assembly of the equipped vehicle.

3. The image sensing system of claim 1, wherein said imaging sensor is at or proximate to the in-cabin surface of the windshield of the equipped vehicle.

4. The image sensing system of claim 3, wherein said imaging sensor is incorporated within an interior rearview mirror assembly of the equipped vehicle.

5. The image sensing system of claim 1, wherein said image sensing system identifies objects based, at least in part, on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

6. The image sensing system of claim 1, wherein detected objects are qualified based, at least in part, on object motion in said field of view of said imaging sensor.

7. The image sensing system of claim 6, wherein detected objects are disqualified based, at least in part, on object motion in said field of view of said imaging sensor.

8. The image sensing system of claim 1, wherein said image sensing system determines an activity level around the equipped vehicle.

9. The image sensing system of claim 1, wherein said image sensing system determines an environment in which the equipped vehicle is being driven, and wherein said image sensing system controls a headlamp of the equipped vehicle at least in part responsive to said determination of the environment in which the equipped vehicle is driven.

10. The image sensing system of claim 1, wherein said image sensing system is operable to detect lane markers on a road being traveled by the equipped vehicle and present in the field of view of said imaging sensor in order to at least one of (a) assist the driver in steering the equipped vehicle and (b) provide a warning to the driver of the equipped vehicle.

11. The image sensing system of claim 10, wherein said detection of lane markers comprises identification of lane markers by spectral signature.

12. The image sensing system of claim 1, wherein said image sensing system is operable to identify traffic signs.

13. The image sensing system of claim 12, wherein said image sensing system is operable to identify traffic signs by at least one of (a) a spectral signature of the traffic signs and (b) a geometric organization of the traffic signs.

14. The image sensing system of claim 1, wherein said image sensing system is operable to detect rain.

15. The image sensing system of claim 1, wherein said array of light sensing photosensor elements is formed on a semiconductor substrate.

16. The image sensing system of claim 15, wherein said array of light sensing photosensor elements and associated circuitry are formed on said semiconductor substrate and wherein said associated circuitry comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

17. The image sensing system of claim 16, wherein said array of light sensing photosensor elements and said associated circuitry are formed on said semiconductor substrate as a CMOS device.

18. The image sensing system of claim 1, wherein at least one of (a) at least a portion of said control is commonly formed with said array of light sensing photosensor elements on a semiconductor substrate as an integrated circuit, (b) said control comprises a logic circuit and at least a portion of said logic circuit comprises a configuration of digital logic elements formed on a semiconductor substrate, and (c) said control comprises a logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

19. The image sensing system of claim 1, wherein said control at least one of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, and (d) generates an alert to the driver of the equipped vehicle in response to said image processing.

20. The image sensing system of claim 1, wherein said image sensing system identifies headlamps of approaching vehicles and taillights of leading vehicles.

21. The image sensing system of claim 1, wherein said pattern recognition comprises detection of (a) a headlight and (b) a taillight.

22. The image sensing system of claim 1, wherein said pattern recognition comprises detection of taillights of a leading vehicle by detection of a triad pattern of lights in said forward field of view.

23. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

wherein said imaging sensor is at or proximate to the in-cabin surface of the windshield of a vehicle equipped with said image sensing system, and wherein said imaging sensor has a forward field of view to the exterior of the equipped vehicle through the windshield of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system identifies objects in said forward field of view of said image sensor via processing of said captured image data by said image processor;

wherein said image processing comprises pattern recognition, and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic; and

wherein objects are at least one of (a) qualified and (b) disqualified based, at least in part, on object motion in said field of view of said imaging sensor.

24. The image sensing system of claim 23, wherein said imaging sensor is at or proximate to an interior rearview mirror assembly of the equipped vehicle.

25. The image sensing system of claim 24, wherein said imaging sensor is incorporated within an interior rearview mirror assembly of the equipped vehicle.

26. The image sensing system of claim 23, wherein said image sensing system processes said image data to identify objects based, at least in part, on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

27. The image sensing system of claim 23, wherein said imaging sensor is operable to capture frames of image data and wherein identification of objects is enhanced by comparing image data of objects over successive frames of said captured image data.

28. The image sensing system of claim 23, wherein said pattern recognition comprises detection of (a) a headlight and (b) a taillight.

29. The image sensing system of claim 23, wherein said pattern recognition comprises detection of taillights of a leading vehicle by detection of a triad pattern of lights in said forward field of view.

30. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor having a forward field of view to the exterior of a vehicle equipped with said image sensing system and through the windshield of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system identifies objects in said forward field of view of said image sensor via processing of said captured image data by said image processor;

wherein said image processing comprises pattern recognition, and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic; and

wherein said control at least one of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, and (d) generates an alert to the driver of the equipped vehicle in response to said image processing.

31. The image sensing system of claim 30, wherein said imaging sensor is operable to capture frames of image data and wherein identification of objects is enhanced by comparing image data of objects over successive frames of said captured image data.

32. The image sensing system of claim 30, wherein said pattern recognition comprises detection of (a) a headlight and (b) a taillight.

33. The image sensing system of claim 30, wherein said pattern recognition comprises detection of taillights of a leading vehicle by detection of a triad pattern of lights in said forward field of view.

34. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor having a forward field of view to the exterior of a vehicle equipped with said image sensing system and through the windshield of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system identifies objects in said forward field of view of said image sensor via processing of said captured image data by said image processor;

wherein said image processing comprises pattern recognition, and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic; and

wherein said image sensing system is operable to at least one of (a) detect lane markers on a road being traveled by the equipped vehicle and present in the field of view of said imaging sensor in order to at least one of (i) assist the driver in steering the equipped vehicle and (ii) provide a warning to the driver of the equipped vehicle, and (b) identify traffic signs.

35. The image sensing system of claim 34, wherein said image sensing system is operable to detect lane markers on a road being traveled by the equipped vehicle and present in the field of view of said imaging sensor in order to at least one of (i) assist the driver in steering the equipped vehicle and (ii) provide a warning to the driver of the equipped vehicle, and wherein said detection of lane markers comprises identification of lane markers by spectral signature.

36. The image sensing system of claim 34, wherein said image sensing system is operable to identify traffic signs, and wherein said image sensing system identifies traffic signs by at least one of (a) a spectral signature of the traffic signs and (b) a geometric organization of the traffic signs.

37. The image sensing system of claim 34, wherein said imaging sensor is operable to capture frames of image data and wherein identification of objects is enhanced by comparing image data of objects over successive frames of said captured image data.

38. The image sensing system of claim 34, wherein said control at least one of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, and (d) generates an alert to the driver of the equipped vehicle in response to said image processing.

39. The image sensing system of claim 34, wherein said image sensing system identifies headlamps of approaching vehicles and taillights of leading vehicles, and wherein said control controls a headlamp of the equipped vehicle in response to identification of headlamps of approaching vehicles and taillights of leading vehicles.

40. The image sensing system of claim 39, wherein objects are at least one of (a) qualified and (b) disqualified based, at least in part, on object motion in said field of view of said imaging sensor.

41. The image sensing system of claim 34, wherein said pattern recognition comprises detection of (a) a headlight and (b) a taillight.

42. The image sensing system of claim 34, wherein said pattern recognition comprises detection of taillights of a leading vehicle by detection of a triad pattern of lights in said forward field of view.

43. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor having a forward field of view to the exterior of a vehicle equipped with said image sensing system and through the windshield of the equipped vehicle;

wherein said imaging sensor is operable to capture image data;

a control comprising an image processor;

wherein said image sensing system identifies objects in said forward field of view of said image sensor via processing of said captured image data by said image processor;

wherein said image processing comprises pattern recognition, and wherein said pattern recognition comprises detection of at least one of (a) a headlight, (b) a taillight and (c) an object, and wherein said pattern recognition is based at least in part on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic;

wherein said image sensing system identifies objects based, at least in part, on at least one of (i) spatial differentiation, (ii) pattern recognition, (iii) shape and (iv) spectral characteristic; and

wherein said control at least one of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, (d) generates an alert to the driver of the equipped vehicle in response to said image processing, (e) detects lane markers on a road being traveled by the equipped vehicle in response to said image processing, and (f) identifies traffic signs in response to said image processing.

44. The image sensing system of claim 43, wherein said imaging sensor is operable to capture frames of image data and wherein identification of objects is enhanced by comparing image data of objects over successive frames of said captured image data.

45. The image sensing system of claim 43, wherein objects are at least one of (a) qualified and (b) disqualified based, at least in part, on object motion in said field of view of said imaging sensor.

46. The image sensing system of claim 43, wherein said control detects lane markers on a road being traveled by the equipped vehicle, and wherein said image sensing system, responsive to said image processing, is operable to at least one of (i) assist the driver in steering the equipped vehicle and (ii) provide a warning to the driver of the equipped vehicle.

47. The image sensing system of claim 43, wherein said control identifies headlamps of approaching vehicles and taillights of leading vehicles, and wherein said control controls a headlamp of the equipped vehicle in response to identification of headlamps of approaching vehicles and taillights of leading vehicles.

48. The image sensing system of claim 43, wherein said control at least two of (a) controls a headlamp of the equipped vehicle as a function of a speed of the equipped vehicle, (b) controls a headlamp of the equipped vehicle in response to said image processing, (c) controls a speed of the equipped vehicle in response to said image processing, (d) generates an alert to the driver of the equipped vehicle in response to said image processing, (e) detects lane markers on a road being traveled by the equipped vehicle in response to said image processing, (f) identifies traffic signs in response to said image processing.

49. The image sensing system of claim 43, wherein said pattern recognition comprises detection of (a) a headlight and (b) a taillight.

50. The image sensing system of claim 43, wherein said pattern recognition comprises detection of taillights of a leading vehicle by detection of a triad pattern of lights in said forward field of view.