Vehicular image sensing system

• US 7,655,894 B2
• Filed: 11/19/2008
• Issued: 02/02/2010
• Est. Priority Date: 03/25/1996
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Vadas, Keith J., Larson, Mark L.
• Primary Examiner(s): Sohn; Seung C
• Application Number:
  US12/273,879
• Publication Number:
  US 20090072124A1
• Time in Patent Office: 440 Days
• Field of Search: 250/208.1, 382/104, 701/200, 701/224, 701/23, 701/27, 701/28, 702/92, 340/901, 348/118
• US Class Current: 250/208.1
• CPC Class Codes: B60Q 1/14 having dimming means
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: G05D 1/0244 using reflecting strips
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

An image sensing system for a vehicle includes an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate. The imaging sensor is disposed at an interior portion of the vehicle, and may be at or proximate to an interior rearview mirror assembly of the vehicle. The system includes a logic and control circuit comprising an image processor for processing image data derived from the imaging sensor. The image sensing system may identify objects of interest based on spectral differentiation or by comparing over successive frames image data associated with objects in the forward field of view of the image sensor or by objects of interest being at least one of qualified and disqualified based at least in part on object motion in the field of view of the imaging sensor.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

wherein said imaging sensor is disposed at an interior portion of the vehicle proximate the windshield of the vehicle and wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through the windshield;

a logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system identifies objects of interest by processing said image data to identify objects of interest based at least on spectral differentiation; and

wherein identification of objects of interest is enhanced by comparing over successive frames image data associated with objects in said forward field of view of said image sensor.

2. The image sensing system of claim 1, wherein said image sensing system processes said image data to identify objects of interest based on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

3. The image sensing system of claim 1, wherein said image processing comprises pattern recognition and wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

4. The image sensing system of claim 1, wherein objects of interest are at least one of qualified and disqualified based at least in part on object motion in said field of view of said imaging sensor.

5. The image sensing system of claim 1, wherein at least one of (a) said array of light sensing photosensor elements and at least a portion of said logic and control circuit are formed on said semiconductor substrate and wherein said portion of said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter, and (b) at least one of (i) said array of light sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on said semiconductor substrate as a CMOS device.

6. The image sensing system of claim 1, wherein at least one of (a) at least a portion of said logic and control circuit is commonly formed with said array of light sensing photosensor elements on said semiconductor substrate as an integrated circuit, (b) said logic and control circuit comprises a logic circuit and at least a portion of said logic circuit comprises a configuration of digital logic elements formed on said semiconductor substrate, and (c) said logic and control circuit comprises a logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

7. The image sensing system of claim 1, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said light sensing photosensor elements to an adjacent one of said light sensing photosensor elements.

8. The image sensing system of claim 1 further comprising a lens imaging light external of the vehicle onto said array of light sensing photosensor elements, and wherein at least one of (a) said lens comprises a molded plastic lens and (b) said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

9. The image sensing system of claim 1, wherein said image sensing system determines a background light level.

10. The image sensing system of claim 1, wherein said logic and control circuit comprises at least one control output and wherein said at least one control output at least one of (a) controls as a function of a speed of the vehicle, (b) controls a headlamp of the vehicle in response to said image processing, and (c) controls a speed of the vehicle in response to said image processing.

11. The image sensing system of claim 1, wherein said image sensing system identifies headlamps of approaching vehicles and taillights of leading vehicles.

12. The image sensing system of claim 1, wherein said image sensing system determines an environment in which the vehicle is being driven, and wherein said image sensing system adjusts control of a headlamp of the vehicle in response to said determination of the environment in which the vehicle is driven.

13. The image sensing system of claim 1, wherein said image sensing system detects lane markers on a road being traveled by the vehicle and present in the field of view of said imaging sensor.

14. The image sensing system of claim 13, wherein said detection of lane markers comprises identification by spectral signature.

15. The image sensing system of claim 1 further comprising a spectral filter for passing visible light that is within a spectral region that generally corresponds with the visible light spectral signature of a taillight of a vehicle, said spectral filter substantially attenuating light having a wavelength greater than about 830 nanometers, said spectral filter disposed in front of some but not all of said photosensor elements.

16. The image sensing system of claim 1, wherein said imaging sensor has a forward field of view to the exterior of the vehicle through an area of the windshield that is swept by a windshield wiper of the vehicle.

17. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

wherein said imaging sensor is disposed at an interior portion of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through an area of the windshield that is swept by a windshield wiper of the vehicle;

a logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said processing of image data comprising, at least in part, a comparison by said logic and control circuit of a frame comprising image data to a successor frame in order to identify, at least in part, an object of interest; and

wherein an object of interest is at least one of qualified and disqualified based at least in part on object motion in said field of view of said imaging sensor.

18. The image sensing system of claim 17, wherein at least one of (a) said logic and control circuit comprises at least one control output and wherein said at least one control output controls as a function of a speed of the vehicle, (b) said logic and control circuit comprises at least one control output and wherein said at least one control output controls a headlamp of the vehicle in response to said image processing, (c) said logic and control circuit comprises at least one control output and wherein said at least one control output controls a speed of the vehicle in response to said image processing, and (d) said image sensing system detects lane markers on a road being traveled by the vehicle and present in the field of view of said imaging sensor.

19. The image sensing system of claim 17, wherein said image sensing system processes said image data to identify objects of interest based on at least one of (i) spatial differentiation, (ii) spectral differentiation, and (iii) pattern recognition.

20. The image sensing system of claim 17, wherein said image processing comprises pattern recognition and wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

21. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

wherein said imaging sensor is disposed at an interior portion of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through the windshield;

a logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system identifies objects of interest by processing said image data to identify objects of interest based on at least one of (i) spatial differentiation, (ii) spectral differentiation, and (iii) pattern recognition;

wherein identification of objects of interest is enhanced by comparing over successive frames image data associated with objects in said forward field of view of said image sensor; and

wherein objects of interest are at least one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

22. The image sensing system of claim 21, wherein at least one of (a) said logic and control circuit comprises at least one control output and wherein said at least one control output controls as a function of a speed of the vehicle, (b) said logic and control circuit comprises at least one control output and wherein said at least one control output controls a headlamp of the vehicle in response to said image processing, (c) said logic and control circuit comprises at least one control output and wherein said at least one control output controls a speed of the vehicle in response to said image processing, and (d) said image sensing system detects lane markers on a road being traveled by the vehicle and present in the field of view of said imaging sensor.

23. The image sensing system of claim 21 further comprising a lens imaging light external of the vehicle onto said array of light sensing photosensor elements, and wherein said lens comprises a molded plastic lens bonded to said imaging sensor.

24. The image sensing system of claim 21, wherein said image processing comprises pattern recognition and wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

25. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

wherein said imaging sensor is disposed at an interior portion of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through an area of the windshield that is swept by a windshield wiper of the vehicle;

a logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system identifies objects of interest by processing said image data to identify objects of interest based at least on spectral differentiation;

a spectral filter disposed in front of some but not all of said photosensor elements, wherein said spectral filter comprises a red filter that passes visible light of wavelength generally in the visible red spectral band and that substantially attenuates light having wavelengths outside the visible red spectral band; and

wherein a comparison is made by said logic and control circuit of a frame comprising image data to a successor frame in order to identify, at least in part, an object of interest.

26. The image sensing system of claim 25, wherein visible light incident at said imaging sensor and not passing through said spectral filter impinges on at least some of said photosensor elements.

27. The image sensing system of claim 26, wherein objects of interest are at least one of qualified and disqualified based at least in part on object motion in said field of view of said imaging sensor.

28. The image sensing system of claim 25, wherein said spectral filter passes visible light that generally corresponds with the visible light spectral signature of a taillight of a vehicle, said spectral filter substantially attenuating light having a wavelength greater than about 830 nanometers.

29. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

wherein said imaging sensor is disposed at an interior portion of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through an area of the windshield that is swept by a windshield wiper;

a logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system processes said image data to identify objects of interest based on at least one of (i) spatial differentiation, (ii) spectral differentiation, and (iii) pattern recognition;

wherein said image sensing system further comprises a lens imaging light external of the vehicle onto said array of light sensing photosensor elements, and wherein said lens is bonded to said imaging sensor; and

wherein objects of interest are at least one of qualified and disqualified based at least in part on object motion in said field of view of said imaging sensor.

30. The image sensing system of claim 29, wherein identification of objects of interest is enhanced by comparing over successive frames image data associated with objects in said forward field of view of said image sensor.

31. The image sensing system of claim 30, wherein said lens comprises a molded plastic lens bonded to said imaging sensor.