Image sensing system for controlling an accessory or headlight of a vehicle

• US 7,388,182 B2
• Filed: 01/09/2007
• Issued: 06/17/2008
• Est. Priority Date: 02/26/1993
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Larson, Mark L, Vadas, Keith J
• Primary Examiner(s): LE, QUE TAN
• Application Number:
  US11/651,289
• Publication Number:
  US 20070176080A1
• Time in Patent Office: 525 Days
• Field of Search: 250/205, 250/208.1, 250/214.R, 250/559.4, 250/221, 250/214.AL, 250/226, 359601-610, 359264-272, 180167-169, 340/426.23, 340/436
• US Class Current: 250/205
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 2300/054 Variable non-standard intensity, i.e. emission of various beam intensities different from standard intensities, e.g. continuous or stepped transitions of intensity
• CPC Class Codes: B60Q 2300/056 Special anti-blinding beams, e.g. a standard beam is chopped or moved in order not to blind
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 2001/1223 with sensors or transducers
• CPC Class Codes: B60S 1/0822 characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: B60S 1/087 including an ambient light sensor control of vehicle lights B60Q
• CPC Class Codes: G01J 1/0488 with spectral filtering
• CPC Class Codes: G01J 1/4228 arrangements with two or more detectors, e.g. for sensitivity compensation
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/51 using colour filters
• CPC Class Codes: G01J 3/513 having fixed filter-detector pairs
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

An image sensing system for a vehicle includes an imaging sensor comprising a two-dimensional array of light sensing photosensor elements, preferably formed on a semiconductor substrate, and a logic and control circuit comprising an image processor for processing image data derived from the imaging sensor. The logic and control circuit generates at least one control output for controlling at least one light of the vehicle. The imaging sensor is disposed at an interior portion of the vehicle proximate the windshield of the vehicle and has a forward field of view to the exterior of the vehicle through an area of the windshield. The at least one control output preferably controls at least one of (i) a beam state of a headlight; (ii) a beam aim of a headlight; (iii) a beam pattern of a headlight and (iv) a beam intensity of a headlight.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said image processing comprising at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition;

said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle;

said imaging sensor having a forward field of view to the exterior of the vehicle through an area of the windshield;

wherein said at least one light of the vehicle comprises a headlight of the vehicle; and

wherein said at least one control output controls at least one of (i) a beam state of said headlight, (ii) a beam aim of said headlight, (iii) a beam pattern of said headlight, and (iv) a beam intensity of said headlight.

2. The image sensing system of claim 1, wherein said array of light sensing photosensor elements comprises at least a first array portion and a second array portion, and wherein a first spectral filter is provided for passing visible light that is within a first spectral region that generally corresponds with the visible light spectral signature of a taillight of a vehicle, and wherein light from external the vehicle that passes via said first spectral filter onto said first array portion of said array does not pass via said first filter onto said second array portion, and wherein at least one of (a) said processing of image data by said image processor comprises comparing the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter to the output of at least one light sensing element of said second array portion sensing light not passing via said first spectrat filter, and (b) a control signal is provided for controlling said at least one light as a function of the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter and the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter.

3. The image sensing system of claim 1, wherein at least one of (a) at least one of said array of light sensing photo sensor elements and at least a portion of said logic and control circuit is formed on said semiconductor substrate as at least one of (i) a CMOS device, (ii) an integrated circuit, and (iii) a logic and control circuit comprising at least one of an analog-to-digital converter, a logic circuit, a clock, random access memory and a digital-to-analog converter, (b) said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory, and (c) said image sensing system further comprises a lens imaging light external of the vehicle onto said array of light sensing photo sensor elements, and wherein said lens comprises a molded plastic lens, and wherein said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

4. The image sensing system of claim 1, wherein at least one of (a) said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle, and wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle, and wherein said windshield area comprises a windshield area that is swept by the windshield wipers, and (b) said at least one control output for controlling at least one light of the vehicle comprises a pulse-width-modulated signal, and wherein said at least one control output controls a beam state of said headlight and wherein said image sensing system controls switching the state of said headlight between a higher beam headlight mode and a lower beam headlight mode, and wherein said image sensing system controls switching to at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode.

5. The image sensing system of claim 1, wherein at least one of (a) said at least one light of the vehicle comprises a headlight having various modes, and wherein said image sensing system controls switching between said various modes, (b) said at least one control output controls a beam state of said headlight and wherein switching between one state to another is provided, (c) said at least one control output controls a beam state of said headlight as a function of a speed of the vehicle, (d) said at least one control output controls a beam aim of said headlight and wherein a continuous variation of the beam aim is provided, (e) said at least one control output controls at least one of (i) a beam aim of said headlight, (ii) a beam pattern of said headlight and (iii) a beam intensity of said headlight, and (f) said image sensing system detects lane markers on a road being traveled by the vehicle and present in said field of view of said imaging sensor, and wherein said detection of lane markers comprises identification by spectral signature.

6. The image sensing system of claim 1, wherein at least one of (a) said image sensing system determines an appropriate zone of interest for the lighting pattern of a light of the vehicle, (b) said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle, and wherein said image sensing system controls a headlighting system of the vehicle in accordance with said determination of a lighting configuration for the vehicle, (c) said image sensing system discriminates between oncoming headlights and leading taillights by taking into account the relative location of sources of light within the scene forward of the vehicle, and (d) said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic, and wherein object recognition is enhanced by comparing identified objects over successive frames, and wherein objects of interest are one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

7. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photo sensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle;

said imaging sensor having a forward field of view to the exterior of the vehicle through an area of the windshield;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein said at least one control output controls at least one of (i) a beam state of said headlight, (ii) a beam aim of said headlight, (iii) a beam pattern of said headlight, and (iv) a beam intensity of said headlight;

wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle; and

wherein said image sensing system controls a headlighting system of the vehicle in accordance with said determination of said lighting configuration for the vehicle.

8. The image sensing system of claim 7, wherein said array of light sensing photosensor elements comprises at least a first array portion and a second array portion, and wherein a first spectral filter is provided for passing visible light that is within a first spectral region that generally corresponds with the visible light spectral signature of a taillight of a vehicle, and wherein light from external the vehicle that passes via said first spectral filter onto said first array portion of said array does not pass via said first filter onto said second array portion, and wherein at least one of(a) said processing of image data by said image processor comprises comparing the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter to the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter, and (b) a control signal is provided for controlling said light as a function of the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter and the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter.

9. The image sensing system of claim 7, wherein said imaging sensor comprising a two-dimensional array of light sensing elements is formed on a semiconductor substrate, and wherein said at least one control output comprises a pulse-width-modulated control signal, and wherein at least one of said array of sensing elements and at least a portion of said logic and control circuit is formed on a semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory and (v) a digital-to-analog converter, and wherein said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory, and wherein said logic and control circuit determines a background light level.

10. The image sensing system of claim 7, wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of a light of the vehicle, and wherein said image processing comprises at least one of (i) spatial differentiation, (ii) spectral signature recognition and (iii) pattern recognition, and wherein discrimination between oncoming headlights and leading taillights is accomplished by taking into account the relative location of sources of light within the scene forward of the vehicle, and wherein said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic, and wherein object recognition is enhanced by comparing identified objects over successive frames, and wherein objects of interest are one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

11. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle;

said imaging sensor having a forward field of view to the exterior of the vehicle through an area of the windshield;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein said at least one control output controls at least one of (i) a beam state of said headlight, (ii) a beam aim of said headlight, (iii) a beam pattern of said headlight, and (iv) a beam intensity of said headlight;

wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle; and

wherein said image sensing system controls a headlighting system of the vehicle in accordance with said determination of said lighting configuration for the vehicle;

wherein said array of light sensing photo sensor elements comprises at least a first array portion and a second array portion;

wherein a first spectral filter is provided for passing visible light that is within a first spectral region, and wherein light from external the vehicle that passes via said first spectral filter onto said first array portion of said array does not pass via said first filter onto said second array portion; and

wherein said processing of image data by said image processor comprises comparing the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter to the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter.

12. The image sensing system of claim 11, wherein said first spectral region generally corresponds with the visible light spectral signature of a taillight of a vehicle, and wherein said image sensing system further comprises providing a control signal for controlling said light as a function of the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter and the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter, and wherein at least one of said array of sensing elements and at least a portion of said logic and control circuit is formed on a common semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit and (vii) a read-only-memory, and wherein said logic and control circuit determines a background light level.

13. The image sensing system of claim 11, wherein said at least one control output controls at least one of (i) a beam aim of said headlight, (ii) a beam pattern of said headlight, (iii) a beam intensity of said headlight, (iv) a beam aim of said headlight wherein a continuous variation of the beam aim is provided, (v) a beam state of said headlight wherein switching between one state to another is provided, and (vi) a beam state of said headlight as a function of a speed of the vehicle, and wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of a light of the vehicle, and wherein said image processing comprises at least one of (i) spatial differentiation, (ii) spectral signature recognition and (iii) pattern recognition, and wherein discrimination between oncoming headlights and leading taillights is accomplished by taking into account the relative location of sources of light within the scene forward of the vehicle, and wherein said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic, and wherein object recognition is enhanced by at least one of (a) comparing identified objects over successive frames and (b) one of qualifying and disqualifying objects of interest based on object motion in said field of view of said imaging sensor.

14. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle;

said imaging sensor having a forward field of view to the exterior of the vehicle through an area of the windshield;

wherein said windshield area comprises a windshield area that is swept by the windshield wipers;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle; and

wherein said image sensing system controls a headlighting system of the vehicle in accordance with said determination of said lighting configuration for the vehicle.

15. The image sensing system of claim 14, wherein said array of light sensing photosensor elements comprises at least a first array portion and a second array portion, and wherein a first spectral filter is provided for passing visible light that is within a first spectral region that generally corresponds with the visible light spectral signature of a taillight of a vehicle and wherein light from external the vehicle that passes via said first spectral filter onto said first array portion of said array does not pass via said first filter onto said second array portion, and wherein at least one of (a) said processing of image data by said image processor comprises comparing the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter to the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter, and (b) a control signal is provided for controlling said light as a function of the output of at least one light sensing element of said first array portion sensing light passing via said first spectral filter and the output of at least one light sensing element of said second array portion sensing light not passing via said first spectral filter.

16. The image sensing system of claim 14, wherein at least one of said array of sensing elements and at least a portion of said logic and control circuit is formed on a semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit, and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit, and (vii) a read-only-memory, and wherein said logic and control circuit determines a background light level, and wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

17. The image sensing system of claim 14, wherein at least one of (a) said at least one control output controls a beam state of said headlight and wherein said image sensing system controls switching the state of said headlight between a higher beam headlight mode and a lower beam headlight mode, (b) said image sensing system controls switching to at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode, (c) said at least one light of the vehicle comprises a headlight having various modes, and wherein said image sensing system controls switching between said various modes, and (d) said at least one control output controls at least one of (i) a beam aim of said headlight, (ii) a beam pattern of said headlight, (iii) a beam intensity of said headlight, (iv) a beam aim of said headlight wherein a continuous variation of the beam aim is provided, (v) a beam state of said headlight wherein switching between one state to another is provided and (vi) a beam state of said headlight as a function of a speed of the vehicle.

18. The image sensing system of claim 14, wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of a light of the vehicle, and wherein at least one of (a) said image processing comprises at least one of (i) spatial differentiation, (ii) spectral signature recognition and (iii) pattern recognition, (b) discrimination between oncoming headlights and leading taillights is accomplished by taking into account the relative location of sources of light within the scene forward of the vehicle, (c) said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance and (iv) spectral characteristic, (d) object recognition is enhanced by at least one of (i) comparing identified objects over successive frames and (ii) one of qualifying and disqualifying objects of interest based on object motion in said field of view of said imaging sensor, (e) said at least one light of the vehicle comprises a fog light and wherein said fog light is controlled responsive to said imaging sensor detecting conditions indicative of fog forward of the vehicle, and (f) said image sensing system detects lane markers on a road being traveled by the vehicle and present in said field of view of said imaging sensor, and wherein said image sensing system controls a headlighting system of the vehicle responsive to said detection of lane markers in said field of view of said imaging sensor, said detection of lane markers comprising identification by spectral signature.

19. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle;

said imaging sensor having a forward field of view to the exterior of the vehicle through an area of the windshield;

wherein said windshield area comprises a windshield area that is swept by the windshield wipers; and

said processing of images by said image sensing system comprising comparing a frame to a successor frame to detect an object of interest, said image processing system providing a control signal for controlling the state of a accessory of the controlled vehicle.

20. The image sensing system of claim 19, wherein said accessory comprises a headlamp of the controlled vehicle and comprising a red filter disposed in front of some of said photosensor elements, said red filter not disposed in front of others of said photosensor elements.

21. The image sensing system of claim 19, wherein said object of interest comprises a light source and wherein said image processing system compares a frame to a successor frame to detect motion of said light source relative to the controlled vehicle.

22. The image sensing system of claim 19, wherein said comparing of frames comprises identifying a pixel grouping having a particular intensity in order to identify the object of interest.

23. The image sensing system of claim 19, including pattern recognition.

24. The image sensing system of claim 19, wherein identified objects of interest are compared over successive frames.

25. The image sensing system of claim 24, wherein said image processing system provides a further control signal for at least one of (a) providing a lane change warning to the driver of the controlled vehicle and (b) a traffic sign recognition.