Image sensing system for a vehicle

• US 7,459,664 B2
• Filed: 01/24/2007
• Issued: 12/02/2008
• Est. Priority Date: 02/26/1993
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Vadas, Keith J., Larson, Mark L.
• Primary Examiner(s): Yam; Stephen
• Application Number:
  US11/626,535
• Publication Number:
  US 20070120706A1
• Time in Patent Office: 678 Days
• Field of Search: 250/208.1, 382/104, 701/200, 701/224, 701/23, 701/27, 701/28, 702/92, 340/901, 348/118
• US Class Current: 250/208.1
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 2300/054 Variable non-standard intensity, i.e. emission of various beam intensities different from standard intensities, e.g. continuous or stepped transitions of intensity
• CPC Class Codes: B60Q 2300/056 Special anti-blinding beams, e.g. a standard beam is chopped or moved in order not to blind
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60R 2001/1223 with sensors or transducers
• CPC Class Codes: B60S 1/0822 characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: B60S 1/087 including an ambient light sensor control of vehicle lights B60Q
• CPC Class Codes: G01J 1/0488 with spectral filtering
• CPC Class Codes: G01J 1/4228 arrangements with two or more detectors, e.g. for sensitivity compensation
• CPC Class Codes: G01J 3/02 Details
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G01J 3/51 using colour filters
• CPC Class Codes: G01J 3/513 having fixed filter-detector pairs
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

An image sensing system for a vehicle includes an imaging sensor and a logic and control circuit. The imaging sensor comprises a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate and is disposed at an interior portion of the vehicle proximate the windshield of the vehicle and has a forward field of view to the exterior of the vehicle through an area of the windshield. The logic and control circuit comprises an image processor for processing image data derived from the imaging sensor. The logic and control circuit generates at least one control output. The image sensing system detects lane markers on a road being traveled by the vehicle and present in the field of view of the imaging sensor.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor being supported by a mounting structure fixedly attached to an interior surface of the vehicle windshield so as to physically fixedly position said imaging sensor within the interior cabin of the vehicle and close to and rearward of the windshield;

said imaging sensor having a fixed field of view to the exterior of the vehicle through a window of the vehicle, the field of view encompassing the road traveled by the vehicle;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said image processing comprising at least one of (i) spatial differentiation, and (ii) pattern recognition; and

wherein said image sensing system detects objects external of the vehicle associated with the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system detecting objects by processing said image data to identify objects based on at least one of (i) spatial differentiation, and (ii) pattern recognition, said image sensing system generating at least one control output responsive to said detection of objects.

2. The image sensing system of claim 1, wherein at least one of (i) said array of light sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on said semiconductor substrate as a CMOS device.

3. The image sensing system of claim 1, wherein said array of light sensing photosensor elements and at least a portion of said logic and control circuit are formed on said semiconductor substrate as a CMOS device.

4. The image sensing system of claim 2, wherein said portion of said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

5. The image sensing system of claim 1 farther comprising a lens imaging light external of the vehicle onto said array of light sensing photosensor elements, and wherein said lens comprises a molded plastic lens.

6. The image sensing system of claim 1, including a lens imaging light external of the vehicle onto said array of light sensing photosensor elements and wherein said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

7. The image sensing system of claim 1, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

8. The image sensing system of claim 1, wherein at least a portion of said logic and control circuit is commonly formed with said array of light sensing photosensor elements on said semiconductor substrate as an integrated circuit.

9. The image sensing system of claim 1, wherein said logic and control circuit comprises a logic circuit, at least a portion of said logic circuit comprising a configuration of digital logic elements formed on said semiconductor substrate.

10. The image sensing system of claim 1, wherein said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

11. The image sensing system of claim 1, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said light sensing photosensor elements to an adjacent one of said light sensing photosensor elements.

12. The image sensing system of claim 1, wherein said logic and control circuit determines a background light level.

13. The image sensing system of claim 1, wherein said at least one control output controls as a function of a speed of the vehicle.

14. The image sensing system of claim 1, wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

15. The image sensing system of claim 1, wherein said field of view has a horizontal field of view and a vertical field of view and wherein said horizontal field of view is wider than said vertical field of view.

16. The image sensing system of claim 1, wherein said image processing comprises pattern recognition and wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

17. The image sensing system of claim 1, wherein said image sensing system identifies objects of interest and wherein object recognition is enhanced by comparing identified objects over successive frames.

18. The image sensing system of claim 1, wherein said image sensing system identifies objects of interest and wherein objects of interest are at least one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

19. The image sensing system of claim 1, wherein said image sensing system detects conditions indicative of fog forward of the vehicle.

20. The image sensing system of claim 1, wherein said detection of objects comprises identification by spectral signature.

21. The image sensing system of claim 1, wherein said imaging sensor is supported by said mounting structure so as to physically fixedly position said imaging sensor close to and rearward of a portion of the windshield swept by a windshield wiper of the vehicle, and wherein said imaging sensor has a fixed forward field of view to the exterior of the vehicle through the portion of the windshield that is swept by the windshield wiper of the vehicle.

22. The image sensing system of claim 21, wherein said image processing comprises at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition, and wherein said image sensing system detects lane markers on the road traveled by the vehicle and present in the field of view of said imaging sensor, said image processing detecting lane markers by processing said image data to identify lane markers based on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition, said image sensing system generating at least one control output responsive to a detection of lane markers to at least one of (a) assist in steering the vehicle and (b) provide a lane change warning to the driver of the vehicle.

23. The image sensing system of claim 1, wherein said image sensing system detects objects by processing said image data to identify objects based on spatial differentiation and pattern recognition.

24. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor being supported by a mounting structure fixedly attached to an interior surface of the vehicle windshield so as to physically fixedly position said imaging sensor within the interior cabin of the vehicle and close to and rearward of the windshield;

said imaging sensor having a fixed field of view to the exterior of the vehicle through a window of the vehicle, the field of view encompassing the road traveled by the vehicle;

a logic and control circuit, wherein at least one of (i) said array of light sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on said semiconductor substrate as a CMOS device;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said image processing comprising at least one of (i) spatial differentiation, and (ii) pattern recognition;

wherein said image sensing system detects objects external of the vehicle associated with the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of objects.

25. The image sensing system of claim 24, wherein said array of light sensing photosensor elements and at least a portion of said logic and control circuit are formed on said semiconductor substrate as a CMOS device and wherein said portion of said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

26. The image sensing system of claim 24, including a lens imaging light external of the vehicle onto said light sensing array, and wherein said lens comprises a molded plastic lens and wherein said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

27. The image sensing system of claim 24, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

28. The image sensing system of claim 24, wherein said field of view has a horizontal field of view and a vertical field of view and wherein said horizontal field of view is wider than said vertical field of view.

29. The image sensing system of claim 24, wherein said image processing comprises pattern recognition and wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

30. The image sensing system of claim 24, wherein said image sensing system identifies objects of interest and wherein object recognition is enhanced by comparing identified objects over successive frames.

31. The image sensing system of claim 24, wherein said image sensing system identifies objects of interest and wherein objects of interest are at least one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

32. The image sensing system of claim 24, wherein said imaging sensor is supported by said mounting structure so as to physically fixedly position said imaging sensor close to and rearward of a portion of the windshield swept by a windshield wiper of the vehicle, and wherein said imaging sensor has a fixed forward field of view to the exterior of the vehicle through the portion of the windshield that is swept by the windshield wiper of the vehicle.

33. The image sensing system of claim 32, wherein said image processing comprises at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition, and wherein said image sensing system detects lane markers on the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of lane markers to at least one of (a) assist in steering the vehicle and (b) provide a lane change warning to the driver of the vehicle, and wherein said image processing detects lane markers based on identification of a spectral signature of the lane markers.

34. The image sensing system of claim 24, wherein said image processing comprises spatial differentiation and pattern recognition.

35. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor being supported by a mounting structure fixedly attached to an interior surface of the vehicle windshield so as to physically fixedly position said imaging sensor within the interior cabin of the vehicle and close to and rearward of the windshield;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system detects objects present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of objects; and

wherein said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) size, (iii) contour, and (iv) motion, and wherein said image sensing system detects objects by processing said image data to identify objects of interest based on at least one of (i) spatial differentiation, and (ii) pattern recognition.

36. The image sensing system of claim 35, wherein at least one of (a) at least one of (i) said array of light sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on said semiconductor substrate as a CMOS device and (b) said array of light sensing photosensor elements and at least a portion of said logic and control circuit are formed on said semiconductor substrate as a CMOS device.

37. The image sensing system of claim 35, wherein said field of view has a horizontal field of view and a vertical field of view and wherein said horizontal field of view is wider than said vertical field of view and wherein object recognition is enhanced by comparing identified objects over successive frames.

38. The image sensing system of claim 35, wherein objects of interest are at least one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

39. The image sensing system of claim 35, wherein said image processing comprises at least one of (i) spatial differentiation and (ii) spectral signature recognition.

40. The image sensing system of claim 35, wherein said detection of lane markers comprises identification by spectral signature.

41. The image sensing system of claim 35, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

42. The image sensing system of claim 35, wherein said imaging sensor is supported by said mounting structure so as to physically fixedly position said imaging sensor close to and rearward of a portion of the windshield swept by a windshield wiper of the vehicle, and wherein said imaging sensor has a fixed forward field of view to the exterior of the vehicle through the portion of the windshield that is swept by the windshield wiper of the vehicle, the field of view encompassing the road traveled by the vehicle.

43. The image sensing system of claim 42, wherein said image sensing system detects lane markers on the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of lane markers to at least one of (a) assist in steering the vehicle and (b) provide a lane change warning to the driver of the vehicle, and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) size, (iii) contour, (iv) motion, (v) reflectivity, (vi) luminance, and (vii) spectral characteristic, and wherein said image processing detects lane markers by processing said image data to identify lane markers based on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

44. The image sensing system of claim 35, wherein said image sensing system identifies objects of interest based on spatial differentiation and pattern recognition.

45. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate as a CMOS device;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle and wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle, said imaging sensor being supported by a mounting structure fixedly attached to an interior surface of the vehicle windshield so as to physically fixedly position said imaging sensor within the interior cabin of the vehicle and close to and rearward of the windshield;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system detects objects present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of objects;

wherein said image processing comprises pattern recognition and wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) size, (iii) contour, and (iv) motion, and wherein said image sensing system detects objects by processing said image data to identify objects based on at least one of (i) spatial differentiation, and (ii) pattern recognition; and

wherein said field of view has a horizontal field of view and a vertical field of view and wherein said horizontal field of view is wider than said vertical field of view.

46. The image sensing system of claim 45, wherein said array of light sensing photosensor elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as a CMOS device.

47. The image sensing system of claim 45, wherein said image processing comprises at least one of (i) spatial differentiation and (ii) spectral signature recognition.

48. The image sensing system of claim 45, wherein said detection of objects comprises detection of lane markers and wherein said detection of lane markers comprises identification by spectral signature.

49. The image sensing system of claim 45, wherein said imaging sensor is supported by said mounting structure so as to physically fixedly position said imaging sensor close to and rearward of a portion of the windshield swept by a windshield wiper of the vehicle, and wherein said imaging sensor has a fixed forward field of view to the exterior of the vehicle through the portion of the windshield that is swept by the windshield wiper of the vehicle, the field of view encompassing the road traveled by the vehicle.

50. The image sensing system of claim 49, wherein said image sensing system detects lane markers on the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of lane markers to at least one of (a) assist in steering the vehicle and (b) provide a lane change warning to the driver of the vehicle.

51. The image sensing system of claim 49, wherein said pattern recognition identifies objects of interest based on at least one of (i) shape, (ii) size, (iii) contour, (iv) motion, (v) reflectivity, (vi) luminance, and (vii) spectral characteristic, and wherein said image processing detects lane markers by processing said image data to identify lane markers based on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

52. The image sensing system of claim 45, wherein said image sensing system identifies objects based on spatial differentiation and pattern recognition.

53. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle and wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle, said imaging sensor being supported by a mounting structure fixedly attached to an interior surface of the vehicle windshield so as to physically fixedly position said imaging sensor within the interior cabin of the vehicle and close to and rearward of the windshield;

said imaging sensor having a fixed field of view to the exterior of the vehicle through a window of the vehicle, the field of view encompassing the road traveled by the vehicle;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said image sensing system detects objects external of the vehicle associated with the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of objects;

wherein said image processing comprises pattern recognition;

wherein said image sensing system identifies objects of interest and wherein object recognition is enhanced by comparing identified objects over successive frames, and wherein said image sensing system detects objects of interest by processing said image data to identify objects of interest based on at least one of (i) spatial differentiation, and (ii) pattern recognition; and

wherein said field of view has a horizontal field of view and a vertical field of view and wherein said horizontal field of view is wider than said vertical field of view.

54. The image sensing system of claim 53, wherein at least one of (i) said array of light sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on said semiconductor substrate as a CMOS device.

55. The image sensing system of claim 53, wherein said image sensing system identifies objects of interest based on at least one of (i) shape, (ii) reflectivity, (iii) luminance, and (iv) spectral characteristic.

56. The image sensing system of claim 53, wherein objects of interest are at least one of qualified and disqualified based on object motion in said field of view of said imaging sensor.

57. The imaging sensing system of claim 53, wherein said imaging sensor is supported by said mounting structure so as to physically fixedly position said imaging sensor close to and rearward of a portion of the windshield swept by a windshield wiper of the vehicle, and wherein said imaging sensor has a fixed forward field of view to the exterior of the vehicle through the portion of the windshield that is swept by the windshield wiper of the vehicle.

58. The imaging sensing system of claim 57, wherein said image sensing system detects lane markers on the road traveled by the vehicle and present in the field of view of said imaging sensor, said image sensing system generating at least one control output responsive to a detection of lane markers to at least one of (a) assist in steering the vehicle and (b) provide a lane change warning to the driver of the vehicle, and wherein said image processing detects lane markers by processing said image data to identify lane markers based on at least one of (i) spatial differentiation, (ii) spectral signature recognition, and (iii) pattern recognition.

59. The image sensing system of claim 53, wherein said image sensing system identifies objects of interest based on spatial differentiation and pattern recognition.