Image sensing system for a vehicle

• US 20070109654A1
• Filed: 01/10/2007
• Published: 05/17/2007
• Est. Priority Date: 02/26/1993
• Status: Active Grant

• Current Assignee: Donnelly Corporation (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Larson, Mark
• Granted Patent: US 7,311,406 B2
• Time in Patent Office: Days
• US Class Current: 359/604
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/0024 for identifying, categorising or investigation of the occupant or object on the seat
• CPC Class Codes: B60N 2210/24 Cameras
• CPC Class Codes: B60Q 1/04 the devices being headlights
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2300/43 following vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: B60R 1/04 mounted inside vehicle B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/29 for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 11/04 : Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1223 : with sensors or transducers
• CPC Class Codes: B60R 2001/1253 : with cameras, video cameras or video screens
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 2300/103 : using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/30 : characterised by the type of image processing
• CPC Class Codes: B60R 2300/8006 : for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8026 : in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/8046 : for replacing a rear-view mirror system
• CPC Class Codes: B60R 2300/8066 : for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8073 : for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: B60R 2300/8093 : for obstacle warning
• CPC Class Codes: B60S 1/0822 : characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 : including a camera
• CPC Class Codes: B60S 1/0885 : the sensor being integrated in a rear-view mirror module
• CPC Class Codes: G01J 1/4204 : with determination of ambient light solar light G01J2001/4266
• CPC Class Codes: G01J 1/4228 : arrangements with two or more detectors, e.g. for sensitivity compensation
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30268 : Vehicle interior
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 25/443 : by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/76 : Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

An image sensing system for a vehicle includes an imaging sensor comprising a two-dimensional array of light sensing photosensor elements, preferably formed on a semiconductor substrate, and a logic and control circuit, preferably with at least a portion of the logic and control circuit commonly formed on a semiconductor with the array of light sensing elements. The logic and control circuit comprises an image processor for processing image data derived from the imaging sensor. The imaging sensor is disposed at an interior portion of the cabin of the vehicle and has a field of view. At least one illumination source illuminates a region of interest within the field of view of the imaging sensor and the illumination source principally radiates in a spectral region that is outside the visible light spectrum and that comprises wavelengths within the responsive range of the imaging sensor.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

a logic and control circuit;

at least a portion of said logic and control circuit commonly formed on said semiconductor with said array of light sensing elements;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said imaging sensor disposed at an interior portion of the vehicle and having a field of view;

at least one illumination source illuminating a region of interest within the field of view of said imaging sensor;

said illumination source principally radiating in a spectral region that is outside the visible light spectrum; and

wherein the spectral region radiated by said illumination source comprises wavelengths within the responsive range of said imaging sensor.

2. The image sensing system of claim 1, wherein said region of interest is within an interior compartment of the vehicle.

3. The image sensing system of claim 1, wherein said illumination source comprises a near-infrared source.

4. The image sensing system of claim 3, wherein said near-infrared source comprises a near-infrared emitting light emitting diode.

5. The image sensing system of claim 1, wherein said illumination source principally radiates in the spectral region between about 700 nm and about 1200 nm.

6. The image sensing system of claim 5, wherein said illumination source comprises one of (i) a laser and (ii) a light-emitting diode.

7. The image sensing system of claim 1, wherein said illumination source radiates near-infrared radiation having a peak spectral output between about 750 nm and about 950 nm.

8. The image sensing system of claim 1, wherein said illumination source comprises a light-emitting diode and wherein said light-emitting diode is chosen from the group consisting of a gallium arsenide light-emitting diode and a gallium aluminum arsenide light-emitting diode.

9. The image sensing system of claim 1, wherein said region of interest is exterior the vehicle.

10. The image sensing system of claim 1 further comprising a lens to distribute the illumination of said illumination source.

11. The image sensing system of claim 10, wherein said lens provides illumination in a cone of at least about 100 degrees.

12. The image sensing system of claim 1, wherein a lens images light onto said imaging sensor and wherein said lens comprises a molded plastic lens.

13. The image sensing system of claim 1, wherein said interior portion is at or proximate to a headliner of the vehicle.

14. The image sensing system of claim 1, wherein said interior portion is generally forward of the front seat of the vehicle and is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

15. The image sensing system of claim 1, wherein said interior portion is at or proximate to an interior rearview mirror assembly location of the vehicle.

16. The image sensing system of claim 1, wherein said illumination source provides illumination only during an exposure period of said imaging sensor.

17. The image sensing system of claim 1, wherein said imaging sensor has a field of view forward through the windshield of the vehicle.

18. The image sensing system of claim 1, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit, and (vii) a read-only-memory.

19. The image sensing system of claim 1, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

20. The image sensing system of claim 1, wherein said logic and control circuit undertakes pattern recognition based on image data derived from said imaging sensor.

21. The image sensing system of claim 1, wherein said region of interest comprises a front passenger seat of the vehicle and wherein said logic and control circuit processes image data derived from said imaging sensor to determine occupancy of the front passenger seat by a person, and wherein a control signal is provided to an air bag deployment system of the vehicle responsive to said determining of occupancy of the front passenger seat by a person, said control signal controlling at least one of (i) a deployment of an air bag, (ii) a non-deployment of an airbag and (iii) a rate of deployment of an airbag.

22. The image sensing system of claim 1, wherein said logic and control circuit processes image data derived from said imaging sensor to determine at least one of (i) the status of an occupant of a front passenger seat, (ii) occupancy by a person of a front passenger seat, and (iii) a presence of a child restraint system at a front passenger seat.

23. The image sensing system of claim 22, wherein said determination is based at least in part on at least one of size, shape, contour and motion image information sensed by said imaging sensor.

24. The image sensing system of claim 1, wherein said light sensing and control circuit outputs a pulse-width-modulated signal.

25. The image sensing system of claim 1, wherein said image sensing system provides a video output to a display device located within the vehicle to provide a display of the imaged scene for viewing by the driver of the vehicle.

26. The image sensing system of claim 25, wherein said display device comprises at least one of (i) a flat panel display device and (ii) a liquid crystal flat panel display device.

27. The image sensing system of claim 1, wherein said imaging sensor has a peak in its normalized spectral response between about 750 nm and about 1200 nm.

28. The image sensing system of claim 1, wherein said imaging sensor has a peak in its normalized spectral response at about 800 nm.

29. The image sensing system of claim 1, wherein said semiconductor substrate is formed using a non-epitaxial silicon process.

30. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor disposed at an interior portion of the vehicle and having a field of view;

at least one illumination source illuminating a region of interest within the field of view of said imaging sensor;

said illumination source principally radiating in a spectral region that is outside the visible light spectrum;

wherein the spectral region radiated by said illumination source comprises wavelengths within the responsive range of said imaging sensor;

wherein said illumination source comprises a near-infrared source, said near-infrared source principally radiating in the spectral region between about 700 nm and about 1200 nm; and

wherein said imaging sensor has a peak in its normalized spectral response between about 750 nm and about 1200 nm.

31. The image sensing system of claim 30, wherein said region of interest is within an interior compartment of the vehicle.

32. The image sensing system of claim 30, wherein said illumination source comprises one of (i) a laser and (ii) a light-emitting diode.

33. The image sensing system of claim 30, wherein said region of interest is exterior the vehicle.

34. The image sensing system of claim 30 further comprising a logic and control circuit, said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor.

35. The image sensing system of claim 34, wherein said array of sensing photosensor elements and at least a portion of said logic and control circuit are commonly formed on a semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit, and (vii) a read-only-memory.

36. The image sensing system of claim 30, wherein a lens images light onto said imaging sensor and wherein said lens comprises a molded plastic lens.

37. The image sensing system of claim 30, wherein said interior portion is at or proximate to an interior rearview mirror assembly location of the vehicle.

38. The image sensing system of claim 30, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

39. The image sensing system of claim 30, wherein said image sensing system provides a video output to a display device located within the vehicle to provide a display of the imaged scene for viewing by the driver of the vehicle.

40. The image sensing system of claim 39, wherein said display device comprises one of (i) a flat panel display device and (ii) a liquid crystal flat panel display device.

41. The image sensing system of claim 30, wherein said interior portion is proximate the windshield of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers.

42. The image sensing system of claim 30, wherein said two-dimensional array of light sensing photosensor elements is formed on a semiconductor substrate as a CMOS device.

43. The image sensing system of claim 30, wherein said semiconductor substrate is formed using a non-epitaxial silicon process.

44. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

at least one illumination source illuminating a region of interest within the field of view of said imaging sensor;

said illumination source principally radiating in a spectral region that is outside the visible light spectrum;

wherein the spectral region radiated by said illumination source comprises wavelengths within the responsive range of said imaging sensor;

wherein said illumination source comprises a near-infrared source; and

wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

45. The image sensing system of claim 44, wherein said infrared source principally radiates in the spectral region between about 700 nm and about 1200 nm.

46. The image sensing system of claim 44, wherein said illumination source comprises one of (i) a laser and (ii) a light-emitting diode.

47. The image sensing system of claim 44, wherein said array of sensing elements is formed on said semiconductor substrate as a CMOS device.

48. The image sensing system of claim 44 further comprising a logic and control circuit, said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor.

49. The image sensing system of claim 48, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit, and (vii) a read-only-memory.

50. The image sensing system of claim 44, wherein a lens images light onto said imaging sensor and wherein said lens comprises a molded plastic lens.

51. The image sensing system of claim 44, wherein said interior portion is at or proximate to an interior rearview mirror assembly location of the vehicle.

52. The image sensing system of claim 44, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

53. The image sensing system of claim 44, wherein said image sensing system provides a video output to a display device located within the vehicle to provide a display of the imaged scene for viewing by the driver of the vehicle.

54. The image sensing system of claim 53, wherein said display device comprises one of (i) a flat panel display device and (ii) a liquid crystal flat panel display device.

55. The image sensing system of claim 44, wherein said imaging sensor has a peak in its normalized spectral response between about 750 nm and about 1200 nm.

56. The image sensing system of claim 44, wherein said semiconductor substrate is formed using a non-epitaxial silicon process.

57. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle and wherein said imaging sensor has a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

at least one illumination source illuminating a region of interest within the field of view of said imaging sensor;

said illumination source principally radiating in a spectral region that is outside the visible light spectrum;

wherein the spectral region radiated by said illumination source comprises wavelengths within the responsive range of said imaging sensor;

wherein said illumination source comprises a near-infrared source; and

wherein said image sensing system provides a video output to a display device located within the vehicle to provide a display of the imaged scene for viewing by the driver of the vehicle.

58. The image sensing system of claim 57, wherein said near-infrared source principally radiates in the spectral region between about 700 nm and about 1200 nm.

59. The image sensing system of claim 57, wherein said illumination source comprises one of (i) a laser and (ii) a light-emitting diode.

60. The image sensing system of claim 57, wherein said imaging sensor has a peak in its normalized spectral response between about 750 nm and about 1200 nm.

61. The image sensing system of claim 57, wherein said array of sensing elements is formed on a semiconductor substrate as a CMOS device.

62. The image sensing system of claim 57 further comprising a logic and control circuit, said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor.

63. The image sensing system of claim 62, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as at least one of (a) a CMOS device, (b) an integrated circuit and (c) a logic and control circuit comprising at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) a random access memory, (v) a digital-to-analog converter, (vi) a central processing unit, and (vii) a read-only-memory.

64. The image sensing system of claim 57, wherein a lens images light onto said imaging sensor and wherein said lens comprises a molded plastic lens.

65. The image sensing system of claim 57, wherein said interior portion is at or proximate to an interior rearview mirror assembly location of the vehicle.

66. The image sensing system of claim 57, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

67. The image sensing system of claim 57, wherein said display device comprises one of (i) a flat panel display device and (ii) a liquid crystal flat panel display device.

68. The image sensing system of claim 57, wherein said array of sensing elements is formed on a semiconductor substrate using a non-epitaxial silicon process.