Image sensing system for a vehicle

• US 7,325,935 B2
• Filed: 01/08/2007
• Issued: 02/05/2008
• Est. Priority Date: 02/26/1993
• Status: Expired due to Fees

• Current Assignee: Donnelly Corporation (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Larson, Mark L
• Primary Examiner(s): Allen; Stephone B.
• Assistant Examiner(s): Pritchett; Joshua L
• Application Number:
  US11/650,646
• Publication Number:
  US 20070109653A1
• Time in Patent Office: 393 Days
• Field of Search: 359/604
• US Class Current: 359/604
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60Q 1/04 the devices being headlights
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2300/43 following vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/04 mounted inside vehicle B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/29 for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1223 : with sensors or transducers
• CPC Class Codes: B60R 2001/1253 : with cameras, video cameras or video screens
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 2300/103 : using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/30 : characterised by the type of image processing
• CPC Class Codes: B60R 2300/8006 : for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8026 : in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/8046 : for replacing a rear-view mirror system
• CPC Class Codes: B60R 2300/8066 : for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8073 : for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: B60R 2300/8093 : for obstacle warning
• CPC Class Codes: B60S 1/0822 : characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 : including a camera
• CPC Class Codes: B60S 1/0885 : the sensor being integrated in a rear-view mirror module
• CPC Class Codes: G01J 1/4204 : with determination of ambient light solar light G01J2001/4266
• CPC Class Codes: G01J 1/4228 : arrangements with two or more detectors, e.g. for sensitivity compensation
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30268 : Vehicle interior
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 25/443 : by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/76 : Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

An image sensing system for a vehicle includes an imaging sensor comprising a two-dimensional array of light sensing photosensor elements. The system includes a logic and control circuit comprising an image processor for processing image data derived from the imaging sensor. The logic and control circuit generates at least one control output for controlling at least one accessory of the vehicle. The imaging sensor is disposed at an interior portion of the cabin of the vehicle and preferably has a field of view exterior of the vehicle through a window of the vehicle.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one accessory of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle and having a field of view exterior of the vehicle through a window of the vehicle;

a lens imaging light external to the vehicle onto said imaging sensor;

at least one object of interest within said field of view of said imaging sensor;

a portion of said array of light sensing photosensor elements capturing an image of said at least one object of interest; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged.

2. The image sensing system of claim 1, wherein said at least one control output comprises an output for controlling a vehicle lighting switch, and wherein said vehicle lighting switch controls a headlight of the vehicle.

3. The image sensing system of claim 1, wherein said array of sensing elements is formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

4. The image sensing system of claim 1, wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

5. The image sensing system of claim 1, wherein said logic and control circuit generates at least two control outputs for controlling respective ones of at least two accessories.

6. The image sensing system of claim 5, wherein one of said at least two accessories comprises a light.

7. The image sensing system of claim 5, wherein one of said at least two accessories comprises a mirror.

8. The image sensing system of claim 5, wherein one of said at least two accessories comprises a vehicle lighting switch and another of said at least two accessories comprises a mirror of the vehicle.

9. The image sensing system of claim 1, wherein said lens comprises a molded plastic lens.

10. The image sensing system of claim 1, wherein said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

11. The image sensing system of claim 1, wherein said interior portion is at or proximate to an interior rearview mirror of the vehicle.

12. The image sensing system of claim 1, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as an integrated circuit.

13. The image sensing system of claim 1, wherein said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

14. The image sensing system of claim 1, wherein a lens correction factor is applied to correct for at least one of (i) cosine effects and (ii) Lambert'"'"'s Law effects.

15. The image sensing system of claim 1, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

16. The image sensing system of claim 1, wherein said light external of the vehicle comprises light from headlights of vehicles.

17. The image sensing system of claim 1, wherein said logic and control circuit determines a background light level, and wherein said logic and control circuit controls a light of the vehicle as function of the determined background light level.

18. The image sensing system of claim 1, wherein said logic and control circuit undertakes pattern recognition based on image data derived from said imaging sensor.

19. The image sensing system of claim 1, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

20. The image sensing system of claim 1, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate.

21. The image sensing system of claim 1, wherein at least a portion of said logic and control circuit comprises at least one selected from the group consisting of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory and (v) a digital-to-analog converter, and wherein said array of sensing elements and said at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate, and wherein at least one of (i) said array of sensing elements and (ii) said at least a portion of said logic and control circuit is formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

22. The image sensing system of claim 1, wherein said at least one control output comprises a pulse-width-modulated control signal.

23. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one accessory of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle and having a field of view exterior of the vehicle through a window of the vehicle;

wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter;

wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate;

at least one object of interest within said field of view of said imaging sensor;

a portion of said array of light sensing photosensor elements capturing an image of said at least one object of interest; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged.

24. The image sensing system of claim 23, wherein said logic and control circuit generates at least two control outputs for controlling respective ones of at least two accessories.

25. The image sensing system of claim 24, wherein one of said at least two accessories comprises a vehicle light.

26. The image sensing system of claim 23 further comprising a lens imaging onto said imaging sensor, and wherein said lens comprises a molded plastic lens.

27. The image sensing system of claim 23, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as an integrated circuit.

28. The image sensing system of claim 23, wherein said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

29. The image sensing system of claim 23, wherein said at least one accessory of the vehicle comprises a headlight.

30. The image sensing system of claim 23, wherein said image sensing system generates an indication of the presence of a vehicle within the field of view of said imaging sensor.

31. The image sensing system of claim 30, wherein said indication comprises at least one of a visual warning and an audible warning.

32. The image sensing system of claim 30, wherein said indication is generated based on at least one of distance and speed of the vehicle.

33. The image sensing system of claim 23, wherein said array of sensing elements is formed on the semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

34. The image sensing system of claim 23, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device, and wherein said portion of said logic and control circuit commonly formed on said semiconductor substrate comprises at least one selected from the group consisting of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory and (v) a digital-to-analog converter.

35. The image sensing system of claim 23, wherein said at least one control output comprises a pulse-width-modulated control signal.

36. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

a logic and control circuit;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

said imaging sensor disposed at an interior portion of the vehicle and having a field of view at least one of (i) interior of the vehicle and (ii) exterior of the vehicle through a window of the vehicle;

at least one object of interest within said field of view of said imaging sensor;

a portion of said array of light sensing photosensor elements capturing an image of said at least one object of interest;

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged;

wherein said image sensing system generates an indication of the presence of an object within the field of view of said imaging sensor; and

wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on a semiconductor substrate.

37. The image sensing system of claim 36 further comprising a lens imaging onto said imaging sensor, and wherein said lens comprises a molded plastic lens.

38. The image sensing system of claim 36, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as an integrated circuit, and wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

39. The image sensing system of claim 36, wherein said logic and control circuit generates at least one control output for controlling at least one accessory of the vehicle.

40. The image sensing system of claim 39, wherein said at least one accessory of the vehicle comprises a light.

41. The image sensing system of claim 39, wherein said at least one accessory comprises at least one of (i) a display and (ii) an alarm.

42. The image sensing system of claim 36, wherein said image sensing system generates an indication of the presence of a vehicle within the field of view of said imaging sensor, and wherein said indication comprises at least one of a visual warning and an audible warning.

43. The image sensing system of claim 36, wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

44. The image sensing system of claim 36, wherein said array of sensing elements is formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

45. The image sensing system of claim 36, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device, and wherein said portion of said logic and control circuit commonly formed on said semiconductor substrate comprises at least one selected from the group consisting of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory and (v) a digital-to-analog converter.

46. The image sensing system of claim 36, wherein said imaging sensor has a field of view exterior of the vehicle through a window of the vehicle.

47. The image sensing system of claim 36, wherein said indication comprises at least one of a visual warning and an audible warning.

48. The image sensing system of claim 36, wherein said image sensing system generates an indication of the presence of a vehicle within the field of view of said imaging sensor and wherein said indication is generated based on at least one of distance and speed of the vehicle.

49. The image sensing system of claim 36, wherein said interior portion is proximate a windshield of the vehicle.

50. The image sensing system of claim 36, wherein said imaging sensor has a field of view interior of the vehicle, and wherein said image sensing system generates an indication of at least one of (i) the presence of a person within the field of view of said imaging sensor and (ii) the presence of a person moving within the vehicle.

51. The image sensing system of claim 36, wherein said at least a portion of said logic and control circuit is selected from the group consisting of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory and (v) a digital-to-analog convert, and wherein said imaging sensor has a field of view exterior of the vehicle through a window of the vehicle, and wherein said indication comprises at least one of a visual warning and an audible warning, and wherein said indication is generated based on at least one of distance and speed of the object.

52. The image sensing system of claim 51, wherein said array of sensing elements is formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

53. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

a logic and control circuit;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor;

wherein said array of sensing elements is formed on a semiconductor substrate;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle and having a field of view exterior of the vehicle through a window of the vehicle;

at least one object of interest within said field of view of said imaging sensor;

a portion of said array of light sensing photosensor elements capturing an image of said at least one object of interest;

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged;

wherein said image sensing system generates an indication of the presence of an object within the field of view of said imaging sensor; and

wherein said indication comprises at least one of a visual warning and an audible warning.

54. The image sensing system of claim 53, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on a semiconductor substrate as an integrated circuit.

55. The image sensing system of claim 53, wherein said logic and control circuit generates at least one control output for controlling at least one accessory of the vehicle.

56. The image sensing system of claim 55, wherein said at least one accessory comprises at least one of (i) a display and (ii) an alarm.

57. The image sensing system of claim 53, wherein said image sensing system generates an indication of the presence of a vehicle within the field of view of said imaging sensor, and wherein said indication is generated based on at least one of distance and speed of the vehicle.

58. The image sensing system of claim 53, wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

59. The image sensing system of claim 53, wherein said array of sensing elements is formed on said semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.

60. The image sensing system of claim 53, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on a semiconductor substrate.

61. The image sensing system of claim 53, wherein said array of sensing elements and at least a portion of said logic and control circuit are commonly formed on a semiconductor substrate as a complementary-metal-oxide-semiconductor (CMOS) device.