Image sensing system for a vehicle

• US 7,325,934 B2
• Filed: 01/08/2007
• Issued: 02/05/2008
• Est. Priority Date: 02/26/1993
• Status: Expired due to Fees

• Current Assignee: Donnelly Corporation (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Larson, Mark L
• Primary Examiner(s): Allen; Stephone B.
• Assistant Examiner(s): Pritchett; Joshua L
• Application Number:
  US11/650,629
• Publication Number:
  US 20070109652A1
• Time in Patent Office: 393 Days
• Field of Search: 359/604
• US Class Current: 359/604
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60Q 1/04 the devices being headlights
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2300/43 following vehicle
• CPC Class Codes: B60Q 2400/30 Daytime running lights [DRL], e.g. circuits or arrangements therefor
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/04 mounted inside vehicle B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/29 for viewing an area inside the vehicle, e.g. for viewing passengers or cargo
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1223 : with sensors or transducers
• CPC Class Codes: B60R 2001/1253 : with cameras, video cameras or video screens
• CPC Class Codes: B60R 21/01538 : for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 2300/103 : using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/30 : characterised by the type of image processing
• CPC Class Codes: B60R 2300/8006 : for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/8026 : in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/8046 : for replacing a rear-view mirror system
• CPC Class Codes: B60R 2300/8066 : for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8073 : for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: B60R 2300/8093 : for obstacle warning
• CPC Class Codes: B60S 1/0822 : characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 : including a camera
• CPC Class Codes: B60S 1/0885 : the sensor being integrated in a rear-view mirror module
• CPC Class Codes: G01J 1/4204 : with determination of ambient light solar light G01J2001/4266
• CPC Class Codes: G01J 1/4228 : arrangements with two or more detectors, e.g. for sensitivity compensation
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30268 : Vehicle interior
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 25/443 : by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/76 : Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 7/18 : Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

An image sensing system for a vehicle includes an imaging sensor having a two-dimensional array of light sensing photosensor elements, and a logic and control circuit having an image processor for processing image data derived from the imaging sensor. The logic and control circuit generates at least one control output for controlling at least one light of the vehicle. The imaging sensor is disposed at an interior portion of the cabin of the vehicle proximate the windshield of the vehicle and has a forward field of view to the exterior of the vehicle, preferably through a windshield area that is swept by the windshield wipers. The imaging sensor is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

Claims

1. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements formed on a semiconductor substrate;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor having a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle;

a portion of said array of light sensing photosensor elements capturing an image of at least one object of interest within said forward field of view of said imaging sensor; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged; and

wherein said at least one control output comprises an output for controlling a vehicle lighting switch, and wherein said vehicle lighting switch controls at least one of (i) a headlight of the vehicle and (ii) a side light of the vehicle.

2. The image sensing system of claim 1, wherein said at least one light of the vehicle comprises a fog light and wherein said fog light is controlled responsive to said imaging sensor detecting conditions indicative of fog forward of the vehicle.

3. The image sensing system of claim 1, wherein at least one of said array of sensing photosensor elements and at least a portion of said logic and control circuit is formed on said semiconductor substrate as at least one of (i) a CMOS device and (ii) an integrated circuit.

4. The image sensing system of claim 1, wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.

5. The image sensing system of claim 1, wherein said logic and control circuit generates at least two control outputs for controlling respective ones of at least two accessories, one of said at least two accessories being said at least one light of the vehicle, and wherein said at least one light of the vehicle comprises a vehicle headlight.

6. The image sensing system of claim 5, wherein another one of said at least two accessories comprises a mirror of the vehicle.

7. The image sensing system of claim 1 further comprising a lens imaging light external to the vehicle onto said imaging sensor and wherein said lens comprises a molded plastic lens, and wherein said lens is at least one of (i) bonded to said imaging sensor and (ii) in close contact with said imaging sensor.

8. The image sensing system of claim 1, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

9. The image sensing system of claim 1, wherein said logic and control circuit comprises a logic circuit, said logic circuit comprising at least one of (i) a central processing unit and (ii) a read-only-memory.

10. The image sensing system of claim 1, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

11. The image sensing system of claim 1, wherein said logic and control circuit determines a background light level, and wherein said logic and control circuit controls a light of the vehicle as function of the determined background light level.

12. The image sensing system of claim 1, wherein said logic and control circuit undertakes pattern recognition based on image data derived from said imaging sensor.

13. The image sensing system of claim 1, wherein said at least one control output for controlling at least one light of the vehicle comprises a pulse-width-modulated signal.

14. The image sensing system of claim 1, wherein said image sensing system controls switching between a higher beam headlight mode and a lower beam headlight mode.

15. The image sensing system of claim 14, wherein said image sensing system controls switching to at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode.

16. The image sensing system of claim 15, wherein said intermediate headlight mode comprises a mid beam headlight mode.

17. The image sensing system of claim 1, wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of a light of the vehicle.

18. The image sensing system of claim 1, wherein said at least one light of the vehicle comprises a headlight having various modes, and wherein said image sensing system controls switching between said various modes.

19. The image sensing system of claim 18 including hysteresis to moderate the frequency of transitions between the various modes.

20. The image sensing system of claim 18, wherein switching from a higher beam mode to a lower beam mode requires less frames of image data derived from said imaging sensor than does switching from a lower beam mode to a higher beam mode.

21. The image sensing system of claim 18, wherein switching from a higher beam mode to a lower beam mode takes less time than does switching from a corresponding lower beam mode to a corresponding higher beam mode.

22. The image sensing system of claim 1, wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle, wherein said image sensing system controls a lighting system of the vehicle in accordance with said determination of a lighting configuration for the vehicle, and wherein said lighting system comprises a headlighting system.

23. The image sensing system of claim 22, wherein said at least one control output comprises a pulse-width-modulated control signal.

24. The image sensing system of claim 1, wherein said array of sensing photosensor elements is formed on said semiconductor substrate as a CMOS device.

25. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor having a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

wherein a lens images light external to the vehicle onto said imaging sensor;

wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein at least one of (i) said array of sensing photosensor elements and (ii) at least a portion of said logic and control circuit is formed on a semiconductor substrate;

wherein a portion of said array of light sensing photosensor elements captures an image of at least one object of interest within said forward field of view of said imaging sensor; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged.

26. The image sensing system of claim 25, wherein said array of sensing photosensor elements is formed on said semiconductor substrate as a CMOS device.

27. The image sensing system of claim 25, wherein said array of sensing photosensor elements and at least a portion of said logic and control circuit are formed on said semiconductor substrate as at least one of (i) a CMOS device and (ii) an integrated circuit.

28. The image sensing system of claim 25, wherein said lens comprises a molded plastic lens.

29. The image sensing system of claim 25, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

30. The image sensing system of claim 25, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

31. The image sensing system of claim 25, wherein said logic and control circuit determines a background light level and wherein said logic and control circuit controls said headlight of the vehicle as function of the determined background light level.

32. The image sensing system of claim 25, wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

33. The image sensing system of claim 25, wherein said at least one control output for controlling said headlight of the vehicle comprises a pulse-width-modulated signal.

34. The image sensing system of claim 25, wherein said image sensing system controls switching said headlight at least between a higher beam headlight mode and a lower beam headlight mode.

35. The image sensing system of claim 34, wherein said image sensing system controls switching to at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode.

36. The image sensing system of claim 34, wherein switching from a higher beam mode of said headlight to a lower beam mode takes less time than does switching from a corresponding lower beam mode to a corresponding higher beam mode.

37. The image sensing system of claim 25, wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of said headlight of the vehicle.

38. The image sensing system of claim 25, wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle, and wherein said image sensing system controls a lighting system of the vehicle in accordance with said determination of a lighting configuration for the vehicle, and wherein said lighting system comprises said headlight.

39. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor having a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle;

wherein said image sensing system controls a lighting system of the vehicle in accordance with said determination of a lighting configuration for the vehicle;

wherein said lighting system comprises said headlight;

a portion of said array of light sensing photosensor elements capturing an image of at least one object of interest within said forward field of view of said imaging sensor; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged.

40. The image sensing system of claim 39, wherein said array of sensing photosensor elements is formed on a semiconductor substrate.

41. The image sensing system of claim 40, wherein said array of sensing photosensor elements is formed on said semiconductor substrate as a CMOS device.

42. The image sensing system of claim 39, wherein a lens images light external to the vehicle onto said imaging sensor and wherein said lens comprises a molded plastic lens.

43. The image sensing system of claim 39, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

44. The image sensing system of claim 39, wherein at least a portion of said logic and control circuit is commonly formed with said array of sensing photosensor elements on said semiconductor substrate as at least one of (i) a CMOS device and (ii) an integrated circuit.

45. The image sensing system of claim 39, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

46. The image sensing system of claim 39, wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

47. The image sensing system of claim 39, wherein said at least one control output for controlling said headlight of the vehicle comprises a pulse-width-modulated signal.

48. The image sensing system of claim 39, wherein said image sensing system controls switching said headlight between various modes, and wherein said various modes comprise a higher beam headlight mode and a lower beam headlight mode.

49. The image sensing system of claim 48, wherein said various modes comprise at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode.

50. The image sensing system of claim 48, wherein switching from a higher beam mode of said headlight to a lower beam mode takes less time than does switching from a corresponding lower beam mode to a corresponding higher beam mode.

51. The image sensing system of claim 39, wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of said headlight of the vehicle.

52. An image sensing system for a vehicle, said image sensing system comprising:

an imaging sensor;

said imaging sensor comprising a two-dimensional array of light sensing photosensor elements;

a logic and control circuit;

said logic and control circuit comprising an image processor for processing image data derived from said imaging sensor, said logic and control circuit generating at least one control output for controlling at least one light of the vehicle;

said imaging sensor disposed at an interior portion of the vehicle proximate the windshield of the vehicle, said imaging sensor having a forward field of view to the exterior of the vehicle through a windshield area that is swept by the windshield wipers;

wherein said at least one light of the vehicle comprises a headlight of the vehicle;

wherein said image sensing system determines a lighting configuration for the vehicle appropriate to improve the forward field of view of a driver of the vehicle;

wherein said image sensing system controls a lighting system of the vehicle in accordance with said determination of a lighting configuration for the vehicle;

wherein said lighting system comprises said headlight;

wherein said image sensing system determines an appropriate zone of interest for the lighting pattern of said headlight of the vehicle;

a portion of said array of light sensing photosensor elements capturing an image of at least one object of interest within said forward field of view of said imaging sensor; and

said logic and control circuit analyzing particular groupings of photosensor elements from said portion of said array while ignoring other particular groupings of photosensor elements from said portion of said array so as to determine at least one of presence, size, shape, contour and motion of said at least one object of interest, wherein said particular groupings of photosensor elements and said other particular groupings of photosensor elements are within a region of said array where an object of interest is expected to be imaged.

53. The image sensing system of claim 52, wherein said array of sensing photosensor elements is formed on a semiconductor substrate.

54. The image sensing system of claim 53, wherein said array of sensing photosensor elements is formed on said semiconductor substrate as a CMOS device.

55. The image sensing system of claim 53, wherein at least a portion of said logic and control circuit is commonly formed with said array of sensing photosensor elements on said semiconductor substrate as at least one of (i) a CMOS device and (ii) an integrated circuit.

56. The image sensing system of claim 52, wherein a lens images light external to the vehicle onto said imaging sensor and wherein said lens comprises a molded plastic lens.

57. The image sensing system of claim 52, wherein said interior portion is at or proximate to an interior rearview mirror assembly of the vehicle.

58. The image sensing system of claim 52, wherein said interior portion is generally centrally located along the vehicle axis and is relatively high in the interior of the vehicle.

59. The image sensing system of claim 52, wherein anti-blooming is provided to mitigate the effect of charge leakage from one of said photosensor elements to an adjacent one of said photosensor elements.

60. The image sensing system of claim 52, wherein said at least one control output for controlling said headlight of the vehicle comprises a pulse-width-modulated signal.

61. The image sensing system of claim 52, wherein said image sensing system controls switching said headlight between various modes, and wherein said various modes comprise a higher beam headlight mode and a lower beam headlight mode.

62. The image sensing system of claim 61, wherein said various modes comprise at least one intermediate headlight mode intermediate between a higher beam headlight mode and a lower beam headlight mode.

63. The image sensing system of claim 61, wherein switching from a higher beam mode of said headlight to a lower beam mode takes less time than does switching from a corresponding lower beam mode to a corresponding higher beam mode.

64. The image sensing system of claim 52, wherein said logic and control circuit comprises at least one of (i) an analog-to-digital converter, (ii) a logic circuit, (iii) a clock, (iv) random access memory, and (v) a digital-to-analog converter.