Vehicle vision system

• US 8,629,768 B2
• Filed: 06/18/2012
• Issued: 01/14/2014
• Est. Priority Date: 08/12/1999
• Status: Expired due to Fees

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Bos, Brent J., Schofield, Kenneth
• Primary Examiner(s): Lieu, Julie
• Application Number:
  US13/525,767
• Publication Number:
  US 20120257060A1
• Time in Patent Office: 575 Days
• Field of Search: None
• US Class Current: 340/461
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/0003 inside the vehicle
• CPC Class Codes: B60R 2011/0026 Windows, e.g. windscreen
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G01B 11/026 by measuring distance between sensor and object G01B11/0608 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 : from stereo images
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A vision system for a vehicle includes at least one imaging sensor having a forward field of view with respect to a forward direction of travel of the vehicle. The imaging sensor captures image data representative of objects present in the forward field of view. A control is responsive to the imaging sensor and processes image data captured by the imaging sensor to determine an object present in the forward field of view. The control processes image data to determine a distance between the vehicle and the object determined present in the forward field of view. The control may determine the distance between the vehicle and the determined object at least in part in response to at least one of (i) size of the determined object, (ii) position of the determined object, (iii) light intensity of the determined object, and (iv) rate of approach of the determined object.

Claims

1. A vision system for a vehicle, said vision system comprising:

at least one imaging sensor having a forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said at least one imaging sensor comprises a lens and a CMOS imaging array comprising a two-dimensional array of photosensing elements, and wherein said at least one imaging sensor captures image data representative of objects present in said forward field of view;

at least one register operable to access photosensing elements of said array of photosensing elements;

wherein said array of photosensing elements comprises a plurality of sub-arrays, said sub-arrays each having at least three neighboring photosensing elements;

wherein at least one photosensing element of each sub-array comprises a red light sensitive photosensing element and at least one other photosensing element of each sub-array comprises a white light sensitive photosensing element;

a control responsive to said at least one imaging sensor;

wherein said control processes image data captured by said at least one imaging sensor to determine an object present in said forward field of view;

wherein said control processes image data captured by said at least one imaging sensor to determine a distance between the equipped vehicle and the object determined present in said forward field of view; and

wherein said control discerns if the determined object is an oncoming vehicle or a leading vehicle at least in part in response to rate of approach of the determined object to the equipped vehicle.

2. The vision system of claim 1, wherein said control is operable to adjust a light beam emitted by at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one imaging sensor.

3. The vision system of claim 1, wherein said array of photosensing elements comprises at least 512 rows of photosensing elements and at least 512 columns of photosensing elements.

4. The vision system of claim 1, wherein said array of photosensing elements comprises at least 262,144 photosensing elements.

5. The vision system of claim 1, wherein the object determined present in said forward field of view comprises a headlight or a taillight of another vehicle and wherein said control is operable to adjust a light beam emitted by at least one headlamp of the equipped vehicle responsive to said processing of said image data.

6. The vision system of claim 1, wherein said control processes image data captured by said at least one imaging sensor to identify a headlamp or taillight of another vehicle present in said forward field of view.

7. The vision system of claim 6, wherein said control processes image data captured by said at least one imaging sensor and determines a distance between the equipped vehicle and the other vehicle.

8. The vision system of claim 7, wherein said control determines the distance between the equipped vehicle and the other vehicle at least in part in response to at least one of (i) size of the determined headlamp or taillight, (ii) position of the determined headlamp or taillight, (iii) light intensity of the determined headlamp or taillight, and (iv) rate of approach of the determined headlamp or taillight.

9. The vision system of claim 1, wherein red filtering is used with at least some of said photosensing elements, and wherein, responsive at least to processing of image data captured by said red filtered photosensing elements, said control is operable to at least partially distinguish the object from other objects present in said forward field of view.

10. The vision system of claim 9, wherein, responsive at least in part to processing of image data captured by said red filtered photosensing elements, said control is operable to distinguish taillights of other vehicles ahead of the equipped vehicle.

11. The vision system of claim 1, wherein said control adjusts a light beam emitted by at least one headlamp of the equipped vehicle responsive to said control (i) processing image data captured by said at least one imaging sensor and (ii) identifying a headlamp or taillight of another vehicle present in said forward field of view.

12. The vision system of claim 11, wherein said control adjusts the light beam emitted by the at least one headlamp of the equipped vehicle by adjusting the at least one headlamp between a first beam and a second beam.

13. The vision system of claim 12, wherein said first beam comprises a lower beam and said second beam comprises a higher beam.

14. The vision system of claim 1, wherein said control is operable to control a system of the equipped vehicle responsive to said processing of image data captured by said at least one imaging sensor.

15. The vision system of claim 14, wherein said controlled system comprises at least one of (i) a lighting system of the equipped vehicle, (ii) a brake system of the equipped vehicle and (iii) a driver assistance system of the equipped vehicle.

16. The vision system of claim 1, wherein said control controls a brake system of the equipped vehicle responsive to determining that the object determined present in said forward field of view is within a threshold distance of the equipped vehicle.

17. The vision system of claim 1, wherein said at least one imaging sensor is disposed behind a windshield of the equipped vehicle and said at least one imaging sensor views through the windshield of the equipped vehicle.

18. The vision system of claim 1, wherein said at least one imaging sensor comprises at least a first imaging sensor and a second imaging sensor, and wherein said first imaging sensor is spaced from said second imaging sensor.

19. The vision system of claim 18, wherein said first and second imaging sensors comprise imaging sensors of a stereoscopic imaging system of the equipped vehicle.

20. A vision system for a vehicle, said vision system comprising:

at least one imaging sensor having a forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said at least one imaging sensor comprises a lens and a CMOS imaging array comprising a two-dimensional array of photosensing elements, and wherein said at least one imaging sensor captures image data representative of objects present in said forward field of view;

at least one register operable to access photosensing elements of said array of photosensing elements;

wherein said array of photosensing elements comprises a plurality of sub-arrays, said sub-arrays each having at least three neighboring photosensing elements;

wherein at least one photosensing element of each sub-array comprises a red light sensitive photosensing element and at least one other photosensing element of each sub-array comprises a white light sensitive photosensing element;

a control responsive to said at least one imaging sensor;

wherein said control processes image data captured by said at least one imaging sensor to determine a headlamp or taillight of another vehicle present in said forward field of view;

wherein said control processes image data captured by said at least one imaging sensor to determine a distance between the equipped vehicle and the other vehicle; and

wherein said control discerns if the other vehicle is an oncoming vehicle or a leading vehicle at least in part in response to rate of approach of the determined headlamp or taillight to the equipped vehicle.

21. The vision system of claim 20, wherein said control is operable to adjust a light beam emitted by at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one imaging sensor.

22. The vision system of claim 20, wherein said array of photosensing elements comprises at least 262,144 photosensing elements.

23. The vision system of claim 20, wherein red filtering is used with at least some of said photosensing elements, and wherein, responsive at least to processing of image data captured by said red filtered photosensing elements, said control is operable to at least partially distinguish the headlamp or taillight of the other vehicle from other objects present in said forward field of view.

24. A vision system for a vehicle, said vision system comprising:

at least one imaging sensor having a forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said at least one imaging sensor comprises a lens and a CMOS imaging array comprising a two-dimensional array of at least 262,144 photosensing elements, and wherein said at least one imaging sensor captures image data representative of objects present in said forward field of view;

at least one register operable to access photosensing elements of said array of photosensing elements;

wherein said array of photosensing elements comprises a plurality of sub-arrays, said sub-arrays each having at least three neighboring photosensing element;

wherein at least one photosensing element of each sub-array comprises a red light sensitive photosensing element and at least one other photosensing element of each sub-array comprises a white light sensitive photosensing element;

a control responsive to said at least one imaging sensor;

wherein said control processes image data captured by said at least one imaging sensor to determine a headlamp or taillight of another vehicle present in said forward field of view;

wherein said control processes image data captured by said at least one imaging sensor to determine a distance between the equipped vehicle and the other vehicle;

wherein red filtering is used with at least some of said photosensing elements, and wherein, responsive at least to processing of image data captured by said red filtered photosensing elements, said control is operable to at least partially distinguish the determined headlamp or taillight of the other vehicle from other objects present in said forward field of view; and

wherein said control discerns if the other is an oncoming vehicle or a leading vehicle at least in part in response to rate of approach of the determined headlamp or taillight to the equipped vehicle.

25. The vision system of claim 24, wherein said control is operable to adjust a light beam emitted by at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one imaging sensor.

26. A vision system for a vehicle, said vision system comprising:

at least one imaging sensor having a forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said at least one imaging sensor comprises a lens and a CMOS imaging array comprising a two-dimensional array of photosensing elements, and wherein said at least one imaging sensor captures image data representative of objects present in said forward field of view;

at least one register operable to access photosensing elements of said array of photosensing elements;

wherein said array of photosensing elements comprises a plurality of sub-arrays, said sub-arrays each having at least three neighboring photosensing elements;

wherein at least one photosensing element of each sub-array comprises a red light sensitive photosensing element and at least one other photosensing element of each sub-array comprises a white light sensitive photosensing element;

wherein said at least one imaging sensor is disposed behind a windshield of the equipped vehicle and said at least one imaging sensor views through the windshield of the equipped vehicle;

a control responsive to said at least one imaging sensor;

wherein said control processes image data captured by said at least one imaging sensor to determine a headlamp or taillight of another vehicle present in said forward field of view;

wherein said control processes image data captured by said at least one imaging sensor to determine a distance between the equipped vehicle and the other vehicle; and

wherein said control discerns if the determined object is an oncoming vehicle or a leading vehicle at least in part in response to rate of approach of the determined headlamp or taillight to the equipped vehicle.

27. The vision system of claim 26, wherein said array of photosensing elements comprises at least 262,144 photosensing elements.

28. The vision system of claim 26, wherein said control is operable to adjust a light beam emitted by at least one headlamp of the equipped vehicle responsive to said processing of said image data.

29. The vision system of claim 26, wherein red filtering is used with at least some of said photosensing elements, and wherein, responsive at least to processing of image data captured by said red filtered photosensing elements, said control is operable to at least partially distinguish the object from other objects present in said forward field of view.

30. The vision system of claim 29, wherein, responsive at least in part to processing of image data captured by said red filtered photosensing elements, said control is operable to distinguish taillights of other vehicles ahead of the equipped vehicle.

31. The vision system of claim 26, wherein said control adjusts a light beam emitted by at least one headlamp of the equipped vehicle responsive to said control (i) processing image data captured by said at least one imaging sensor and (ii) identifying a headlamp or taillight of another vehicle present in said forward field of view.

32. The vision system of claim 26, wherein said control is operable to control a system of the equipped vehicle responsive to said processing of image data captured by said at least one imaging sensor, and wherein said controlled system comprises at least one of (i) a lighting system of the equipped vehicle, (ii) a brake system of the equipped vehicle and (iii) a driver assistance system of the equipped vehicle.