VEHICLE VISION SYSTEM

• US 20120053795A1
• Filed: 11/09/2011
• Published: 03/01/2012
• Est. Priority Date: 08/12/1999
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Bos, Brent J., Schofield, Kenneth
• Granted Patent: US 8,203,443 B2
• Time in Patent Office: Days
• US Class Current: 701/48
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/0003 inside the vehicle
• CPC Class Codes: B60R 2011/0026 Windows, e.g. windscreen
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G01B 11/026 by measuring distance between sensor and object G01B11/0608 takes precedence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10024 : Color image
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/593 : from stereo images
• CPC Class Codes: G06T 7/97 : Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 : Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 25/00 : Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A vision system for a vehicle includes a first imaging sensor having a first forward field of view and a second imaging sensor spaced from the first imaging sensor and having a second forward field of view, which at least partially overlaps with the first forward field of view. A control processes image data captured by at least one of the first and second imaging sensors to determine an object present in the first and/or second forward fields of view. The control is operable to modulate a headlamp of the vehicle responsive to the processing of image data captured by the at least one of the first and second imaging sensors. The control may process image data captured by both of the imaging sensors to determine a distance between the equipped vehicle and an object present in the overlap of the first and second forward fields of view.

Claims

1. A vision system for a vehicle, said vision system comprising:

a first imaging sensor having a first forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said first imaging sensor comprises a first two-dimensional array of photosensing elements and a first lens, and wherein said first imaging sensor images objects present in said first forward field of view;

a second imaging sensor spaced from said first imaging sensor and having a second forward field of view with respect to the forward direction of travel of the equipped vehicle, wherein said second imaging sensor comprises a second two-dimensional array of photosensing elements and a second lens, and wherein said second forward field of view at least partially overlaps with said first forward field of view, and wherein said second imaging sensor images objects in said second forward field of view;

a control responsive to said first and second imaging sensors;

wherein (a) said control processes image data captured by at least one of said first and second imaging sensors to determine an object present in at least one of said first and second forward fields of view and (b) said control is operable to modulate at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one of said first and second imaging sensors; and

wherein (i) said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and an object present in said overlap of said first forward field of view with said second forward field of view and (ii) said control is operable to control a brake system of the equipped vehicle responsive to said processing of image data captured by both of said first and second imaging sensors.

2. The vision system of claim 1, wherein the object determined to be present in at least one of said first and second forward fields of view comprises a headlight or a taillight of another vehicle and wherein said control is operable to modulate the at least one headlamp of the equipped vehicle responsive to determination of the headlight or a taillight of another vehicle being present in at least one of said first and second forward fields of view.

3. The vision system of claim 1, wherein said control processes image data captured by at least one of said first and second imaging sensors to identify a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

4. The vision system of claim 3, wherein said control processes image data captured by both of said first and second imaging sensors and determines a distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view.

5. The vision system of claim 4, wherein said control processes image data captured by both of said first and second imaging sensors and, responsive to a size of the imaged headlamp or taillight, determines the distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view.

6. The vision system of claim 3, wherein said control processes image data captured by both of said first and second imaging sensors and determines a distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view in response to at least one of (i) a size of the imaged headlamp or taillight, (ii) a position of the imaged headlamp or taillight, (iii) an intensity of the imaged headlamp or taillight, and (iv) a rate of approach of the imaged headlamp or taillight.

7. The vision system of claim 3, wherein said at least one of said first and second imaging sensors comprises a pixelated array of photosensing pixels, and wherein red filtering is used with at least some of said photosensing pixels, and wherein, responsive at least to processing of image data captured by said red filtered photosensing pixels, said control is operable to at least partially distinguish a red colored object from other objects.

8. The vision system of claim 7, wherein, responsive to processing of image data captured by said red filtered photosensing pixels, said control is operable to distinguish taillights of other vehicles ahead of the equipped vehicle.

9. The vision system of claim 1, wherein said first imaging sensor comprises a pixelated array of photosensing pixels and wherein spectral filtering is used with at least some of said photosensing pixels.

10. The vision system of claim 1, wherein said control processes image data captured by at least one of said first and second imaging sensors to determine that an object of interest to the driver of the equipped vehicle is present in at least one of said first and second forward fields of view.

11. The vision system of claim 10, wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and the determined object of interest when the determined object of interest is present in said overlap of said first forward field of view with said second forward field of view.

12. The vision system of claim 1, wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and at least one of (i) a headlamp or taillight of another vehicle present in said overlap of said first forward field of view with said second forward field of view and (ii) an object of interest to the driver of the equipped vehicle present in said overlap of said first forward field of view with said second forward field of view.

13. The vision system of claim 1, wherein said control modulates at least one headlamp of the equipped vehicle responsive to said control (i) processing image data captured by at least one of said first and second imaging sensors and (ii) identifying a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

14. The vision system of claim 13, wherein said control modulates the at least one headlamp of the equipped vehicle by switching the at least one headlamp between a first beam and a second beam.

15. The vision system of claim 14, wherein said first beam comprises a lower beam and said second beam comprises a higher beam.

16. The vision system of claim 1, wherein said control controls a brake system of the equipped vehicle responsive to said control processing image data captured by both of said first and second imaging sensors and determining that an object present in said overlap of said first forward field of view with said second forward field of view is within a threshold distance of the equipped vehicle.

17. The vision system of claim 1, wherein said first array of photosensing elements comprises at least 512 rows of photosensing elements and said second array of photosensing elements comprises at least 512 rows of photosensing elements.

18. The vision system of claim 17, wherein said first array of photosensing elements comprises at least 512 columns of photosensing elements and said second array of photosensing elements comprises at least 512 columns of photosensing elements.

19. The vision system of claim 1, wherein said first and second imaging sensors are disposed behind a windshield of the equipped vehicle and said first and second imaging sensors view through the windshield of the equipped vehicle.

20. A vision system for a vehicle, said vision system comprising:

a first imaging sensor having a first forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said first imaging sensor comprises a first two-dimensional array of photosensing elements and a first lens, and wherein said first imaging sensor comprises a pixelated array of photosensing pixels and wherein spectral filtering is used with at least some of said photosensing pixels, and wherein said first imaging sensor images objects present in said first forward field of view;

a second imaging sensor spaced from said first imaging sensor and having a second forward field of view with respect to the forward direction of travel of the equipped vehicle, wherein said second imaging sensor comprises a second two-dimensional array of photosensing elements and a second lens, and wherein said second forward field of view at least partially overlaps with said first forward field of view, and wherein said second imaging sensor images objects in said second forward field of view;

wherein said first and second imaging sensors are disposed behind a windshield of the equipped vehicle and said first and second imaging sensors view through the windshield of the equipped vehicle;

a control responsive to said first and second imaging sensors;

wherein (a) said control processes image data captured by at least one of said first and second imaging sensors to determine an object present in at least one of said first and second forward fields of view and (b) said control is operable to modulate at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one of said first and second imaging sensors;

wherein (i) said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and an object present in said overlap of said first forward field of view with said second forward field of view and (ii) said control is operable to control a brake system of the equipped vehicle responsive to said processing of image data captured by both of said first and second imaging sensors; and

wherein said control processes image data captured by at least one of said first and second imaging sensors to identify a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

21. The vision system of claim 20, wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view.

22. The vision system of claim 20, wherein said spectral filtering comprises red filtering and wherein, responsive at least to processing of image data captured by said red filtered photosensing pixels, said control is operable to at least partially distinguish taillights of other vehicles ahead of the equipped vehicle.

23. The vision system of claim 20, wherein said control processes image data captured by at least one of said first and second imaging sensors to determine that an object of interest to the driver of the equipped vehicle is present in at least one of said first and second forward fields of view, and wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and the determined object of interest when the determined object of interest is present in said overlap of said first forward field of view with said second forward field of view.

24. The vision system of claim 20, wherein said control modulates at least one headlamp of the equipped vehicle responsive to said control (i) processing image data captured by at least one of said first and second imaging sensors and (ii) identifying a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

25. The vision system of claim 20, wherein said control controls a brake system of the equipped vehicle responsive to said control processing image data captured by both of said first and second imaging sensors and determining that an object present in said overlap of said first forward field of view with said second forward field of view is within a threshold distance of the equipped vehicle.

26. A vision system for a vehicle, said vision system comprising:

a first imaging sensor having a first forward field of view with respect to a forward direction of travel of a vehicle equipped with said vision system, wherein said first imaging sensor comprises a first two-dimensional array of photosensing elements and a first lens, and wherein said first imaging sensor images objects present in said first forward field of view;

a second imaging sensor spaced from said first imaging sensor and having a second forward field of view with respect to the forward direction of travel of the equipped vehicle, wherein said second imaging sensor comprises a second two-dimensional array of photosensing elements and a second lens, and wherein said second forward field of view at least partially overlaps with said first forward field of view, and wherein said second imaging sensor images objects in said second forward field of view;

wherein said first and second imaging sensors are disposed behind a windshield of the equipped vehicle and said first and second imaging sensors view through the windshield of the equipped vehicle;

a control responsive to said first and second imaging sensors; and

wherein (a) said control processes image data captured by at least one of said first and second imaging sensors to determine that an object of interest to the driver of the equipped vehicle is present in at least one of said first and second forward fields of view and (b) said control is operable to modulate at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one of said first and second imaging sensors.

27. The vision system of claim 26, wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and the object of interest when the object of interest is present in said overlap of said first forward field of view with said second forward field of view.

28. The vision system of claim 27, wherein said control is operable to control a brake system of the equipped vehicle responsive to said processing of image data captured by both of said first and second imaging sensors.

29. The vision system of claim 28, wherein said control is operable to control said brake system responsive to determining that the object of interest is present in said overlap of said first forward field of view with said second forward field of view and is within a threshold distance of the equipped vehicle.

30. The vision system of claim 26, wherein said first imaging sensor comprises a pixelated array of photosensing pixels and wherein spectral filtering is used with at least some of said photosensing pixels, and wherein said spectral filtering comprises red filtering and wherein, responsive at least to processing of image data captured by said red filtered photosensing pixels, said control is operable to at least partially distinguish a red object from other objects.

31. A vision system for a vehicle, said vision system comprising:

a first imaging sensor having a first forward field of view with respect to, a forward direction of travel of a vehicle equipped with said vision system, wherein said first imaging sensor comprises a first two-dimensional array of photosensing elements and a first lens, and wherein said first imaging sensor images objects present in said first forward field of view;

a second imaging sensor spaced from said first imaging sensor and having a second forward field of view with respect to the forward direction of travel of the equipped vehicle, wherein said second imaging sensor comprises a second two-dimensional array of photosensing elements and a second lens, and wherein said second forward field of view at least partially overlaps with said first forward field of view, and wherein said second imaging sensor images objects in said second forward field of view;

wherein said first and second imaging sensors are disposed behind a windshield of the equipped vehicle and said first and second imaging sensors view through the windshield of the equipped vehicle;

a control responsive to said first and second imaging sensors;

wherein (a) said control processes image data captured by at least one of said first and second imaging sensors to determine an object present in at least one of said first and second forward fields of view and (b) said control is operable to modulate at least one headlamp of the equipped vehicle responsive to said processing of image data captured by said at least one of said first and second imaging sensors; and

wherein said control processes image data captured by at least one of said first and second imaging sensors to identify a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

32. The vision system of claim 31, wherein said control processes image data captured by both of said first and second imaging sensors and determines a distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view.

33. The vision system of claim 31, wherein said control processes image data captured by both of said first and second imaging sensors and determines a distance between the equipped vehicle and the identified headlamp or taillight when the identified headlamp or taillight is present in said overlap of said first forward field of view with said second forward field of view in response to at least one of (i) a size of the imaged headlamp or taillight, (ii) a position of the imaged headlamp or taillight, (iii) an intensity of the imaged headlamp or taillight, and (iv) a rate of approach of the imaged headlamp or taillight.

34. The vision system of claim 31, wherein said at least one of said first and second imaging sensors comprises a pixelated array of photosensing pixels, and wherein red filtering is used with at least some of said photosensing pixels, and wherein, responsive at least to processing of image data captured by said red filtered photosensing pixels, said control is operable to at least partially distinguish a red colored object from other objects.

35. The vision system of claim 34, wherein, responsive to processing of image data captured by said red filtered photosensing pixels, said control is operable to distinguish taillights of other vehicles ahead of the equipped vehicle.

36. The vision system of claim 31, wherein said first imaging sensor comprises a pixelated array of photosensing pixels and wherein spectral filtering is used with at least some of said photosensing pixels.

37. The vision system of claim 31, wherein said control processes image data captured by at least one of said first and second imaging sensors to determine that an object of interest to the driver of the equipped vehicle is present in at least one of said first and second forward fields of view, and wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and the determined object of interest when the determined object of interest is present in said overlap of said first forward field of view with said second forward field of view.

38. The vision system of claim 31, wherein said control processes image data captured by both of said first and second imaging sensors to determine a distance between the equipped vehicle and at least one of (i) a headlamp or taillight of another vehicle present in said overlap of said first forward field of view with said second forward field of view and (ii) an object of interest to the driver of the equipped vehicle present in said overlap of said first forward field of view with said second forward field of view.

39. The vision system of claim 31, wherein said control modulates at least one headlamp of the equipped vehicle responsive to said control (i) processing image data captured by at least one of said first and second imaging sensors and (ii) identifying a headlamp or taillight of another vehicle present in at least one of said first and second forward fields of view.

40. The vision system of claim 39, wherein said control modulates the at least one headlamp of the equipped vehicle by switching the at least one headlamp between a first beam and a second beam.

41. The vision system of claim 31, wherein said control controls a brake system of the equipped vehicle responsive to said control processing image data captured by both of said first and second imaging sensors and determining that an object present in said overlap of said first forward field of view with said second forward field of view is within a threshold distance of the equipped vehicle.