MULTI-CAMERA VISION SYSTEM FOR A VEHICLE

• US 20130250046A1
• Filed: 05/15/2013
• Published: 09/26/2013
• Est. Priority Date: 05/22/1996
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Larson, Mark L., Vadas, Keith J.
• Granted Patent: US 9,131,120 B2
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 2300/103 using camera systems provided with artificial illumination device, e.g. IR light source
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/107 using stereoscopic cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A multi-camera vision system for a vehicle includes first, second and third image capture devices having respective first, second and third fields of view exterior of the vehicle. The first field of view overlaps with the third field of view defining a first overlap zone, and the second field of view overlaps with the third field of view defining a second overlap zone. Responsive to processing of captured image data, a synthesized image is generated without duplication of objects present in the first and second overlap zones and the synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the vehicle. The synthesized image is displayed by a single display screen that is viewable by a driver of the vehicle and the synthesized image assists the driver in executing a maneuver of the vehicle with a trailer attached thereto.

Claims

1. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

at least three image capture devices disposed at a vehicle equipped with said multi-camera vision system;

said at least three image capture devices comprising a first image capture device disposed at a driver-side portion of the equipped vehicle at a first location;

said at least three image capture devices comprising a second image capture device disposed at a passenger-side portion of the equipped vehicle at a second location;

said at least three image capture devices comprising a third image capture device disposed at a rear portion of the equipped vehicle at a third location;

wherein said first image capture device has a first field of view exterior of the equipped vehicle;

wherein said second image capture device has a second field of view exterior of the equipped vehicle;

wherein said third image capture device has a third field of view exterior of the equipped vehicle;

wherein said first field of view of said first image capture device overlaps with said third field of view of said third image capture device defining a first overlap zone;

wherein said second field of view of said second image capture device overlaps with said third field of view of said third image capture device defining a second overlap zone;

wherein said first image capture device captures first image data;

wherein said second image capture device captures second image data;

wherein said third image capture device captures third image data;

an image processor;

wherein first image data captured by said first image capture device is received at said image processor;

wherein second image data captured by said second image capture device is received at said image processor;

wherein third image data captured by said third image capture device is received at said image processor;

wherein, responsive to processing by said image processor of received image data, a synthesized image is generated without duplication of objects present in said first overlap zone and in said second overlap zone and wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by a single display screen of a display device that is viewable by a driver of the equipped vehicle when normally operating the equipped vehicle;

wherein a trailer is attached at the rear of the equipped vehicle; and

wherein said synthesized image assists the driver of the equipped vehicle in executing a maneuver of the equipped vehicle with the trailer attached thereto.

2. The multi-camera vision system of claim 1, wherein first image data captured by said first image capture device is received at said image processor via a digital data stream and wherein second image data captured by said second image capture device is received at said image processor via a digital data stream and wherein third image data captured by said third image capture device is received at said image processor via a digital data stream, and wherein image data transmission from said first, second and third image capture devices to said image processor is by electrically conductive leads over a vehicle communication bus.

3. The multi-camera vision system of claim 2, wherein said vehicle communication bus comprises one of a serial bus and a parallel bus.

4. The multi-camera vision system of claim 1, wherein the image displayed by said display device includes a visual indication of the location of the equipped vehicle in said view and wherein at least one of (i) said visual indication approximates the footprint occupied by the equipped vehicle and (ii) said visual indication comprises an outline of an area substantially occupied by the equipped vehicle.

5. The multi-camera vision system of claim 1, wherein said first image capture device is disposed at the equipped vehicle at substantially the same height relative to ground as is said second image capture device, and wherein said first location where said first image capture device is disposed at the driver-side of the equipped vehicle is a first distance forward of where the driver sits when operating the equipped vehicle, and wherein said second location where said second image capture device is disposed at the passenger-side of the equipped vehicle is a second distance forward of where the driver sits when operating the equipped vehicle, and wherein said first distance is substantially the same as said second distance, and wherein said third image capture device is at a height relative to ground that is lower than the height relative to ground of said first and second image capture devices, and wherein said third location is a third distance rearward of where the driver sits when operating the equipped vehicle, and wherein said third distance is larger than either of said first and second distances.

6. The multi-camera vision system of claim 1, wherein the image displayed by said display screen comprises a graphic overlay and wherein at least one of (i) said graphic overlay comprises indicia of the anticipated path of travel of the equipped vehicle, (ii) said graphic overlay comprises indicia indicating distance to objects exterior the equipped vehicle, (iii) said graphic overlay comprises enhancements that assist visually distinguishing objects close to the equipped vehicle, (iv) said graphic overlay comprises indicia that responds to the equipped vehicle'"'"'s steering system, (v) said graphic overlay draws the attention of the driver of the equipped vehicle to a potential hazard present in the image displayed by said display device, and (vi) said graphic overlay includes indicia of the anticipated path of travel of the equipped vehicle.

7. The multi-camera vision system of claim 1, wherein said display device comprises a reconfigurable display device and wherein said reconfigurable display device comprises a flat-panel a back-lit liquid crystal display device.

8. The multi-camera vision system of claim 1, wherein said multi-camera vision system provides guidance when backing the equipped vehicle with the trailer attached thereto.

9. The multi-camera vision system of claim 8, wherein said multi-camera vision system provides guidance when backing the equipped vehicle into a loading dock with the trailer attached thereto.

10. The multi-camera vision system of claim 1, wherein at least one of (i) said synthesized image displayed by said display device approximates a panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle and (ii) said synthesized image displayed by said display device approximates a substantially seamless panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle.

11. The multi-camera vision system of claim 1, wherein said maneuver of the equipped vehicle with the trailer attached thereto comprises a backing maneuver of the equipped vehicle and wherein said synthesized image minimizes the consequence of at least partial blockage by the attached trailer of a rearward view of at least one of said first image capture device and said second image capture device.

12. The multi-camera vision system of claim 11, wherein said synthesized image changes when the rearward view of at least one of said first image capture device and said second image capture device is not blocked by the trailer attached at the rear of the equipped vehicle.

13. The multi-camera vision system of claim 1, wherein first image data captured by said first image capture device is received at said image processor via an analog data stream and wherein second image data captured by said second image capture device is received at said image processor via an analog data stream and wherein third image data captured by said third image capture device is received at said image processor via an analog data stream, and wherein image data transmission from said first, second and third image capture devices to said image processor is by electrically conductive leads.

14. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

at least three image capture devices disposed at a vehicle equipped with said multi-camera vision system;

said at least three image capture devices comprising a first CMOS image capture device disposed at a driver-side portion of the equipped vehicle at a first location;

said at least three image capture devices comprising a second CMOS image capture device disposed at a passenger-side portion of the equipped vehicle at a second location;

said at least three image capture devices comprising a third CMOS image capture device disposed at a rear portion of the equipped vehicle at a third location;

wherein said first CMOS image capture device has a first field of view exterior of the equipped vehicle;

wherein said second CMOS image capture device has a second field of view exterior of the equipped vehicle;

wherein said third CMOS image capture device has a third field of view exterior of the equipped vehicle;

wherein said first field of view of said first CMOS image capture device overlaps with said third field of view of said third CMOS image capture device defining a first overlap zone;

wherein said second field of view of said second CMOS image capture device overlaps with said third field of view of said third CMOS image capture device defining a second overlap zone;

wherein said first CMOS image capture device captures first image data;

wherein said second CMOS image capture device captures second image data;

wherein said third CMOS image capture device captures third image data;

an image processor;

wherein first image data captured by said first CMOS image capture device is received at said image processor via at least one of an analog data stream and a digital data stream;

wherein second image data captured by said second CMOS image capture device is received at said image processor via at least one of an analog data stream and a digital data stream;

wherein third image data captured by said third CMOS image capture device is received at said image processor via at least one of an analog data stream and a digital data stream;

wherein, responsive to processing by said image processor of received image data, a synthesized image is generated without duplication of objects present in said first overlap zone and in said second overlap zone and wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by a single display screen of a display device that is viewable by a driver of the equipped vehicle when normally operating the equipped vehicle;

wherein a trailer is attached at the rear of the equipped vehicle;

wherein said synthesized image assists the driver of the equipped vehicle in executing a maneuver of the equipped vehicle with the trailer attached thereto; and

wherein at least one of (i) said multi-camera vision system provides guidance when backing the equipped vehicle with the trailer attached thereto and (ii) said multi-camera vision system provides guidance when backing the equipped vehicle into a loading dock with the trailer attached thereto.

15. The multi-camera vision system of claim 14, wherein said first CMOS image capture device is disposed at the equipped vehicle at substantially the same height relative to ground as is said second CMOS image capture device, and wherein said first location where said first CMOS image capture device is disposed at the driver-side of the equipped vehicle is a first distance forward of where the driver sits when operating the equipped vehicle, and wherein said second location where said second CMOS image capture device is disposed at the passenger-side of the equipped vehicle is a second distance forward of where the driver sits when operating the equipped vehicle, and wherein said first distance is substantially the same as said second distance, and wherein said third CMOS image capture device is at a height relative to ground that is lower than the height relative to ground of said first and second CMOS image capture devices, and wherein said third location is a third distance rearward of where the driver sits when operating the equipped vehicle, and wherein said third distance is larger than either of said first and second distances.

16. The multi-camera vision system of claim 15, wherein the image displayed by said display screen comprises a graphic overlay and wherein at least one of (i) said graphic overlay comprises indicia of the anticipated path of travel of the equipped vehicle, (ii) said graphic overlay comprises indicia indicating distance to objects exterior the equipped vehicle, (iii) said graphic overlay comprises enhancements that assist visually distinguishing objects close to the equipped vehicle, (iv) said graphic overlay comprises indicia that responds to the equipped vehicle'"'"'s steering system, (v) said graphic overlay draws the attention of the driver of the equipped vehicle to a potential hazard present in the image displayed by said display device, and (vi) said graphic overlay includes indicia of the anticipated path of travel of the equipped vehicle.

17. The multi-camera vision system of claim 16, wherein said display device comprises a reconfigurable display device and wherein said reconfigurable display device comprises a flat-panel a back-lit liquid crystal display device.

18. The multi-camera vision system of claim 14, wherein said multi-camera vision system provides guidance when backing the equipped vehicle into a loading dock with the trailer attached thereto.

19. The multi-camera vision system of claim 14, wherein at least one of (i) said synthesized image displayed by said display device approximates a panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle and (ii) said synthesized image displayed by said display device approximates a substantially seamless panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle.

20. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

at least three image capture devices disposed at a vehicle equipped with said multi-camera vision system;

said at least three image capture devices comprising a first CMOS image capture device disposed at a driver-side portion of the equipped vehicle at a first location;

said at least three image capture devices comprising a second CMOS image capture device disposed at a passenger-side portion of the equipped vehicle at a second location;

said at least three image capture devices comprising a third CMOS image capture device disposed at a rear portion of the equipped vehicle at a third location;

wherein said first CMOS image capture device has a first field of view exterior of the equipped vehicle;

wherein said second CMOS image capture device has a second field of view exterior of the equipped vehicle;

wherein said third CMOS image capture device has a third field of view exterior of the equipped vehicle;

wherein said first field of view of said first CMOS image capture device overlaps with said third field of view of said third CMOS image capture device defining a first overlap zone;

wherein said second field of view of said second CMOS image capture device overlaps with said third field of view of said third CMOS image capture device defining a second overlap zone;

wherein said first CMOS image capture device captures first image data;

wherein said second CMOS image capture device captures second image data;

wherein said third CMOS image capture device captures third image data;

an image processor;

wherein first image data captured by said first CMOS image capture device is received at said image processor;

wherein second image data captured by said second CMOS image capture device is received at said image processor;

wherein third image data captured by said third CMOS image capture device is received at said image processor;

wherein, responsive to processing by said image processor of received image data, a synthesized image is generated without duplication of objects present in said first overlap zone and in said second overlap zone and wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by a single display screen of a display device that is viewable by a driver of the equipped vehicle when normally operating the equipped vehicle;

wherein a trailer is attached at the rear of the equipped vehicle;

wherein said synthesized image assists the driver of the equipped vehicle in executing a maneuver of the equipped vehicle with the trailer attached thereto;

wherein said first CMOS image capture device is disposed at the equipped vehicle at substantially the same height relative to ground as is said second CMOS image capture device, and wherein said first location where said first CMOS image capture device is disposed at the driver-side of the equipped vehicle is a first distance forward of where the driver sits when operating the equipped vehicle, and wherein said second location where said second CMOS image capture device is disposed at the passenger-side of the equipped vehicle is a second distance forward of where the driver sits when operating the equipped vehicle, and wherein said first distance is substantially the same as said second distance and wherein said third CMOS image capture device is at a height relative to ground that is lower than the height relative to ground of said first and second CMOS image capture devices, and wherein said third location is a third distance rearward of where the driver sits when operating the equipped vehicle, and wherein said third distance is larger than either of said first and second distances;

wherein the image displayed by said display screen comprises a graphic overlay and wherein at least one of (i) said graphic overlay comprises indicia of the anticipated path of travel of the equipped vehicle, (ii) said graphic overlay comprises indicia indicating distance to objects exterior the equipped vehicle, (iii) said graphic overlay comprises enhancements that assist visually distinguishing objects close to the equipped vehicle, (iv) said graphic overlay comprises indicia that responds to the equipped vehicle'"'"'s steering system, (v) said graphic overlay draws the attention of the driver of the equipped vehicle to a potential hazard present in the image displayed by said display device, and (vi) said graphic overlay includes indicia of the anticipated path of travel of the equipped vehicle; and

wherein at least one of (i) said synthesized image displayed by said display device approximates a panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle and (ii) said synthesized image displayed by said display device approximates a substantially seamless panoramic view as would be viewed from a single virtual camera located exterior the equipped vehicle.

21. The multi-camera vision system of claim 20, wherein said display device comprises a reconfigurable display device and wherein said reconfigurable display device comprises a flat-panel a back-lit liquid crystal display device.

22. The multi-camera vision system of claim 20, wherein said multi-camera vision system provides guidance when backing the equipped vehicle into a loading dock with the trailer attached thereto.