VEHICULAR MULTI-CAMERA VISION SYSTEM INCLUDING REAR BACKUP CAMERA

• US 20200172013A1
• Filed: 02/10/2020
• Published: 06/04/2020
• Est. Priority Date: 09/14/2004
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Karner, Lee, Schofield, Kenneth, Lynam, Niall R.
• Granted Patent: US 10,703,274 B2
• Time in Patent Office: Days
• CPC Class Codes: B29C 45/0017 moulding interconnected elements which are movable with respect to one another, e.g. chains or hinges
• CPC Class Codes: B29C 45/14639 for obtaining an insulating effect, e.g. for electrical components
• CPC Class Codes: B29C 45/14754 being in movable or releasable engagement with the coating, e.g. bearing assemblies
• CPC Class Codes: B29C 45/1671 with an insert
• CPC Class Codes: B29L 2011/0058 Mirrors
• CPC Class Codes: B29L 2031/30 Vehicles, e.g. ships or aircraft, or body parts thereof vanes or blades B29L2031/08, air bags B29L2022/027
• CPC Class Codes: B60C 9/00 Reinforcements or ply arrangement of pneumatic tyres inserts having reinforcing means B60C5/08; bead structure, e.g. turnup or overlap construction, B60C15/00
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/04 mounted inside vehicle B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/06 mounted on vehicle exterior B60R1/025, B60R1/08 take precedence
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1215 with information displays
• CPC Class Codes: B60R 2001/1223 with sensors or transducers
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2001/1269 with sun visors
• CPC Class Codes: B60R 2011/0033 Rear-view mirrors
• CPC Class Codes: B60R 2300/105 : using multiple cameras
• CPC Class Codes: B60R 2300/308 : by overlaying the real scene, e.g. through a head-up display on the windscreen
• CPC Class Codes: B60R 2300/8026 : in addition to a rear-view mirror system
• CPC Class Codes: B60R 2300/8066 : for monitoring rearward traffic
• CPC Class Codes: B60R 2300/808 : for facilitating docking to a trailer
• CPC Class Codes: B60R 2300/8093 : for obstacle warning
• CPC Class Codes: G06T 2207/10016 : Video; Image sequence
• CPC Class Codes: G06T 2207/30232 : Surveillance
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/70 : Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: G09G 2300/023 : Display panel composed of stacked panels
• CPC Class Codes: G09G 2300/0456 : with a reflective area and a transmissive area combined in one pixel, such as in transflectance pixels
• CPC Class Codes: G09G 2340/12 : Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2380/10 : Automotive applications
• CPC Class Codes: G09G 3/001 : using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/346 : based on modulation of the reflection angle, e.g. micromirrors micromirrors devices per se G02B26/0833
• CPC Class Codes: G09G 5/026 : Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A multi-camera vision system for a vehicle includes a rear backup camera, a forward viewing camera that views through the vehicle windshield, and a central data processor disposed in the vehicle. Captured image data provided to the central data processor is processed to detect objects that are within the field of view of the forward viewing camera and/or the field of view of the rear backup camera. Captured radar data is also provided to the central data processor and, via processing of captured image data and captured radar data, a potential hazard is determined to exist. A display device includes a video screen that displays video images captured by the rear backup camera for viewing by a driver of the vehicle during the rearward maneuver of the vehicle, and an overlay overlays the displayed video images captured by the rear backup camera during the rearward maneuver of the equipped vehicle.

Claims

1. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

a rear backup camera at a rear portion of a vehicle equipped with said multi-camera vision system;

wherein said rear backup camera comprises a CMOS imaging array;

said rear backup camera having a field of view at least rearward of the equipped vehicle;

wherein the field of view of said rear backup camera encompasses at least a portion of a vehicle bumper of the equipped vehicle;

said rear backup camera captures video image data at least rearward of the equipped vehicle during a rearward maneuver of the equipped vehicle;

a forward viewing camera disposed at a windshield of the equipped vehicle and viewing through the windshield forward of the equipped vehicle;

said forward viewing camera captures video image data at least forward of the equipped vehicle;

a central data processor disposed in the equipped vehicle;

wherein image data captured by said forward viewing camera is provided to said central data processor and image data captured by said rear backup camera is provided to said central data processor;

wherein captured image data provided to said central data processor is processed at said central data processor in order to detect objects that are within at least one selected from the group consisting of (i) the field of view of said forward viewing camera and (ii) the field of view of said rear backup camera;

wherein detection of objects via processing at said central data processor of provided captured image data comprises use of an edge detection algorithm;

wherein the equipped vehicle is further equipped with at least one radar sensor;

wherein said at least one radar sensor captures radar data in a region external of the equipped vehicle;

wherein captured radar data is provided to said central data processor;

wherein, via processing at said central data processor of at least one of provided captured image data and provided captured radar data, a potential hazard is determined to exist;

a display device comprising a video screen that displays video images for viewing by a driver of the equipped vehicle during the rearward maneuver of the equipped vehicle, wherein image data captured by said rear backup camera is processed at said central data processor to generate video images for display on said video screen; and

wherein an overlay overlays displayed video images captured by said rear backup camera during the rearward maneuver of the equipped vehicle.

2. The multi-camera vision system of claim 1, wherein data is provided to said central data processor via a vehicle network of the equipped vehicle.

3. The multi-camera vision system of claim 1, wherein, via processing at said central data processor of provided captured image data captured by said rear backup camera, the potential hazard is determined at least in part by determining presence of rear-approaching traffic.

4. The multi-camera vision system of claim 3, wherein the rear-approaching traffic approaches the equipped vehicle in a side traffic lane adjacent to a traffic lane being traveled by the equipped vehicle.

5. The multi-camera vision system of claim 1, wherein, via processing at said central data processor of at least one selected from the group consisting of (i) provided captured image data and (ii) provided captured radar data, presence of an obstacle exterior the equipped vehicle is determined.

6. The multi-camera vision system of claim 1, wherein, via processing at said central data processor of at least one selected from the group consisting of (i) provided captured image data and (ii) provided captured radar data, presence of a person exterior the equipped vehicle is determined.

7. The multi-camera vision system of claim 1, wherein said central data processor processes fused image data and radar data.

8. The multi-camera vision system of claim 1, wherein the overlay that overlays displayed video images captured by said rear backup camera guides the driver when hitching a trailer to a trailer hitch of the equipped vehicle.

9. The multi-camera vision system of claim 8, wherein the overlay aids in guiding connection of the trailer hitch of the equipped vehicle to a trailer tongue of the trailer.

10. The multi-camera vision system of claim 1, wherein said video screen of said display device comprises a backlit thin film transistor LCD video screen backlit by at least one white light emitting light emitting diode.

11. The multi-camera vision system of claim 1, wherein said video screen comprises an organic light emitting diode (OLED) video screen.

12. The multi-camera vision system of claim 1, wherein said display device is operable to display video images with a display intensity, as viewed by the driver of the equipped vehicle, greater than 200 candelas/sq. meter.

13. The multi-camera vision system of claim 1, wherein said display device is operable to display video images with a display intensity, as viewed by the driver of the equipped vehicle, greater than 400 candelas/sq. meter.

14. The multi-camera vision system of claim 1, wherein said display device is disposed at an interior rearview mirror assembly of the equipped vehicle and wherein at least one of (a) said display device displays video images for viewing through a mirror reflective element of said interior rearview mirror assembly by the driver of the equipped vehicle and (b) said display device automatically extends from said interior rearview mirror assembly responsive to the vehicle transmission being shifted into reverse gear, and wherein said interior rearview mirror assembly comprises a molded plastic mounting arm having at least one ball member at at least one end thereof and having electrical conducting elements extending therethrough, and wherein a mirror mounting portion is molded over said ball member, and wherein said mounting arm comprises a mounting arm polymeric material having a first linear mold shrinkage factor and a first flexural modulus, and wherein said mirror mounting portion comprises a mirror mounting polymeric material having a second linear mold shrinkage factor and a second flexural modulus, and wherein said second linear mold shrinkage factor is greater than said first linear mold shrinkage factor, and wherein said second flexural modulus is lower than said first flexural modulus.

15. The multi-camera vision system of claim 1, comprising a driver-side camera at a driver side portion of the equipped vehicle and a passenger-side camera at a passenger side portion of the equipped vehicle, wherein said driver-side camera has at least a sideward field of view external of the equipped vehicle, and wherein said passenger-side camera has at least a sideward field of view external of the equipped vehicle, and wherein said driver-side camera captures video image data at least sideward of the equipped vehicle at least during the rearward maneuver of the equipped vehicle, and wherein said passenger-side camera captures video image data at least sideward of the equipped vehicle at least during the rearward maneuver of the equipped vehicle, and wherein image data captured by said driver-side camera is provided to said central data processor and image data captured by said passenger-side camera is provided to said central data processor.

16. The multi-camera vision system of claim 15, wherein image data captured by said driver-side camera and said passenger-side camera is processed at said central data processor in order to detect objects that are within at least one selected from the group consisting of (i) the field of view of said driver-side camera and (ii) the field of view of said passenger-side camera.

17. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

a rear backup camera at a rear portion of a vehicle equipped with said multi-camera vision system;

wherein said rear backup camera comprises a CMOS imaging array;

said rear backup camera having a field of view at least rearward of the equipped vehicle;

wherein the field of view of said rear backup camera encompasses at least a portion of a vehicle bumper of the equipped vehicle;

said rear backup camera captures video image data at least rearward of the equipped vehicle during a rearward maneuver of the equipped vehicle;

a forward viewing camera disposed at a windshield of the equipped vehicle and viewing through the windshield forward of the equipped vehicle;

said forward viewing camera captures image data at least forward of the equipped vehicle;

a central data processor disposed in the equipped vehicle;

wherein image data captured by said forward viewing camera is provided to said central data processor and image data captured by said rear backup camera is provided to said central data processor;

wherein captured image data provided to said central data processor is processed at said central data processor in order to detect objects that are within at least one selected from the group consisting of (i) the field of view of said forward viewing camera and (ii) the field of view of said rear backup camera;

wherein detection of objects via processing at said central data processor of provided captured image data comprises use of an edge detection algorithm;

wherein the equipped vehicle is further equipped with at least one radar sensor;

wherein said at least one radar sensor captures radar data in a region external of the equipped vehicle;

wherein captured radar data is provided to said central data processor;

wherein said central data processor processes fused image data and radar data;

wherein, via processing at said central data processor of at least one selected from the group consisting of (i) provided captured image data and (ii) provided captured radar data, at least one selected from the group consisting of (a) presence of an obstacle exterior the equipped vehicle is determined and (b) presence of a person exterior the equipped vehicle is determined;

wherein, via processing at said central data processor of at least one of provided captured image data and provided captured radar data, a potential hazard is determined to exist based at least in part by determining presence of rear-approaching traffic;

a display device comprising a video screen that displays video images for viewing by a driver of the equipped vehicle during the rearward maneuver of the equipped vehicle, wherein image data captured by said rear backup camera is processed at said central data processor to generate video images for display on said video screen; and

wherein an overlay overlays displayed video images captured by said rear backup camera during the rearward maneuver of the equipped vehicle.

18. The multi-camera vision system of claim 17, wherein the rear-approaching traffic approaches the equipped vehicle in a side traffic lane adjacent to a traffic lane being traveled by the equipped vehicle.

19. The multi-camera vision system of claim 17, wherein said video screen comprises an organic light emitting diode (OLED) video screen.

20. A multi-camera vision system for a vehicle, said multi-camera vision system comprising:

a rear backup camera at a rear portion of a vehicle equipped with said multi-camera vision system;

wherein said rear backup camera comprises a CMOS imaging array;

said rear backup camera having a field of view at least rearward of the equipped vehicle;

wherein the field of view of said rear backup camera encompasses at least a portion of a vehicle bumper of the equipped vehicle;

said rear backup camera captures video image data at least rearward of the equipped vehicle during a rearward maneuver of the equipped vehicle;

a forward viewing camera disposed at a windshield of the equipped vehicle and viewing through the windshield forward of the equipped vehicle;

said forward viewing camera captures image data at least forward of the equipped vehicle;

a driver-side camera at a driver side portion of the equipped vehicle;

wherein said driver-side camera has at least a sideward field of view external of the equipped vehicle;

wherein said driver-side camera captures video image data at least sideward of the equipped vehicle at least during the rearward maneuver of the equipped vehicle;

a passenger-side camera at a passenger side portion of the equipped vehicle;

wherein said passenger-side camera has at least a sideward field of view external of the equipped vehicle;

wherein said passenger-side camera captures video image data at least sideward of the equipped vehicle at least during the rearward maneuver of the equipped vehicle;

a central data processor disposed in the equipped vehicle;

wherein image data captured by said forward viewing camera is provided to said central data processor and image data captured by said rear backup camera is provided to said central data processor and image data captured by said driver-side camera is provided to said central data processor and image data captured by said passenger-side camera is provided to said central data processor;

wherein captured image data provided to said central data processor is processed at said central data processor in order to detect objects that are within at least one selected from the group consisting of (i) the field of view of said forward viewing camera, (ii) the field of view of said rear backup camera, (iii) the field of view of said driver-side camera and (iv) the field of view of said passenger-side camera;

wherein detection of objects via processing at said central data processor of provided captured image data comprises use of an edge detection algorithm;

wherein the equipped vehicle is further equipped with at least one radar sensor;

wherein said at least one radar sensor captures radar data in a region external of the equipped vehicle;

wherein captured radar data is provided to said central data processor;

wherein, via processing at said central data processor of at least one of provided captured image data and provided captured radar data, a potential hazard is determined to exist based at least in part by determining presence of rear-approaching traffic;

a display device comprising a video screen that displays video images for viewing by a driver of the equipped vehicle during the rearward maneuver of the equipped vehicle, wherein image data captured by said rear backup camera is processed at said central data processor to generate video images for display on said video screen; and

wherein an overlay overlays displayed video images captured by said rear backup camera during the rearward maneuver of the equipped vehicle.

21. The multi-camera vision system of claim 20, wherein the rear-approaching traffic approaches the equipped vehicle in a side traffic lane adjacent to a traffic lane being traveled by the equipped vehicle.