DRIVER ASSIST SYSTEM FOR VEHICLE

• US 20160229342A1
• Filed: 04/18/2016
• Published: 08/11/2016
• Est. Priority Date: 03/02/2000
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Taylor, David W., McCarthy, Kevin C., Lynam, Niall R., Schofield, Kenneth
• Granted Patent: US 9,809,168 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60C 23/0408 transmitting the signals by non-mechanical means from the wheel or tyre to a vehicle body mounted receiver
• CPC Class Codes: B60N 2/976 massaging systems
• CPC Class Codes: B60Q 1/2665 on rear-view mirrors lighting devices for vehicle interior on or for rear-view mirrors B60Q3/258
• CPC Class Codes: B60Q 5/006 indicating risk of collision between vehicles or with pedestrians
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60R 1/088 using a cell of electrically changeable optical characteristic, e.g. liquid-crystal or electrochromic mirrors
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1215 with information displays
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/406 using wireless transmission
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/36 : Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/365 : Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01S 13/862 : Combination of radar systems with sonar systems
• CPC Class Codes: G01S 13/865 : Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 : Combination of radar systems with cameras
• CPC Class Codes: G06T 2207/30252 : Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/80 : Geometric correction
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

A driver assist system for a vehicle includes at least one non-visual sensor and a forward-viewing camera disposed that is part of a multi-camera vision system of the vehicle that includes at least two side cameras and a rear backup camera. A video display screen is viewable by a driver of the vehicle when the driver is normally operating the vehicle. During a driving or parking maneuver of the vehicle, images derived, at least in part, from image data captured by at least some of the cameras are displayed by the video display screen, and at least one indication is provided to the driver of the vehicle during the driving maneuver. A unitary image is synthesized from image data captured by the multi-camera vision system and approximates a view as would be seen by a virtual camera at a single location exterior of the vehicle.

Claims

1. A driver assist system for a vehicle, said driver assist system comprising:

a forward-viewing color video camera disposed at a vehicle equipped with said driver assist system;

said forward-viewing color video camera having a field of view exterior of the equipped vehicle;

wherein said forward-viewing color video camera is disposed at a front portion of the equipped vehicle and has a field of view at least forward of the equipped vehicle;

said forward-viewing color video camera comprising a lens and a two-dimensional array of photosensing elements arranged in rows of photosensing elements and columns of photosensing elements;

wherein said forward-viewing color video camera is part of a multi-camera vision system of the equipped vehicle, said multi-camera vision system comprising at least three other color video cameras;

wherein each of said three other color video cameras has a field of view exterior of the equipped vehicle;

wherein a first one of said three other color video cameras is mounted at a driver-side of the equipped vehicle and a second one of said three other color video cameras is mounted at a passenger-side of the equipped vehicle and a third one of said three other color video cameras comprises a rear backup camera mounted at a rear portion of the equipped vehicle;

wherein the first one of said three other color video cameras mounted at the driver-side has a field of view at the driver-side of the equipped vehicle and the second one of said three other color video cameras mounted at the passenger-side has a field of view at the passenger-side of the equipped vehicle;

wherein the field of view of the first one of said three other color video cameras that is mounted at the driver-side overlaps the field of view of said rear backup camera, and wherein the field of view of the second one of said three other color video cameras that is mounted at the passenger-side overlaps the field of view of said rear backup camera;

at least one non-visual sensor;

a video display screen viewable by a driver of the equipped vehicle when the driver is normally operating the equipped vehicle;

wherein said video display screen comprises a multi-pixel display screen;

wherein, during a parking maneuver of the equipped vehicle, images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle are displayed by said video display screen to assist the driver in operating the equipped vehicle;

wherein at least one indication is provided to the driver of the equipped vehicle during the parking maneuver;

wherein at least one of (a) said at least one indication comprises an indication provided, at least in part, responsive to detection by said non-visual sensor of at least one object external of the equipped vehicle and (b) said at least one indication comprises an indication provided, at least in part, responsive to image processing by an image processor of image data captured by at least one camera of said multi-camera vision system;

wherein a unitary image is synthesized from image data captured by said multi-camera vision system of the equipped vehicle;

wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by said video display screen that is viewable by the driver of the equipped vehicle when normally operating the equipped vehicle;

wherein distortion of images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle, and displayed by said video display screen, is electronically corrected;

wherein the field of view of said rear backup camera encompasses a ground area to the rear of the equipped vehicle having a width at least substantially an entire rear fender width of the equipped vehicle; and

wherein the field of view of said forward-viewing color video camera encompasses a ground area to the front of the equipped vehicle having a width at least substantially an entire front fender width of the equipped vehicle.

2. The driver assist system of claim 1, wherein said rear backup camera mounted at the rear portion of the equipped vehicle is positioned generally at the longitudinal centerline of the equipped vehicle and has a horizontal field of view that is generally symmetrical about the longitudinal axis of the equipped vehicle.

3. The driver assist system of claim 2, wherein said at least one indication comprises a visual indication displayed by said video display screen.

4. The driver assist system of claim 3, wherein said visual indication indicates distance to an object exterior of the equipped vehicle.

5. The driver assist system of claim 3, wherein said visual indication indicates proximity to an object exterior of the equipped vehicle.

6. The driver assist system of claim 3, wherein said visual indication highlights an object exterior of the equipped vehicle.

7. The driver assist system of claim 3, wherein said at least one indication comprises an indication provided, at least in part, responsive to image processing by an image processor of image data captured by at least one camera of said multi-camera vision system and wherein said at least one camera of said multi-camera vision system comprises said rear backup camera.

8. The driver assist system of claim 7, wherein said video display screen is operable to display images with a display intensity greater than 200 candelas/sq. meter for viewing by the driver of the equipped vehicle.

9. The driver assist system of claim 3, wherein said at least one indication comprises an indication provided, at least in part, responsive to detection by said non-visual sensor of at least one object external of the equipped vehicle, and wherein said non-visual sensor comprises an ultrasonic sensor.

10. The driver assist system of claim 3, wherein said at least one indication comprises an indication provided, at least in part, responsive to detection by said non-visual sensor of at least one object external of the equipped vehicle, and wherein said non-visual sensor comprises a radar sensor.

11. The driver assist system of claim 3, wherein said at least one indication comprises an indication provided, at least in part, responsive to detection by said non-visual sensor of at least one object external of the equipped vehicle, and wherein said non-visual sensor comprises an infrared sensor.

12. The driver assist system of claim 1, comprising at least one of (a) a control operable to control at least one accessory of the equipped vehicle in accordance with a characteristic of an occupant of the equipped vehicle, and wherein said at least one accessory comprises a seat adjustment system operable to adjust a seat of the equipped vehicle in accordance with a biometric characteristic of an occupant of the equipped vehicle, and wherein said seat adjustment system is operable to adjust a seat of the equipped vehicle in response to biometric data, said biometric data pertaining to the occupant of the seat of the equipped vehicle, and (b) a control operable to communicate with an external service provider via a wireless communication link between the equipped vehicle and the external service provider, wherein said control receives an input from the driver of the equipped vehicle and responsive thereto establishes said wireless communication link between the equipped vehicle and the external service provider, and wherein said control controls at least one accessory of the equipped vehicle responsive to a geographic location of the equipped vehicle as determined by a global positioning system of the equipped vehicle, and wherein data from the external service provider is downloaded to said control via said wireless communication link, and wherein said control comprises memory for storing downloaded data at least after said wireless communication link established between said control and the external service provider is disconnected, and wherein said downloaded data comprises downloaded driving instruction data useful for instructing the driver of the equipped vehicle how to drive from an initial location to a destination location, and wherein driving instructions derived, at least in part, from said downloaded driving instruction data are displayed, when the equipped vehicle is not executing a reversing maneuver, by said video display screen for viewing by the driver of the equipped vehicle, and wherein said driving instructions are displayed by said video display screen in a step-by-step manner, with at least some driving instruction steps being displayed by said video display screen after said wireless communication link between said control and the external service provider is disconnected.

13. A driver assist system for a vehicle, said driver assist system comprising:

a forward-viewing color video camera disposed at a vehicle equipped with said driver assist system;

said forward-viewing color video camera having a field of view exterior of the equipped vehicle;

wherein said forward-viewing color video camera is disposed at a front portion of the equipped vehicle and has a field of view at least forward of the equipped vehicle;

said forward-viewing color video camera comprising a lens and a two-dimensional array of photosensing elements arranged in rows of photosensing elements and columns of photosensing elements;

wherein said forward-viewing color video camera is part of a multi-camera vision system of the equipped vehicle, said multi-camera vision system comprising at least three other color video cameras;

wherein each of said three other color video cameras has a field of view exterior of the equipped vehicle;

wherein a first one of said three other color video cameras is mounted at a driver-side of the equipped vehicle and a second one of said three other color video cameras is mounted at a passenger-side of the equipped vehicle and a third one of said three other color video cameras comprises a rear backup camera mounted at a rear portion of the equipped vehicle;

wherein the first one of said three other color video cameras mounted at the driver-side has a field of view at the driver-side of the equipped vehicle and the second one of said three other color video cameras mounted at the passenger-side has a field of view at the passenger-side of the equipped vehicle;

wherein the field of view of the first one of said three other color video cameras that is mounted at the driver-side overlaps the field of view of said rear backup camera, and wherein the field of view of the second one of said three other color video cameras that is mounted at the passenger-side overlaps the field of view of said rear backup camera;

at least one non-visual sensor;

a video display screen viewable by a driver of the equipped vehicle when the driver is normally operating the equipped vehicle;

wherein said video display screen comprises a multi-pixel display screen;

wherein, during a driving maneuver of the equipped vehicle, images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle are displayed by said video display screen to assist the driver in operating the equipped vehicle;

wherein at least one indication is provided to the driver of the equipped vehicle during the driving maneuver;

wherein said at least one indication comprises an indication provided, at least in part, responsive to image processing by an image processor of image data captured by at least one camera of said multi-camera vision system;

wherein a unitary image is synthesized from image data captured by said multi-camera vision system of the equipped vehicle;

wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by said video display screen that is viewable by the driver of the equipped vehicle when normally operating the equipped vehicle;

wherein the field of view of said rear backup camera encompasses a ground area to the rear of the equipped vehicle having a width at least substantially an entire rear fender width of the equipped vehicle;

wherein the field of view of said forward-viewing color video camera encompasses a ground area to the front of the equipped vehicle having a width at least substantially an entire front fender width of the equipped vehicle; and

wherein said at least one indication comprises a visual indication displayed by said video display screen.

14. The driver assist system of claim 13, wherein distortion of images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle, and displayed by said video display screen, is electronically corrected.

15. The driver assist system of claim 14, wherein said driving maneuver comprises a parking maneuver.

16. The driver assist system of claim 15, wherein said visual indication indicates distance to an object exterior of the equipped vehicle.

17. The driver assist system of claim 15, wherein said visual indication indicates proximity to an object exterior of the equipped vehicle.

18. The driver assist system of claim 15, wherein said visual indication highlights an object exterior of the equipped vehicle.

19. The driver assist system of claim 13, comprising at least one non-visual sensor, and wherein said at least one non-visual sensor is selected from the group consisting of an ultrasonic sensor, a radar sensor and an infrared sensor.

20. The driver assist system of claim 13, wherein said at least one indication comprises an indication provided, at least in part, responsive to image processing by an image processor of image data captured by said rear backup camera of said multi-camera vision system.

21. The driver assist system of claim 20, wherein said video display screen is operable to display images with a display intensity greater than 200 candelas/sq. meter for viewing by the driver of the equipped vehicle.

22. The driver assist system of claim 20, wherein said rear backup camera mounted at the rear portion of the equipped vehicle is positioned generally at the longitudinal centerline of the equipped vehicle and has a horizontal field of view that is generally symmetrical about the longitudinal axis of the equipped vehicle.

23. A driver assist system for a vehicle, said driver assist system comprising:

a forward-viewing color video camera disposed at a vehicle equipped with said driver assist system;

said forward-viewing color video camera having a field of view exterior of the equipped vehicle;

wherein said forward-viewing color video camera is disposed at a front portion of the equipped vehicle and has a field of view at least forward of the equipped vehicle;

said forward-viewing color video camera comprising a lens and a two-dimensional array of photosensing elements arranged in rows of photosensing elements and columns of photosensing elements;

wherein said forward-viewing color video camera is part of a multi-camera vision system of the equipped vehicle, said multi-camera vision system comprising at least three other color video cameras;

wherein each of said three other color video cameras has a field of view exterior of the equipped vehicle;

wherein a first one of said three other color video cameras is mounted at a driver-side of the equipped vehicle and a second one of said three other color video cameras is mounted at a passenger-side of the equipped vehicle and a third one of said three other color video cameras comprises a rear backup camera mounted at a rear portion of the equipped vehicle;

wherein the first one of said three other color video cameras mounted at the driver-side has a field of view at the driver-side of the equipped vehicle and the second one of said three other color video cameras mounted at the passenger-side has a field of view at the passenger-side of the equipped vehicle;

wherein the field of view of the first one of said three other color video cameras that is mounted at the driver-side overlaps the field of view of said rear backup camera, and wherein the field of view of the second one of said three other color video cameras that is mounted at the passenger-side overlaps the field of view of said rear backup camera;

at least one non-visual sensor;

wherein said at least one non-visual sensor is selected from the group consisting of an ultrasonic sensor, a radar sensor and an infrared sensor;

a video display screen viewable by a driver of the equipped vehicle when the driver is normally operating the equipped vehicle;

wherein said video display screen comprises a multi-pixel display screen;

wherein, during a driving maneuver of the equipped vehicle, images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle are displayed by said video display screen to assist the driver in operating the equipped vehicle;

wherein at least one indication is provided to the driver of the equipped vehicle during the driving maneuver;

wherein said at least one indication comprises an indication provided, at least in part, responsive to detection by said non-visual sensor of at least one object external of the equipped vehicle;

wherein a unitary image is synthesized from image data captured by said multi-camera vision system of the equipped vehicle;

wherein said synthesized image approximates a view as would be seen by a virtual camera at a single location exterior of the equipped vehicle;

wherein said synthesized image is displayed by said video display screen that is viewable by the driver of the equipped vehicle when normally operating the equipped vehicle;

wherein distortion of images derived, at least in part, from image data captured by at least some cameras of said multi-camera vision system of the equipped vehicle, and displayed by said video display screen, is electronically corrected;

wherein the field of view of said rear backup camera encompasses a ground area to the rear of the equipped vehicle having a width at least substantially an entire rear fender width of the equipped vehicle;

wherein the field of view of said forward-viewing color video camera encompasses a ground area to the front of the equipped vehicle having a width at least substantially an entire front fender width of the equipped vehicle;

wherein said rear backup camera mounted at the rear portion of the equipped vehicle is positioned generally at the longitudinal centerline of the equipped vehicle and has a horizontal field of view that is generally symmetrical about the longitudinal axis of the equipped vehicle; and

wherein said at least one indication comprises a visual indication displayed by said video display screen.

24. The driver assist system of claim 23, wherein said visual indication indicates distance to an object exterior of the equipped vehicle.

25. The driver assist system of claim 23, wherein said visual indication indicates proximity to an object exterior of the equipped vehicle.

26. The driver assist system of claim 23, wherein said visual indication highlights an object exterior of the equipped vehicle.

27. The driver assist system of claim 23, wherein said non-visual sensor comprises an ultrasonic sensor.

28. The driver assist system of claim 23, wherein said non-visual sensor comprises a radar sensor.

29. The driver assist system of claim 23, wherein said non-visual sensor comprises an infrared sensor.

30. The driver assist system of claim 23, wherein said at least one indication comprises an indication provided, at least in part, responsive to image processing by an image processor of image data captured by at least one camera of said multi-camera vision system and wherein said at least one camera of said multi-camera vision system comprises said rear backup camera.

31. The driver assist system of claim 30, wherein said driving maneuver comprises a parking maneuver.

32. The driver assist system of claim 31, wherein said video display screen is operable to display images with a display intensity greater than 200 candelas/sq. meter for viewing by the driver of the equipped vehicle.