Vehicle vision system

• US 9,738,224 B2
• Filed: 03/07/2016
• Issued: 08/22/2017
• Est. Priority Date: 05/10/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gieseke, Achim, Biemer, Michael
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US15/062,527
• Publication Number:
  US 20160185293A1
• Time in Patent Office: 533 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A vision system suitable for use for a vehicle includes a plurality of exterior viewing cameras disposed at the vehicle and having respective fields of view exterior of the vehicle. A display system is operable to display images for viewing by the driver of the vehicle. Responsive at least in part to determination that the gaze direction of the driver of the vehicle is towards a non-transparent portion of the vehicle, the display system displays an image derived from image data captured by at least one of the plurality of exterior viewing cameras to display a representation of the driver'"'"'s view as it would be through the non-transparent portion of the vehicle. The gaze direction of the driver may be determined by a monitoring system that processes image data captured by an interior viewing camera having a field of view that encompasses the head of the driver.

Claims

1. A vision system suitable for use for a vehicle, said vision system comprising:

a monitoring system operable to determine a gaze direction of a driver of a vehicle equipped with said vision system, said monitoring system comprising an interior viewing camera having a field of view interior of the equipped vehicle and directed generally towards the head of the driver of the equipped vehicle, wherein, responsive to processing of image data captured by said interior viewing camera, said monitoring system determines the gaze direction of the driver of the equipped vehicle;

a plurality of exterior viewing cameras disposed at the equipped vehicle and having respective fields of view exterior of the equipped vehicle;

a display system operable to display images for viewing by the driver of the equipped vehicle; and

wherein, responsive at least in part to determination by said monitoring system that the gaze direction of the driver is towards a non-transparent portion of the equipped vehicle, said display system displays an image derived from image data captured by at least one of said plurality of exterior viewing cameras to display a representation of the driver'"'"'s view as it would be through the non-transparent portion of the equipped vehicle.

2. The vision system of claim 1, further comprising a gesture control system operable to determine a gesture of the driver and wherein at least one accessory of the equipped vehicle is controlled responsive to the determined gesture.

3. The vision system of claim 2, wherein said gesture control system is operable to determine movement of the driver'"'"'s hand.

4. The vision system of claim 1, wherein said display system is operable to display information pertaining to at least one vehicle accessory for viewing by the driver of the equipped vehicle.

5. The vision system of claim 4, wherein said display system comprises a heads up display system and is operable to display accessory information responsive at least in part to the determined gaze direction of the driver.

6. The vision system of claim 1, wherein said display system is operable to display information responsive at least in part to a driving condition.

7. The vision system of claim 6, wherein said driving condition comprises a parking maneuver of the equipped vehicle.

8. The vision system of claim 6, wherein said driving condition comprises a forward driving condition.

9. The vision system of claim 8, wherein, during the forward driving condition, said monitoring system determines that the gaze direction of the driver is towards the engine compartment of the equipped vehicle.

10. The vision system of claim 1, wherein the non-transparent portion of the equipped vehicle comprises a door of the equipped vehicle.

11. The vision system of claim 10, wherein said plurality of exterior viewing cameras comprises a sideward viewing camera of the equipped vehicle.

12. The vision system of claim 1, wherein the non-transparent portion of the equipped vehicle comprises a rear door or decklid or trunk of the equipped vehicle.

13. The vision system of claim 12, wherein said plurality of exterior viewing cameras comprises a rearward viewing camera of the equipped vehicle.

14. The vision system of claim 1, wherein the non-transparent portion of the equipped vehicle comprises a vehicle hood of the equipped vehicle.

15. The vision system of claim 14, wherein said plurality of exterior viewing cameras comprises a forward viewing camera of the equipped vehicle.

16. The vision system of claim 1, wherein the non-transparent portion comprises a vehicle portion that occludes or prevents the driver from seeing an exterior region exterior of that portion of the equipped vehicle and at a blind spot area that exists due to that non-transparent portion blocking the driver'"'"'s view.

17. The vision system of claim 1, wherein display of a representation of the driver'"'"'s view as it would be through the non-transparent portion of the equipped vehicle is responsive to a driving condition, and wherein said driving condition comprises at least one driving condition selected from the group consisting of (i) a parking maneuver, (ii) the equipped vehicle being in reverse and driving at a reverse speed below a threshold level and (iii) the equipped vehicle driving forward at a forward speed below a threshold level.

18. A vision system suitable for use for a vehicle, said vision system comprising:

a plurality of exterior viewing cameras disposed at a vehicle equipped with said vision system and having respective fields of view exterior of the equipped vehicle;

a display system operable to display images for viewing by a driver of the equipped vehicle;

wherein said display system displays an image derived from image data captured by at least one of said plurality of exterior viewing cameras to display a representative of the driver'"'"'s view as it would be through a non-transparent portion of the equipped vehicle;

wherein the non-transparent portion comprises a vehicle portion that occludes or prevents the driver from seeing an exterior region exterior of that portion of the equipped vehicle and at a blind spot area that exists due to that non-transparent portion blocking the driver'"'"'s view; and

wherein display of a representation of the driver'"'"'s view as it would be through the non-transparent portion of the equipped vehicle is responsive to a driving condition, and wherein said driving condition comprises at least one driving condition selected from the group consisting of (i) a parking maneuver, (ii) the equipped vehicle being in reverse and driving at a reverse speed below a threshold level and (iii) the equipped vehicle driving forward at a forward speed below a threshold level.

19. The vision system of claim 18, wherein display of a representation of the driver'"'"'s view as it would be through the non-transparent portion of the equipped vehicle is responsive to the equipped vehicle being in reverse and driving at a reverse speed below a threshold level.

20. The vision system of claim 19, wherein said plurality of exterior viewing cameras comprises a rearward viewing camera of the equipped vehicle and wherein said threshold level is five miles per hour.

21. The vision system of claim 19, comprising a monitoring system operable to determine a gaze direction of the driver of the equipped vehicle, said monitoring system comprising an interior viewing camera having a field of view interior of the equipped vehicle and directed generally towards the head of the driver of the equipped vehicle, wherein, responsive to processing of image data captured by said interior viewing camera, said monitoring system determines the gaze direction of the driver of the equipped vehicle.

22. A vision system suitable for use for a vehicle, said vision system comprising:

a plurality of exterior viewing cameras disposed at a vehicle equipped with said vision system and having respective fields of view exterior of the equipped vehicle;

a display system operable to display images for viewing by a driver of the equipped vehicle;

wherein said display system displays an image derived from image data captured by at least one of said plurality of exterior viewing cameras to display a representative of the driver'"'"'s view as it would be through a non-transparent portion of the equipped vehicle;

wherein the non-transparent portion comprises a vehicle portion that occludes or prevents the driver from seeing an exterior region exterior of that portion of the equipped vehicle and at a blind spot area that exists due to that non-transparent portion blocking the driver'"'"'s view;

wherein the non-transparent portion of the equipped vehicle comprises one of a driver side portion of the equipped vehicle and a passenger side portion of the equipped vehicle; and

wherein said plurality of exterior viewing cameras comprises a sideward viewing camera of the equipped vehicle.

23. The vision system of claim 22, wherein display of a representation of the driver'"'"'s view as it would be through the non-transparent portion of the equipped vehicle is responsive to a driving condition.

24. The vision system of claim 23, wherein said driving condition comprises at least one driving condition selected from the group consisting of (i) a parking maneuver, (ii) the equipped vehicle being in reverse and driving at a reverse speed below a threshold level and (iii) the equipped vehicle driving forward at a forward speed below a threshold level.

25. The vision system of claim 24, comprising a monitoring system operable to determine a gaze direction of the driver of the equipped vehicle with said vision system, said monitoring system comprising an interior viewing camera having a field of view interior of the equipped vehicle and directed generally towards the head of the driver of the equipped vehicle, wherein, responsive to processing of image data captured by said interior viewing camera, said monitoring system determines the gaze direction of the driver of the equipped vehicle.