Vehicular vision system

• US 10,875,453 B2
• Filed: 05/13/2019
• Issued: 12/29/2020
• Est. Priority Date: 05/10/2013
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gieseke, Achim, Biemer, Michael
• Primary Examiner(s): Beaulieu, Yonel
• Application Number:
  US16/410,086
• Publication Number:
  US 20190263321A1
• Time in Patent Office: 596 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A vehicular vision system includes a camera disposed at the vehicle and a video display device disposed in the vehicle. Image data captured by the camera is provided to an ECU of the vehicle. A first portion of a scene that is within the field of view of the camera is not viewable by a driver of the vehicle and a second portion of the scene exterior of the vehicle that is within the field of view of the camera is viewable by the driver. Responsive at least in part to processing at the ECU of image data captured by the camera, the vehicular vision system determines an augmented reality overlay. Responsive to processing at the ECU of image data captured by the camera, the video display screen displays video images of the exterior scene. The video display device displays the augmented reality overlay at the displayed video images.

Claims

1. A vehicular vision system, said vehicular vision system comprising:

a camera disposed at a vehicle equipped with said vehicular vision system, said camera having a field of view exterior of the vehicle;

wherein said camera captures image data within the field of view of said camera;

wherein a first portion of a scene exterior of the vehicle that is within the field of view of said camera is not viewable by a driver of the vehicle seated within an interior cabin of the vehicle;

wherein the driver'"'"'s view from within an interior cabin of the vehicle toward the first portion of the scene exterior of the vehicle is obstructed by a non-transparent object;

wherein a second portion of the scene exterior of the vehicle that is within the field of view of said camera is viewable by the driver seated within the interior cabin of the vehicle;

a video display device disposed in the interior cabin of the vehicle and viewable by the driver of the vehicle seated within the interior cabin of the vehicle;

wherein said video display device comprises a video display screen that is operable to display video images derived from image data captured by said camera;

wherein image data captured by said camera is provided to an electronic control unit (ECU) of the vehicle;

wherein said ECU comprises an image processor, said image processor processing image data captured by said camera;

wherein, responsive at least in part to processing at said ECU of image data captured by said camera, said vehicular vision system determines an augmented reality overlay;

wherein, responsive to processing at said ECU of image data captured by said camera, said video display screen displays video images of the exterior scene;

wherein the displayed video images (i) are derived from image data captured by said camera of the first portion of the exterior scene that is within the field of view of said camera and (ii) are derived from image data captured by said camera of the second portion of the exterior scene that is within the field of view of said camera; and

wherein said video display device displays the augmented reality overlay at the displayed video images.

2. The vehicular vision system of claim 1, wherein the displayed video images represent the first portion of the scene exterior of the vehicle at the other side of the non-transparent object from the driver, such as would be seen by the driver if the driver viewed through the non-transparent object.

3. The vehicular vision system of claim 1, wherein the non-transparent object comprises a non-transparent portion of the vehicle, and wherein the displayed video images represent the first portion of the scene exterior of the vehicle at the other side of the non-transparent portion of the vehicle from the driver, such as would be seen by the driver if the driver viewed through the non-transparent portion of the vehicle.

4. The vehicular vision system of claim 1, comprising a monitoring system that determines a gaze direction of the driver of the vehicle, and wherein said video display device displays video images responsive at least in part to said monitoring system.

5. The vehicular vision system of claim 4, wherein said monitoring system comprises an interior viewing camera disposed in the interior cabin of the vehicle and having a field of view interior of the vehicle that encompasses the head of the driver of the vehicle, and wherein, responsive to processing of image data captured by said interior viewing camera, said monitoring system determines the gaze direction of the driver of the vehicle.

6. The vehicular vision system of claim 4, wherein the displayed video images are dependent on the determined gaze direction of the driver of the vehicle.

7. The vehicular vision system of claim 6, comprising a plurality of cameras disposed at the vehicle and having respective fields of view exterior of the vehicle, wherein each camera of said plurality of cameras captures image data, and wherein, as the vehicle is driven by the driver, said video display device, responsive to processing at said ECU of image data captured by said cameras, displays video images representative of a portion of the field of view of at least one of said cameras that is not viewable by the driver of the vehicle when the driver the vehicle seated within the interior cabin of the vehicle, and wherein the displayed video images are dependent on the determined gaze direction of the driver of the vehicle.

8. The vehicular vision system of claim 1, wherein the augmented reality overlay displayed pertains to at least one accessory of the vehicle.

9. The vehicular vision system of claim 8, wherein the displayed augmented reality overlay is responsive at least in part to a determined gaze direction of the driver of the vehicle.

10. The vehicular vision system of claim 1, wherein the displayed video images are responsive at least in part to a driving condition of the vehicle.

11. The vehicular vision system of claim 10, wherein the driving condition comprises a parking maneuver of the vehicle.

12. The vehicular vision system of claim 11, wherein the driving condition comprises a forward driving condition of the vehicle.

13. A vehicular vision system, said vehicular vision system comprising:

a camera disposed at a vehicle equipped with said vehicular vision system, said camera having a field of view exterior of the vehicle;

wherein said camera captures image data within the field of view of said camera;

wherein a first portion of a scene exterior of the vehicle that is within the field of view of said camera is not viewable by a driver of the vehicle seated within an interior cabin of the vehicle;

wherein the driver'"'"'s view from within an interior cabin of the vehicle toward the first portion of the scene exterior of the vehicle is obstructed by a non-transparent object;

wherein a second portion of the scene exterior of the vehicle that is within the field of view of said camera is viewable by the driver seated within the interior cabin of the vehicle;

a video display device disposed in the interior cabin of the vehicle and viewable by the driver of the vehicle seated within the interior cabin of the vehicle;

wherein said video display device comprises a video display screen that is operable to display video images derived from image data captured by said camera;

wherein image data captured by said camera is provided to an electronic control unit (ECU) of the vehicle;

wherein said ECU comprises an image processor, said image processor processing image data captured by said camera;

wherein, responsive at least in part to processing at said ECU of image data captured by said camera, said vehicular vision system determines an augmented reality overlay;

wherein, responsive to processing at said ECU of image data captured by said camera, said video display screen displays video images of the exterior scene;

wherein the displayed video images (i) are derived from image data captured by said camera of the first portion of the exterior scene that is within the field of view of said camera and (ii) are derived from image data captured by said camera of the second portion of the exterior scene that is within the field of view of said camera;

wherein said video display device displays the augmented reality overlay at the displayed video images;

wherein the displayed augmented reality overlay is displayed at least in part responsive to a determined gaze direction of the driver of the vehicle; and

wherein the displayed video images and the displayed augmented reality overlay are displayed responsive at least in part to a driving condition of the vehicle, and wherein the driving condition comprises a forward driving condition of the vehicle.

14. The vehicular vision system of claim 13, wherein the displayed video images represent the first portion of the scene exterior of the vehicle at the other side of the non-transparent object from the driver, such as would be seen by the driver if the driver viewed through the non-transparent object.

15. The vehicular vision system of claim 13, wherein the non-transparent object comprises a non-transparent portion of the vehicle, and wherein the displayed video images represent the first portion of the scene exterior of the vehicle at the other side of the non-transparent portion of the vehicle from the driver, such as would be seen by the driver if the driver viewed through the non-transparent portion of the vehicle.

16. The vehicular vision system of claim 13, wherein the displayed video images are dependent on the determined gaze direction of the driver of the vehicle.

17. The vehicular vision system of claim 13, wherein the augmented reality overlay displayed pertains to at least one accessory of the vehicle.

18. A vehicular vision system, said vehicular vision system comprising:

a plurality of cameras disposed at a vehicle equipped with said vehicular vision system, wherein each camera of said plurality of cameras has a respective field of view exterior of the vehicle;

wherein each camera of said plurality of cameras captures image data within the field of view of the respective camera;

wherein a first portion of a scene exterior of the vehicle that is within the field of view of at least one camera of said plurality of cameras is not viewable by a driver of the vehicle seated within an interior cabin of the vehicle;

wherein the driver'"'"'s view from within an interior cabin of the vehicle toward the first portion of the scene exterior of the vehicle is obstructed by a non-transparent object;

wherein a second portion of the scene exterior of the vehicle that is within the field of view of said at least one camera is viewable by the driver seated within the interior cabin of the vehicle;

a video display device disposed in the interior cabin of the vehicle and viewable by the driver of the vehicle seated within the interior cabin of the vehicle;

wherein said video display device comprises a video display screen that is operable to display video images derived from image data captured by said at least one camera;

wherein image data captured by said at least one camera is provided to an electronic control unit (ECU) of the vehicle;

wherein said ECU comprises an image processor, said image processor processing image data captured by said at least one camera;

wherein, responsive at least in part to processing at said ECU of image data captured by said at least one camera, said vehicular vision system determines an augmented reality overlay;

wherein, responsive to processing at said ECU of image data captured by said at least one camera, said video display screen displays video images of the exterior scene;

wherein the displayed video images (i) are derived from image data captured by said at least one camera of the first portion of the exterior scene that is within the field of view of said at least one camera and (ii) are derived from image data captured by said at least one camera of the second portion of the exterior scene that is within the field of view of said at least one camera;

wherein said video display device displays the augmented reality overlay at the displayed video images; and

wherein the displayed video images and the displayed augmented reality overlay are displayed dependent on a determined gaze direction of the driver of the vehicle.

19. The vehicular vision system of claim 18, wherein the augmented reality overlay displayed pertains to at least one accessory of the vehicle.

20. The vehicular vision system of claim 18, wherein the displayed video images are responsive at least in part to a driving condition of the vehicle.