AUGMENTED REALITY SYSTEM FOR VEHICLE BLIND SPOT PREVENTION

• US 20160379411A1
• Filed: 06/26/2015
• Published: 12/29/2016
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: PACCAR Incorporated
• Original Assignee: PACCAR Incorporated
• Inventors: Lee, Bruce, Harbach, Andrew Paul
• Granted Patent: US 10,373,378 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/081 avoiding blind spots, e.g. by using a side-by-side association of mirrors
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 11/0235 of flat type, e.g. LCD
• CPC Class Codes: B60R 2001/1215 with information displays
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2011/0017 Head-rests
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/50 characterised by the display information being shared, e.g. external display, data transfer to other traffic participants or centralised traffic controller
• CPC Class Codes: B60R 2300/8026 in addition to a rear-view mirror system
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G09G 2380/10 Automotive applications
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The present disclosure relates to systems and methods for providing various types of information to a vehicle driver. Such information can be used by the vehicle driver singularly or in conjunction with other information available to the vehicle driver in order to allow the driver to operate the vehicle in an increasingly safe manner and/or to reduce the likelihood of property damage and/or possible bodily injuries to the driver, etc. In some instances, such information is presented to the driver as an augmented reality environment such that the driver can “see through” objects that may be occluding the driver'"'"'s vision.

Claims

1. In a vehicle having one or more displays, a method implemented in computer-executable instructions for displaying information about vehicle surroundings to the driver of the vehicle, the method comprising:

obtaining vehicle environment data from one or more information sources, the vehicle environment data indicative of at least a part of a scene occluded from view of a driver when operating the vehicle; and

presenting to the driver of the vehicle, with the aid of the one or more displays, an augmented reality environment based on the vehicle surroundings data and representative of an area surrounding the vehicle but obstructed from operator view.

2. The method of claim 1, wherein said presenting includes

causing the one or more displays to render one or more virtual design elements representative of one or more objects located in the scene occluded from view of the driver.

3. The method of claim 2, wherein said presenting includes

generating one or more virtual design elements representative of one or more objects located in the scene occluded from view of the driver.

4. The method of claim 1, wherein said presenting includes

generating one or more virtual design elements representative of one or more objects located in the scene occluded from view of the driver;

obtaining a real image of the scene, the real image based on the point of view of the driver; and

causing the virtual design elements and the real image to be displayed together by the one or more displays.

5. The method of claim 4, wherein said causing the virtual design elements and the real images to be displayed together by the one or more displays includes

arranging the generated one or more virtual design elements over a preselected area of the real image.

6. The method of claim 1, wherein said obtaining vehicle surroundings data includes

obtaining data from a source external the vehicle, the external source including one of a local vehicle, a beacon, and a traffic camera.

7. The method of claim 1, wherein the vehicle surroundings data is associated with one or more sensing zones that correspond to one or more external areas of the vehicle, wherein the one or more external areas of the vehicle represent at least in part areas occluded from view of the driver when operating the vehicle.

8. A computer-readable medium having modules for conveying information to a vehicle driver regarding vehicle surroundings, comprising:

an information gathering module configured to collect data from one or more information sources associated with one or more sensing zones;

an augmented reality image rendering module configured to generate from the collected data one or more virtual design elements representative of objects occluded from view of the vehicle driver; and

a display module configured to cause the virtual design elements to be presented to a display.

9. The computer-readable medium of claim 8, wherein the display module is further configured to cause a real image to be presented with the virtual design elements, wherein the real images and the virtual design elements together form an augmented reality environment.

10. The computer-readable medium of claim 8, wherein the augmented reality image rendering module is further configured to arrange and align the generated one or more virtual design elements over a preselected area of the real image.

11. The computer-readable medium of claim 8, wherein the one or more information sources includes information sources selected from the group consisting of a digital cameras, a thermal imaging device, radar, and lidar.

12. The computer-readable medium of claim 8, wherein the one or more sensing zones correspond to one or more external areas of the vehicle.

13. The computer-readable medium of claim 12, wherein the one or more external areas of the vehicle represent at least in part areas occluded from view of the driver when operating the vehicle.

14. A system for providing information to a vehicle driver, comprising:

one or more displays;

one or more information sources configured to generate data associated with one or more sensing zones located at one or more areas surrounding the vehicle; and

a display generator coupled to the one or more information sources, the display generator configured to;

collect data from one or more information sources associated with one or more sensing zones;

generate from the collected data one or more virtual design elements representative of objects occluded from view of the vehicle driver; and

cause the virtual design elements to be presented to a display.

15. The system of claim 14, wherein the one or more displays includes a transparent display configured to overlay one of a transparent layer and a reflective layer.

16. The system of claim 15, wherein the transparent layer includes a window and the reflective layer includes a mirror.

17. The system of claim 16, wherein the mirror includes one of a side mirror and a rear view mirror.

18. The system of claim 14, wherein the display generator includes

a memory storing one or more modules having program instructions for conveying vehicle information on the graphical display;

and

a processor configured to execute the program instructions of the one or more modules.

19. The system of claim 14, wherein the one or more information sources includes information sources selected from the group consisting of a digital cameras, a thermal imaging device, radar, and lidar.

20. The system of claim 14, wherein one of the one or more information sources includes a receiver configured to receive data associated with one or more of the sensing zones from a source external the vehicle, wherein the source external the vehicle is one of an a local vehicle, a beacon, and a traffic camera.