SYSTEM AND METHOD FOR AUGMENTED REALITY HEAD UP DISPLAY FOR VEHICLES

• US 20180059416A1
• Filed: 08/23/2017
• Published: 03/01/2018
• Est. Priority Date: 08/23/2016
• Status: Active Grant

• Current Assignee: Vijyat Bhalla
• Original Assignee: 8696322 Canada, Inc.
• Inventors: Bhalla, Vijyat
• Granted Patent: US 10,281,721 B2
• Time in Patent Office: Days
• CPC Class Codes: B60K 2360/176 Camera images
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 2360/5899 Internet
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 2370/152 Displays
• CPC Class Codes: B60K 2370/1529 Head-up displays
• CPC Class Codes: B60K 2370/176 Camera images
• CPC Class Codes: B60K 2370/177 Augmented reality
• CPC Class Codes: B60K 2370/5899 Internet
• CPC Class Codes: B60K 2370/592 External database involved
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/08 : Interaction between the driver and the control system
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 : characterised by the informative content of the display
• CPC Class Codes: G02B 27/01 : Head-up displays
• CPC Class Codes: G02B 27/0101 : characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06F 16/24 : Querying
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/95 : structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/20 : in augmented reality scenes
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/62 : Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G06V 40/18 : Eye characteristics, e.g. of the iris
• CPC Class Codes: G08G 1/0137 : for specific applications
• CPC Class Codes: G08G 1/0175 : by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages

Abstract

What is disclosed is a method for augmented reality head up display in a user vehicle comprising: capturing, by input devices, input data; receiving, by a communications manager, said captured input data from said input devices; receiving, by said communications manager, data from a communications path from at least one of a data center, and one or more external parties; providing, by said communications manager, said received captured input data from said input devices, and said data received from said communications path to a central processing unit (CPU) coupled to said communications manager; generating, by said CPU, custom data based on at least one of said received captured input data, and said data received from said communications path; and displaying, by a windshield projector, said generated custom data on a windshield of said vehicle.

Claims

1. An augmented reality head up display (AR-HUD) system for a user vehicle, comprising:

input devices;

a windshield projector;

a communications path coupled to a data center and one or more external parties;

a central processing unit (CPU) communicatively coupled to

said input devices,

said communications path, and

said windshield projector

via a communications network manager;

further wherein

said input devices are used to capture input data;

said communications network manager is used to

receive said captured input data from said input devices,

receive data from said communications path from at least one of said data center and said one or more external parties, and

provide said received captured input data and said data received from said communications path to said CPU;

said CPU is used to generate custom data based on at least one of

said received captured input data, and

said data received from said communications path; and

said windshield projector is used to display said custom data generated by said CPU on a windshield of the vehicle.

2. The system of claim 1, wherein said custom data comprises one or more augmented reality overlay elements.

3. The system of claim 2, wherein

said input data comprises

a detected user eye position, and

one or more detected positions corresponding to one or more real world objects viewed through said windshield;

further wherein

at least some of said one or more augmented reality overlay elements are generated based on said detected user eye position and said one or more detected real world object positions; and

said at least some of said generated one or more augmented reality overlay elements are displayed on said windshield proximate to one or more views of one or more detected real world objects on said windshield.

4. The system of claim 3, further wherein:

said one or more real world objects comprise a person;

said input data comprises an image associated with said person;

said CPU transmits a search request comprising said captured image to said data center via said coupled communications network manager and said communications path;

said data received from said communications path by said communications network manager comprises one or more records retrieved based on said search request; and

further wherein said at least some of said one or more augmented reality overlay elements are

generated based on said retrieved one or more records, and

displayed on said windshield proximate to one or more views of said person on said windshield.

5. The system of claim 2, further wherein

said data received from said communications path comprises records corresponding to one or more vehicles of interest;

said input data comprises a vehicle license plate image; and

said one or more augmented reality overlay elements comprise information obtained based on said CPU cross referencing said vehicle license plate image and said records corresponding to one or more vehicles of interest.

6. The system of claim 2, further wherein

said data received from said communications path comprises records corresponding to one or more persons of interest;

said input data comprises an image associated with a person; and

said one or more augmented reality overlay elements comprise information obtained based on said CPU cross referencing said image associated with said person and said records corresponding to one or more persons of interest.

7. The system of claim 3, further wherein:

said one or more real world objects comprise one or more target vehicles;

said input data comprises one or more speeds corresponding to said one or more target vehicles; and

further wherein said at least some of said one or more augmented reality overlay elements comprise said one or more speeds, and said one or more speeds are displayed on said windshield proximate to one or more views of said one or more target vehicles on said windshield.

8. The system of claim 3, further wherein:

said one or more real world objects comprise a target vehicle;

said input data comprises an image of said target vehicle license plate;

said CPU transmits a search request comprising said captured image to said data center via said coupled communications network manager and said communications path;

said data received from said communications path by said communications network manager comprises one or more records retrieved based on said search request; and

further wherein said at least some of said one or more augmented reality overlay elements are

generated based on said retrieved one or more records, and

displayed on said windshield proximate to one or more views of said target vehicle on said windshield.

9. The system of claim 1, wherein

said communications path comprises a vehicle-to-vehicle (V2V) communication manager;

said one or more external parties comprise other vehicles;

said data received from said communications path comprises data received from said other vehicles via said V2V communications manager; and

said CPU generates said custom data based on said data received from said other vehicles.

10. The system of claim 1, wherein

said data received from said communications path comprises at least one of

one or more emails,

one or more instant messages,

one or more broadcast messages, and

one or more telephone calls;

and

said custom data comprises at least one of

one or more notifications,

one or more messenger icons,

one or more icons associated with the one or more emails,

one or more icons associated with the one or more broadcast messages,

a scrolling banner, and

an icon associated with telephone calls.

11. A method for augmented reality head up display in a user vehicle comprising:

capturing, by input devices, input data;

receiving, by a communications manager, said captured input data from said input devices;

receiving, by said communications manager, data from a communications path from at least one of

a data center, and

one or more external parties;

providing, by said communications manager,

said received captured input data from said input devices, and

said data received from said communications path

to a central processing unit (CPU) coupled to said communications manager;

generating, by said CPU, custom data based on at least one of

said received captured input data, and

said data received from said communications path; and

displaying, by a windshield projector, said generated custom data on a windshield of said vehicle.

12. The method of claim 11, wherein said custom data comprises one or more augmented reality overlay elements.

13. The method of claim 12, wherein

said input data comprises

a detected user eye position, and

one or more detected positions corresponding to one or more real world objects viewed through said windshield;

at least some of said one or more augmented reality overlay elements are generated based on said detected user eye position and said one or more detected real world object positions; and

said at least some of said generated one or more augmented reality overlay elements are displayed on said windshield proximate to one or more views of one or more detected real world objects on said windshield.

14. The method of claim 13, further wherein:

said one or more real world objects comprise a person;

said input data comprises an image associated with said person;

said method further comprising

transmitting, by said CPU, a search request comprising said image associated with said person to said data center via said communications network manager and said communications path;

said data received from said communications path by said communications network manager comprises one or more records retrieved based on said search request; and

further wherein said at least some of said one or more augmented reality overlay elements are

generated based on said retrieved one or more records, and

displayed on said windshield proximate to one or more views of said person on said windshield.

15. The method of claim 12, further wherein

said data received from said communications path comprises records corresponding to one or more vehicles of interest;

said input data comprises a vehicle license plate image; and

said one or more augmented reality overlay elements comprise information obtained based on said CPU cross-referencing said vehicle license plate image and said records corresponding to one or more vehicles of interest.

16. The method of claim 12, further wherein

said data received from said communications path comprises records corresponding to one or more persons of interest;

said input data comprises an image associated with a person; and

said one or more augmented reality overlay elements comprise information obtained based on said CPU cross-referencing said image associated with said person and said records corresponding to one or more persons of interest.

17. The method of claim 13, further wherein:

said one or more real world objects comprise one or more target vehicles;

said input data comprises one or more speeds corresponding to said one or more target vehicles; and

further wherein said at least some of said one or more augmented reality overlay elements comprise said one or more speeds, and said one or more speeds are displayed on said windshield proximate to one or more views of said one or more target vehicles on said windshield.

18. The method of claim 13, further wherein:

said one or more real world objects comprise a target vehicle;

said input data comprises an image of said target vehicle license plate;

said CPU transmits a search request comprising said captured image to said data center via said coupled communications network manager and said communications path;

said data received from said communications path by said communications network manager comprises one or more records retrieved based on said search request; and

further wherein said at least some of said one or more augmented reality overlay elements are

generated based on said retrieved one or more records, and

displayed on said windshield proximate to one or more views of said target vehicle on said windshield.

19. The method of claim 11, wherein

said communications path comprises a vehicle-to-vehicle (V2V) communication manager;

said one or more external parties comprise other vehicles;

said data received from said communications path comprises data received from said other vehicles via said V2V communications manager; and

said CPU generates said custom data based on said data received from said other vehicles.

20. The method of claim 11, wherein

said data received from said communications path comprises at least one of

one or more emails,

one or more instant messages,

one or more broadcast messages, and

one or more telephone calls;

and

said custom data comprises at least one of

a notification associated with one of the one or more emails,

a messenger icon associated with one of the one or more instant messages,

an email icon associated with the one or more emails,

a broadcast message icon associated with the one or more broadcast messages,

a scrolling banner, and

a telephone icon associated with telephone calls.