Enhanced vision system full-windshield HUD

• US 8,629,903 B2
• Filed: 04/02/2009
• Issued: 01/14/2014
• Est. Priority Date: 04/02/2009
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Seder, Thomas A., Szczerba, Joseph F., Cui, Dehua
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US12/417,077
• Publication Number:
  US 20100253593A1
• Time in Patent Office: 1,748 Days
• Field of Search: 353/31, 353/79, 340/932.2, 345/7, 348 25-172, 348699-702, 348818-843, 707/100, 701/36
• US Class Current: 348/115
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/9325 for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G01S 2013/93271 in the front of the vehicles
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/161 Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 2380/10 : Automotive applications
• CPC Class Codes: G09G 3/02 : by tracing or scanning a light beam on a screen
• CPC Class Codes: G09G 3/03 : specially adapted for displays having non-planar surfaces, e.g. curved displays
• CPC Class Codes: G09G 5/377 : for mixing or overlaying two or more graphic patterns G09G5/02, G09G5/397 take precedence

Abstract

A method to represent graphical images upon a transparent windscreen head up display of a vehicle includes monitoring informational inputs describing the operational environment for the vehicle, processing the informational inputs to synthesize an operational environment model, monitoring data related to an occupant eye location, converting the model into the graphical images registering the critical information according the model upon the transparent windscreen head up display based upon the monitored data related to the occupant eye location, and displaying the graphical images upon the transparent windscreen head up display.

Claims

1. Method to represent graphical images upon a transparent windscreen head up display of a vehicle describing an operational environment for the vehicle, the method comprising:

monitoring informational inputs describing the operational environment for the vehicle;

processing the informational inputs to synthesize an operational environment model, the processing comprising;

determining the informational inputs to indicate critical information;

determining spatial relationships of the critical information to the vehicle; and

synthesizing the critical information based upon the spatial relationships to form the operational environment model;

monitoring data related to an occupant eye location;

converting the environmental model into the graphical images registering the critical information according the environmental model upon the transparent windscreen head up display based upon the monitored data related to the occupant eye location;

displaying the graphical images upon the transparent windscreen head up display;

estimating an occupant gaze location based on the monitored data related to the occupant eye location, the occupant gaze location describing a point on the transparent windscreen where eyes of the occupant are focused;

comparing the occupant gaze location to the displayed graphical images registering the critical information;

when the occupant gaze location is not in view of the displayed graphical images, registering a prompting graphic at the point of the occupant gaze location to direct the attention of the operator toward the displayed graphical images; and

displaying the prompting graphic upon the transparent windscreen head up display.

2. The method of claim 1, wherein the transparent windscreen head up display comprises a substrate combined with light emitting particles, wherein the particles are sized between 50 nm and 500 nm.

3. The method of claim 1, wherein the transparent windscreen head up display comprises microstructures that direct a light emission into a confined and directed viewing cone.

4. The method of claim 1, wherein monitoring the informational inputs describing the operational environment for the vehicle comprises fusing radar information and camera information.

5. The method of claim 1, wherein monitoring the informational inputs describing the operational environment for the vehicle comprises fusing global positioning information, 3D map, and camera information.

6. The method of claim 1, wherein processing the informational inputs further comprises:

determining an importance for the critical information; and

wherein the converting the environmental model into the graphical images is selectively emphasized based upon the determined importance for the critical information.

7. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises identifying critical lane keeping information.

8. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises identifying critical lane marker visibility conditions.

9. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises identifying critical object tracking data.

10. The method of claim 9, wherein identifying critical object tracking data comprises determining a likelihood of collision.

11. The method of claim 9, wherein the identifying critical object tracking data comprises comparing a range to a target object to a recommended range.

12. The method of claim 9, wherein identifying critical object tracking data comprises evaluating pedestrian presence.

13. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises evaluating a potential lane change.

14. The method of claim 13, further comprising displaying a simulated overhead view based upon the environmental model upon the transparent windscreen head up display.

15. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises utilizing global positioning information to identify route critical information.

16. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises monitoring information from other vehicles to indicate critical information.

17. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises identifying semi-autonomous actions by the vehicle for communication to an operator of the vehicle, the semi-autonomous actions by the vehicle including at least one of adaptive cruise control and automatic vehicle lateral control.

18. The method of claim 1, wherein the vehicle is an emergency vehicle.

19. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises evaluating information from in-vehicle systems regarding the operation of the vehicle.

20. The method of claim 1, wherein converting the environmental model into the graphical images comprises:

evaluating the environmental model based upon the monitored data related to the occupant eye location to determine whether relevant features corresponding to the critical information are visible to the occupant through the transparent windscreen head up display; and

when evaluating the environmental model determines that the relevant features are not visible through the transparent windscreen head up display, registering graphical images to the transparent windscreen head up display indicating the critical information and a direction corresponding to the critical information.

21. The method of claim 1, wherein displaying the graphical images upon the transparent windscreen head up display comprises displaying a virtual rearview mirror.

22. The method of claim 21, wherein displaying the virtual rearview mirror includes a panoramic view of areas to sides and rear of the vehicle.

23. The method of claim 1, further comprising displaying upon the transparent windscreen head up display infotainment information.

24. The method of claim 1, wherein the converting the environmental model into the graphical images comprises converting infrared camera information into a graphical depiction.

25. The method of claim 1, wherein displaying the graphical images upon the transparent windscreen head up display comprises utilizing a display theme selectable by the occupant.

26. The method of claim 1, wherein determining the informational inputs to indicate critical information comprises evaluating parking maneuver requirements.

27. System to display graphical images upon a windscreen of a vehicle describing an operational environment for the vehicle, comprising:

the windscreen comprising a transparent windscreen head up display;

a vehicle sensor system monitoring conditions in the operational environment;

a vehicle operation sensor system monitoring operational characteristics of the vehicle;

an information system including communicative devices accessing remote systems;

an enhanced vision system (EVS) system manager

monitoring the informational inputs describing the operational environment for the vehicle from the vehicle sensor system, the vehicle operation sensor system, and the information system,

determining critical information from the monitored informational inputs warranting display upon the windscreen, and

generating display requirements based upon the critical information;

an occupant eye location sensing system;

a graphics system communicating with the EVS system manager and with the occupant eye location sensing system and generating the graphical images to be displayed based upon the display requirements, wherein the graphical images are registered to the windscreen based upon information from the occupant eye location sensing system;

a graphics projection system communicating with the graphics system and displaying the graphical images;

wherein the EVS system manager is further configured to;

estimate an occupant gaze location based on the monitored data related to the occupant eye location, the occupant gaze location describing a point on the transparent windscreen where eyes of the occupant are focused;

compare the occupant gaze location to the displayed graphical images registering the critical information;

wherein the graphics system is further configured to communicate with the EVS system manager and register a prompting graphic at the point of the occupant gaze location to direct the attention of the operator toward the displayed graphical images when the occupant gaze location is not in view of the displayed graphical images; and

wherein the display system is further configured to communicate with the graphics system and display the prompting graphic upon the transparent windscreen head up display.

28. Apparatus to display registered graphical images within a vehicle describing an operational environment for the vehicle, the apparatus comprising:

a surface within the vehicle;

a device monitoring the operational environment of the vehicle;

an enhanced vision system (EVS) system manager

monitoring information from the device configured to monitor the operational environment of the vehicle, and

generating display requirements based upon the monitored information;

a graphics system communicating with the EVS system manager and generating graphics commands based upon the display requirements, wherein the graphics commands include registration to the surface based upon an estimated operator head location;

a graphics projection system projecting the registered graphical images upon the surface based upon the graphics commands;

wherein the EVS system manager is further configured to;

estimate an occupant gaze location based on the estimated operator head location, the occupant gaze location describing a point on the transparent windscreen where eyes of the occupant are focused;

compare the occupant gaze location to the displayed graphical images registering the critical information;

wherein the graphics system is further configured to communicate with the EVS system manager and register a prompting graphic at the point of the occupant gaze location to direct the attention of the operator toward the displayed graphical images when the occupant gaze location is not in view of the displayed graphical images; and

wherein the display system is further configured to communicate with the graphics system and display the prompting graphic upon the transparent windscreen head up display.

29. The apparatus of claim 28, wherein the surface within the vehicle is a substantially transparent windscreen head up display.

30. The apparatus of claim 29, wherein the substantially transparent windscreen head up display comprises a substrate combined with light emitting particles, wherein the particles are sized between 50 nm and 500 nm.

31. The apparatus of claim 29, wherein the substantially transparent windscreen head up display comprises microstructures that direct a light emission into a confined and directed viewing cone.

32. The apparatus of claim 28, wherein the surface within the vehicle is a side window.

33. The apparatus of claim 28, wherein the surface within the vehicle is a rear window.

34. The apparatus of claim 28, wherein the surface within the vehicle is an A-pillar surface.

35. The apparatus of claim 28, wherein the device monitoring the operation environment of the vehicle is selected from the group consisting of a camera system, a radar system, and a GPS device.

36. The apparatus of claim 28, wherein the graphics projection system utilizes an ultraviolet excitation light.

37. The apparatus of claim 28, wherein estimated operator head location is based upon information from an occupant eye location sensing system.

38. The apparatus of claim 28, wherein estimated operator head location is based upon an occupant seat position and a rearview mirror adjustment angle.