Enhanced road vision on full windshield head-up display

• US 8,704,653 B2
• Filed: 06/19/2009
• Issued: 04/22/2014
• Est. Priority Date: 04/02/2009
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Seder, Thomas A., Szczerba, Joseph F., Cui, Dehua
• Primary Examiner(s): TRIEU, VAN THANH
• Application Number:
  US12/487,926
• Publication Number:
  US 20100253540A1
• Time in Patent Office: 1,768 Days
• Field of Search: 340/435, 340/436, 340/438, 340/903, 340/932.2, 340/995.19, 340/461, 340/905, 701/431, 701/409, 353/31, 353/79, 345/7, 180/167, 180/169
• US Class Current: 340/461
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/86 Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/87 Combinations of radar systems, e.g. primary radar and secondary radar
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 2013/932 using own vehicle data, e.g. ground speed, steering wheel direction
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/9325 for inter-vehicle distance regulation, e.g. navigating in platoons
• CPC Class Codes: G01S 2013/93271 : in the front of the vehicles
• CPC Class Codes: G01S 7/22 : Producing cursor lines and indicia by electronic means
• CPC Class Codes: G02B 2027/0118 : comprising devices for improving the contrast of the display / brillance control visibility
• CPC Class Codes: G02B 2027/0138 : comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 : comprising information/image processing systems
• CPC Class Codes: G02B 27/01 : Head-up displays
• CPC Class Codes: G06F 18/253 : of extracted features
• CPC Class Codes: G06V 10/806 : of extracted features
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/165 : for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: G08G 1/168 : Driving aids for parking, e.g. acoustic or visual feedback on parking space

Abstract

A method to selectively project graphical images upon a transparent windscreen head up display of a vehicle based upon a need for enhanced vision includes monitoring informational inputs describing an operational environment for the vehicle, processing the informational inputs to generate display requirements, generating the graphical images to be projected based upon the display requirements, and projecting the graphical images upon the transparent windscreen head up display. Processing of the informational inputs can include determining the need for enhanced vision based upon the informational inputs and generating the display requirements based upon the informational inputs and the need for enhanced vision.

Claims

1. Method to selectively project graphical images upon a transparent windscreen head up display of a vehicle based upon a need for enhanced vision, the method comprising:

monitoring informational inputs corresponding to features of a roadway needed for travel, critical information perceived as a threat to the vehicle and an operational environment for the vehicle;

determining a degree of importance of the critical information;

determining a measure of visual conditions based on the operational environment for the vehicle;

comparing the measure of visual conditions to a threshold visual condition, said threshold visual condition adjustable based on the degree of importance of the critical information;

determining the need for enhanced vision based on the measure of visual conditions violating the threshold visual condition;

generating display requirements for the features of the roadway based on the need for enhanced vision;

generating the graphical images corresponding to the features of the roadway to be projected based upon the display requirements; and

projecting the graphical images upon the transparent windscreen head up display.

2. The method of claim 1, wherein monitoring informational inputs corresponding to the features of the roadway comprises monitoring sensor inputs corresponding to the features of the roadway;

and

wherein the generating the graphical images comprises generating lines indicating lane boundaries based upon the monitored sensor inputs corresponding to the features of the roadway.

3. The method of claim 2, wherein monitoring sensor inputs corresponding to the features of the roadway comprises:

monitoring a vehicle coordinate position through a GPS device;

utilizing the monitored vehicle coordinate position to locate the vehicle with a map database; and

determining the features of the roadway based upon the location of the vehicle.

4. The method of claim 2, wherein monitoring sensor inputs corresponding to the features of the roadway comprises:

monitoring sensor inputs corresponding to a view in front of the vehicle; and

determining the features of the roadway based upon the monitored sensor inputs.

5. The method of claim 4, wherein monitoring sensor inputs comprises

monitoring visual images from a camera device.

6. The method of claim 4, wherein monitoring sensor inputs comprises

monitoring infrared images from an infrared sensor device.

7. The method of claim 1, further comprising:

monitoring data related to an occupant eye location;

analyzing the monitored data to identify a pattern consistent with trying to find obscured lane markers; and

determining the need for enhanced vision based upon identifying the pattern consistent with trying to find obscured lane markers; and

wherein generating the graphical images corresponding to the features of the roadway comprises generating lines indicating lane boundaries.

8. The method of claim 1, further comprising:

analyzing the monitored informational inputs to identify a situation likely to surprise; and

indicating the need for enhanced vision based upon identifying the situation likely to surprise.

9. The method of claim 1, further comprising

monitoring data related to an occupant eye location;

and

wherein generating the graphical images corresponding to the features of the roadway to be projected based upon the display requirements includes registering the graphical images based upon the monitored data related to the occupant eye location.

10. System to display graphical images upon a windscreen of a vehicle, selectively enhancing a view through the windscreen, the system comprising:

the windscreen comprising a transparent windscreen head up display;

a device monitoring a measure of visual conditions;

a device monitoring informational inputs corresponding to features of a roadway needed for travel, critical information perceived as a threat to the vehicle and an operational environment for the vehicle;

an enhanced vision system system manager configured to;

determine a degree of importance of the critical information;

compare the measure of visual conditions to a threshold visual condition, wherein the threshold visual condition is adjustable based on the degree of importance of the critical information; and

when the comparison of the measure of visual conditions to the threshold visual condition indicates impaired visual conditions, generate display requirements for the features of the roadway;

a graphics system generating the graphical images corresponding to the features of the roadway to be displayed based upon the display requirements; and

a graphics projection system communicating with the graphics system and displaying the graphical images.

11. The system of claim 10, wherein the graphical images corresponding to the features of the roadway comprise a lane boundary.

12. The system of claim 10, wherein the graphical images corresponding to the features of the roadway comprise a road edge.

13. The system of claim 10, wherein the graphical images corresponding to the features of the roadway comprise an indication of a target vehicle position.

14. The system of claim 10, wherein the graphical images corresponding to the features of the roadway comprise an indication of a pedestrian position.

15. The system of claim 10, wherein the enhanced vision system system manager is further configured to determine an importance of the generated display requirements;

and

wherein the graphical images include a description of determined importance.

16. The system of claim 10, further comprising an occupant eye location sensing system;

and

wherein the graphics system generating the graphical images corresponding to the features of the roadway to be displayed based upon the display requirements includes registration of the graphical images to the windscreen based upon data from the occupant eye location sensing system.

17. An apparatus to selectively display registered graphics upon a substantially transparent head up display of a vehicle enhancing a view through the substantially transparent head up display, the apparatus comprising:

a system manager configured to;

monitor sensor inputs corresponding to a road surface in front of the vehicle and critical information perceived as a threat to the vehicle;

monitor a measure of visual conditions;

determine a degree of importance of the critical information;

compare the measure of visual conditions to a threshold visual condition, the threshold visual condition adjustable based on the degree of importance of the critical information;

determine a need for enhancing the view based upon the measure of visual conditions violating the threshold visual condition; and

when the need for enhancing the view is determined, generate display requirements for the road surface based upon the monitored sensor inputs; and

a graphics system generating the registered graphical images corresponding to the road surface based upon the display requirements.