VEHICLE OPERATION AND CONTROL SYSTEM FOR AUTONOMOUS VEHICLES ON FULL WINDSHIELD DISPLAY

• US 20120173069A1
• Filed: 12/29/2010
• Published: 07/05/2012
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Szczerba, Joseph F., Seder, Thomas A., Cui, Dehua, Tsimhoni, Omer
• Granted Patent: US 8,924,150 B2
• Time in Patent Office: Days
• US Class Current: 701/25
• CPC Class Codes: B60K 2360/141 Activation of instrument input devices by approaching fingers or pens
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60W 2540/215 Selection or confirmation of options
• CPC Class Codes: B60W 30/18163 Lane change; Overtaking manoeuvres
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays

Abstract

A method to navigate a vehicle utilizing a graphic projection display, includes monitoring a navigation status graphic representing a navigation intent displayed upon the graphic projection display, monitoring a user input indicated to a portion of the graphic projection display, initiating a user-defined navigation command based on the monitored navigation status graphic and the monitored user input, and operating the vehicle in accordance with the user-defined navigation command.

Claims

1. Method to navigate a vehicle utilizing a graphic projection display, comprising:

monitoring a navigation status graphic representing a navigation intent displayed upon the graphic projection display;

monitoring a user input indicated to a portion of the graphic projection display;

initiating a user-defined navigation command based on the monitored navigation status graphic and the monitored user input; and

operating the vehicle in accordance with the user-defined navigation command.

2. The method of claim 1 wherein the graphic projection display comprises a substantially transparent windscreen head-up display comprising one of light emitting particles or microstructures over a predefined region of the windscreen permitting luminescent display while permitting vision therethrough.

3. The method of claim 1 wherein the graphic projection display comprises one of a touch screen display, a human machine interface display, non-transparent components and a display monitor.

4. The method of claim 3 further comprising:

projecting an image of a view corresponding to a driving scene of the vehicle upon the graphic projection display.

5. Method of claim 1 wherein monitoring a navigation status graphic representing navigation intent comprises:

monitoring an autonomous driving system;

determining the navigation intent of the vehicle based on the monitored autonomous driving system;

determining the navigation status graphic representing the navigation intent;

dynamically registering a location of the navigation status graphic upon the graphic projection display corresponding to a driving scene of the vehicle; and

displaying the navigation status graphic upon the graphic projection display.

6. Method of claim 1 wherein initiating the user-defined navigation command based on the monitored navigation status graphic and the monitored user input comprises:

manipulating the navigation status graphic based upon the monitored user input;

initiating the user-defined navigation command corresponding to the manipulated navigation status graphic;

determining a navigation command graphic representing the user-defined navigation command;

dynamically registering a location of the navigation command graphic upon the graphic projection display corresponding to a driving scene of the autonomous driving vehicle; and

displaying the navigation command graphic upon the graphic projection display.

7. The method of claim 5 further comprising monitoring an occupant eye location;

and

wherein dynamically registering the location of the navigation status graphic upon the graphic projection display corresponding to the driving scene is based upon the occupant eye location.

8. The method of claim 6 further comprising monitoring an occupant eye location;

and

wherein dynamically registering the location of the navigation command graphic upon the graphic projection display corresponding to the driving scene is based upon the occupant eye location.

9. The method of claim 1 further comprising:

determining a feedback graphic illustrating movement of the monitored user input indicated to the portion of the graphic projection display;

dynamically registering a location of the feedback graphic upon the graphic projection display corresponding to a driving scene of the vehicle; and

displaying the feedback graphic upon the graphic projection display.

10. The method of claim 1 wherein said portion of the graphic projection display corresponds to a location at which the navigation status graphic is displayed.

11. The method of claim 10 wherein monitoring the user input indicated to the portion of the graphic projection display comprises:

monitoring a location and an orientation of a hand of the user;

processing an optical image of hand movement of the user based on the monitored location and orientation of the hand of the user; and

wherein the optical image of the hand movement provides a gesture-based user interface for manipulating the navigation status graphic representing the navigation intent displayed upon the graphic projection display.

12. The method of claim 10 wherein monitoring the user input indicated to the portion of the graphic projection display comprises:

receiving touch gestures of a hand of the user upon a touch screen, the touch gestures indicated to the portion of the graphic projection display; and

wherein the received touch gestures provides a touch-based user interface for manipulating the navigation status graphic representing the navigation intent displayed upon the graphic projection display.

13. The method of claim 10 wherein monitoring the user input indicated to the portion of the graphic projection display comprises:

receiving user manipulations at a controller device, the user manipulations indicated to the portion of the graphic projection display; and

wherein the received user manipulations provides a controller-based interface for manipulating the navigation status graphic representing the navigation intent displayed upon the graphic projection display.

14. Method to navigate a vehicle utilizing a substantially transparent windscreen head-up display, comprising:

monitoring a navigation status graphic representing navigation intent displayed upon the substantially transparent windscreen head-up display;

monitoring a user input indicated to a portion of the substantially transparent windscreen head-up display;

manipulating the navigation status graphic displayed upon the substantially transparent windscreen head-up display based upon the monitored user input;

initiating a user-defined navigation command corresponding to the manipulated navigation status graphic;

determining a navigation command graphic representing the user-defined navigation command;

dynamically registering a location of the navigation command graphic upon the substantially transparent windscreen head-up display corresponding to the driving scene of the vehicle;

displaying the navigation command graphic upon the substantially transparent windscreen head-up display; and

operating the vehicle in accordance with the user-defined navigation command;

wherein the substantially transparent windscreen head-up display comprises one of light emitting particles or microstructures over a predefined region of the windscreen permitting luminescent display while permitting vision therethrough.

15. The method of claim 14 further comprising:

determining a feedback graphic illustrating movement of the monitored user input indicated to the portion of the substantially transparent windscreen head-up display;

dynamically registering a location of the feedback graphic upon the substantially transparent windscreen head-up display corresponding to the driving scene of the vehicle; and

displaying the feedback graphic upon the substantially transparent windscreen head-up display.

16. The method of claim 15 wherein the feedback graphic comprises a cursor graphic.

17. The method of claim 14 wherein the navigation status graphic comprises an arrow representing future autonomous navigation maneuvers for the vehicle.

18. The method of claim 14 wherein the navigation status graphic comprises a speed gauge graphic representing a current vehicle speed.

19. The method of claim 14 wherein the user input comprises a gesture-based user interface, the method further comprising:

receiving a grab-and-drag gesture hand movement indicated to the portion of the substantially transparent windscreen head-up display corresponding to a location at which the navigation status graphic is displayed; and

wherein manipulating the navigation status graphic displayed upon the substantially transparent windscreen head-up display comprises manipulating the navigation status graphic based upon the grab-and-drag gesture hand movement.

20. System to navigate a vehicle utilizing a graphic projection display, comprising:

an enhanced vision system system manager;

monitoring a navigation status graphic representing a navigation intent displayed upon the graphic projection display;

monitoring a user input indicated to a portion of the graphic projection display;

initiating a user-defined navigation command based upon the monitored navigation status graphic and the monitored user input;

operating the vehicle in accordance with the user-defined navigation command;

a graphics system generating a navigation command graphic representing a user-defined navigation command and the navigation status graphic representing the navigation intent; and

a graphics projection system communicating with the graphics system and displaying the navigation command graphic and the navigation status graphic upon the substantially transparent windscreen head-up display.