Virtual cursor for road scene object selection on full windshield head-up display

• US 9,057,874 B2
• Filed: 12/30/2010
• Issued: 06/16/2015
• Est. Priority Date: 12/30/2010
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Seder, Thomas A., Szczerba, Joseph F., Cui, Dehua
• Primary Examiner(s): Tran, Tuyetlien
• Assistant Examiner(s): RAMSEY, PATRICK RANDAL
• Application Number:
  US12/982,478
• Publication Number:
  US 20120174004A1
• Time in Patent Office: 1,629 Days
• Field of Search: 715/764, 345/7, 340/435, 340/903, 340/905
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays

Abstract

A method to select a feature in a view in front of a vehicle includes monitoring the view in front of the vehicle visible through a graphic projection display, monitoring a user input to the graphic projection display, determining the selected feature based upon the monitored view and the monitored user input, determining a registered graphic representing the selected feature for display upon the graphic projection display, and displaying the registered graphic upon the graphic projection display.

Claims

1. Method to select a feature in a view in front of a vehicle, the method comprising:

monitoring the view in front of the vehicle visible through a graphic projection display, the graphic projection display always remaining static with respect to the view in front of the vehicle;

monitoring a user input to the graphic projection display, the user input corresponding to a first viewer and indicated to a portion of the graphic projection display corresponding to a portion of the view in front of the vehicle that includes the feature;

monitoring an eye location of the user;

estimating a gaze location of the user based on the monitored eye location, the gaze location of the user describing a point on a transparent windscreen where eyes of the user are focused;

selecting the feature based upon the monitored view, the gaze location of the user, and the monitored user input, wherein the feature is represented as viewed in the real environment in front of the vehicle;

only after the feature is selected, ascertaining information related to the selected feature;

determining a first registered graphic representing the ascertained information related to the selected feature for display upon the graphic projection display; and

displaying the first registered graphic upon the graphic projection display, wherein displaying the first registered graphic upon the graphic projection display comprises projecting the first registered graphic and a second registered graphic upon a pixelated field of view limited architecture, comprising;

monitoring a first location corresponding to the first viewer;

monitoring a second location corresponding to a second viewer, wherein the first and second locations are configured on the interior of the vehicle;

determining the second graphic representing the ascertained information related to the selected feature, wherein the first graphic is respective to the first location corresponding to the first viewer and the second graphic is respective to the second location corresponding to the second viewer;

dynamically registering first and second viewer-defined locations of respective ones of the first and second graphics upon an interior side of the graphic projection display corresponding to the driving scene of the vehicle, comprising

the first graphic dynamically registered at the first viewer-defined location upon the interior side of the graphic projection display to enable view of the first graphic only from a first limited direction oriented toward the first viewer location corresponding to the first viewer, and

the second graphic dynamically registered at the second viewer-defined location upon the interior side of the graphic projection display to enable view of the second graphic only from a second limited direction oriented toward the second viewer location corresponding to the second viewer; and

projecting the first and second graphics upon the interior side of the graphic projection display.

2. The method of claim 1, further comprising identifying the selected feature.

3. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as a road feature.

4. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as another vehicle.

5. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as a building.

6. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as a parking lot.

7. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as an advertising sign.

8. The method of claim 2, wherein identifying the selected feature comprises identifying the selected feature as a sign indicating a potential navigation destination.

9. The method of claim 1, wherein the graphic projection display is a transparent windscreen head-up display.

10. The method of claim 9, wherein monitoring the user input to the graphic projection display comprises monitoring a hand gesture indicated to the graphic projection display;

wherein determining the registered graphic representing the selected feature is further based upon the monitored hand gesture.

11. The method of claim 10, wherein monitoring the hand gesture comprises utilizing an imaging device to generate a three-dimensional mapping of the hand gesture.

12. The method of claim 11, wherein utilizing the imaging device to generate the three-dimensional mapping of the hand gesture comprises capturing images with a device utilizing one of infrared light, ultraviolet light, and a combination of infrared and ultraviolet light.

13. The method of claim 11, wherein utilizing the imaging device to generate the three-dimensional mapping of the hand gesture comprises utilizing a laser speckle pattern.

14. The method of claim 11, wherein utilizing the imaging device to generate the three-dimensional mapping of the hand gesture comprises utilizing pulsed modulation.

15. The method of claim 11, wherein utilizing the imaging device to generate the three-dimensional mapping of the hand gesture comprises illuminating the hand gesture with beams of radiation and generating the three-dimensional mapping based upon intensity-based mapping.

16. The method of claim 1, wherein the graphic projection display comprises a touch screen device displaying the view in front of the vehicle.

17. The method of claim 1, wherein the graphic projection display comprises a non-transparent surface comprising luminescent materials.