User Interface for a Head Mounted Display

• US 20140267400A1
• Filed: 03/14/2013
• Published: 09/18/2014
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: MABBUTT, Paul, MACIOCCI, Giuliano, AUGER, Louise M.
• Granted Patent: US 9,041,741 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3664 Details of the user input interface, e.g. buttons, knobs or sliders, including those provided on a touch screen; remote controllers; input using gestures
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A user interface (UI) of a head mounted display (HMD) is provided that allows a user to access one or more persistent data elements that are otherwise outside the user'"'"'s initial field of view by using a head movement, such as a head tilt (i.e., movement about a horizontal axis) and/or rotation (i.e., movement about a vertical axis). Embodiments also can provide for further movement and/or other manipulation of data of persistent data elements with further detected movement of the user'"'"'s head.

Claims

1. A method for displaying data by a see-through head mounted display (HMD):

determining a first field of view of a user of the HMD;

receiving sensor input from at least one sensor;

detecting, using the sensor input, a first movement of the user'"'"'s head about an axis, causing the user to have a second field of view; and

causing the HMD to display one or more persistent data elements in the second field of view, based on the detected first movement.

2. The method of claim 1, further comprising determining the first field of view is an expected field of view.

3. The method of claim 1, wherein the axis is a horizontal axis or a vertical axis.

4. The method of claim 1, wherein:

detecting the first movement comprises detecting a downward tilt of the user'"'"'s head; and

the one or more persistent data elements comprises a map.

5. The method of claim 4, wherein the map includes at least one of:

a destination,

a location of the user, or

a point of interest.

6. The method of claim 5, wherein the map includes the point of interest, the method further comprising displaying, within the first field of view of the user, a data element indicative that the map includes the point of interest.

7. The method of claim 4, wherein the map appears, from a perspective of the user, in a plane of a physical surface on which the user is located.

8. The method of claim 4, further comprising:

receiving an input from the user, and

zooming the map in or out, based on the received input.

9. The method of claim 1, further comprising:

detecting a second movement; and

moving the one or more persistent data elements in a manner corresponding to the second movement.

10. The method of claim 1, wherein the one or more persistent data elements comprise one or more icons representative of executable applications.

11. The method of claim 1, wherein the at least one sensor comprises at least one of:

an accelerometer,

a gyroscope,

a magnetometer,

a camera,

a Satellite Positioning System (SPS) receiver, or

an altimeter.

12. An apparatus comprising:

a memory; and

a processing unit communicatively coupled with;

the memory;

one or more motion sensors configured to sense movement of a user of a see-through head mounted display (HMD); and

a display of the HMD, the display configured to show data elements in a user'"'"'s field of view while allowing the user to view physical objects in the user'"'"'s environment;

wherein the processing unit is configured to;

determine a first field of view of the user of the HMD;

receive sensor input from the one or more motion sensors;

detect, using the sensor input, a first movement of the user'"'"'s head about an axis, causing the user to have a second field of view; and

cause the display to show one or more persistent data elements in the second field of view, based on the detected first movement.

13. The apparatus of claim 12, wherein the processing unit is further configured to determine the first field of view is an expected field of view.

14. The apparatus of claim 12, wherein the processing unit is configured to:

detect the first movement by detecting a downward tilt of the user'"'"'s head; and

cause the display to show the one or more persistent data elements wherein the one or more persistent data elements comprises a map.

15. The apparatus of claim 14, wherein the map includes at least one of:

a destination,

a location of the user, or

a point of interest.

16. The apparatus of claim 15, wherein the map includes the point of interest, the processing unit further configured to cause the display to show, within the first field of view of the user, a data element indicative that the map includes the point of interest.

17. The apparatus of claim 14, wherein the processing unit is configured to cause the map to appear, from a perspective of the user, in a plane of a physical surface on which the user is located.

18. The apparatus of claim 14, wherein the processing unit is further configured to perform functions comprising:

receiving an input from the user, and

causing the map to zoom in or out, based on the received input.

19. The apparatus of claim 12, wherein the processing unit is further configured to perform functions comprising:

detecting a second movement; and

causing the display to move the one or more persistent data elements in a manner corresponding to the second movement.

20. The apparatus of claim 12, wherein the apparatus comprises the HMD.

21. The apparatus of claim 12, wherein the apparatus comprises a device communicatively coupled with the HMD.

22. The apparatus of claim 12, wherein processing unit is configured to cause the display to show the one or more persistent data elements by causing the display to show one or more icons representative of executable applications.

23. The apparatus of claim 12, wherein the one or more motion sensors comprise at least one of:

an accelerometer,

a gyroscope,

a magnetometer,

a camera,

a Satellite Positioning System (SPS) receiver, or

an altimeter.

24. A computer-readable storage medium having instructions embedded thereon for displaying data elements on a see-through head mounted display (HMD), the instructions including computer-executable code for:

determining a first field of view of a user of the HMD;

receiving sensor input from at least one sensor;

detecting, using the sensor input, a first movement of the user'"'"'s head about an axis, causing the user to have a second field of view; and

displaying, with the HMD, one or more persistent data elements in the second field of view, based on the detected first movement.

25. An apparatus comprising:

means for determining a first field of view of a user of the HMD;

means for receiving sensor input from at least one sensor;

means for detecting, using the sensor input, a first movement of the user'"'"'s head about an axis, causing the user to have a second field of view; and

means for displaying, via the HMD, one or more persistent data elements in the second field of view, based on the detected first movement.

26. The apparatus of claim 25, further comprising means for determining the first field of view is an expected field of view.

27. The apparatus of claim 25, wherein:

the means for detecting the first movement comprises computer-executable code for detecting a downward tilt of the user'"'"'s head; and

the one or more persistent data elements comprises a map.

28. The apparatus of claim 27, wherein the map includes at least one of:

a destination,

a location of the user, or

a point of interest.

29. The apparatus of claim 28, wherein the map includes the point of interest, the apparatus further comprising means for displaying, within the first field of view of the user, a data element indicative that the map includes the point of interest.

30. The apparatus of claim 27, comprising means for causing the map to appear, from a perspective of the user, in a plane of a physical surface on which the user is located.

31. The apparatus of claim 27, further comprising:

means for receiving an input from the user, and

means for zooming the map in or out, based on the received input.

32. The apparatus of claim 25, further comprising:

means for detecting a second movement; and

means for moving the one or more persistent data elements in a manner corresponding to the second movement.

33. The apparatus of claim 25, wherein the means for displaying the one or more persistent data elements comprises means for displaying one or more icons representative of executable applications.

34. The apparatus of claim 25, wherein the means for receiving sensor input comprise means for receiving input from at least one of:

an accelerometer,

a gyroscope,

a magnetometer,

a camera,

a Satellite Positioning System (SPS) receiver, or

an altimeter.