Apparatus and method of providing user interface on head mounted display and head mounted display thereof

• US 9,448,624 B2
• Filed: 06/12/2013
• Issued: 09/20/2016
• Est. Priority Date: 09/14/2012
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Jihwan
• Primary Examiner(s): Nguyen, Jimmy H
• Application Number:
  US13/916,397
• Publication Number:
  US 20140078043A1
• Time in Patent Office: 1,196 Days
• Field of Search: 345 7- 9, 345156-184, 715/773, 715/863, 715/862, 715700-702
• US Class Current: 1/1
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

An apparatus and method of providing a user interface (UI) on head mounted display and the head mounted display (HMD) thereof are discussed. The apparatus comprises a sensor unit detecting whether an object exists in the proximity of the HMD and if the object is detected, the sensor unit senses a distance between the object and the HMD. The apparatus further comprises a processor controlling a User Interface (UI) of the HMD based on a result of the sensor unit. A physical User Interface (UI) mode is applied if the detected object is within a predetermined distance from the HMD and a non-physical User Interface (UI) mode is applied if the object is not detected or is not within the predetermined distance from the HMD.

Claims

1. A method providing a User Interface (UI) for a Head Mounted Display (HMD), the method comprising:

displaying a first UI for a physical UI mode if an object is within a predetermined distance from the HMD, and displaying a second UI different from the first UI for a non-physical UI mode if the object is not detected or is not within the predetermined distance from the HMD;

changing the physical UI mode to the non-physical UI mode if a state of the object is changed to be beyond the predetermined distance from the HMD, and changing the non-physical UI mode to the physical UI mode if the state of the object is changed to be within the predetermined distance from the HMD; and

receiving a user command in the physical UI mode by physically contacting the object and in the non-physical UI mode by not physically contacting the object.

2. The method of claim 1, wherein the displaying the first UI for the physical UI mode includes displaying a virtual UI on a surface of the object.

3. The method of claim 1, wherein the first UI for the physical UI mode comprises at least one of a virtual keyboard User Interface (UI) or a drawing User Interface (UI) on a surface of the object.

4. The method of claim 1, wherein the second UI for the non-physical UI mode comprises at least one of a voice recognition User Interface (UI) or a gesture User Interface (UI).

5. The method of claim 1, further comprising:

determining whether the object is within a viewing angle of the HMD when the object is within the predetermined distance from the HMD.

6. The method of claim 5, further comprising:

displaying the first UI for the physical UI mode if the object is within the viewing angle of the HMD.

7. The method of claim 5, further comprising:

changing the physical UI mode to the non-physical UI mode if the object is detected as not still being within the predetermined distance from the HMD after a predetermined amount of time.

8. The method of claim 5, further comprising:

determining a type of the object; and

changing the physical UI mode to the non-physical UI mode if the type of object is determined as not being compatible with a virtual interface in the physical UI mode.

9. A User Interface (UI) apparatus for a Head Mounted Display (HMD) comprising:

a display unit configured to display a first UI for a physical UI mode if an object is within a predetermined distance from the HMD, and display a second UI different from the first UI for a non-physical UI mode if the object is not detected or is not within the predetermined distance from the HMD; and

a processor configured to change the physical UI mode to the non-physical UI mode if a state of the object is changed to be beyond the predetermined distance from the HMD, and change the non-physical UI mode to the physical UI mode if the state of the object is changed to be within the predetermined distance from the HMD,

wherein the processor is further configured to receive a user command in the physical UI mode by physically contacting the object and in the non-physical UI mode by not physically contacting the object.

10. The apparatus of claim 9, wherein the display unit is configured to display in the physical UI mode a virtual UI on a surface of the object.

11. The apparatus of claim 9, wherein the first UI for the physical UI mode comprises at least one of a virtual keyboard User Interface (UI) or a drawing User Interface (UI) on a surface of the object.

12. The apparatus of claim 9, wherein the second UI for the non-physical UI mode comprises at least one of a voice recognition User Interface (UI) or a gesture User Interface (UI).

13. The apparatus of claim 9, wherein the processor is further configured to:

determine whether the object is within a viewing angle of the HMD when the object is within the predetermined distance from the HMD.

14. The apparatus of claim 13, wherein the display unit is further configured to display the first UI for the physical UI mode if the object is within the viewing angle of the HMD.

15. The apparatus of claim 13, wherein the processor is further configured to:

change the physical UI mode to the non-physical UI mode if the object is detected as not still being within the predetermined distance from the HMD after a predetermined amount of time.

16. The apparatus of claim 13, wherein the processor is further configured to:

determine a type of the object, and

change the physical UI mode to the non-physical UI mode if the type of the object is determined as not being compatible with a virtual interface in the physical UI mode.

17. The apparatus of claim 9, wherein the processor is further configured to:

detect an external device within the predetermined distance from the HMD if the object is detected as not still being within the predetermined distance from the HMD, and

wirelessly communicate with the external device and using a display of the external device as the physical UI mode.

18. A method providing a User Interface (UI) for a Head Mounted Display (HMD), the method comprising:

displaying a first UI for a physical UI mode if the object is a predetermined distance from a user wearing the HMD, and displaying a second UI different from the first UI for a non-physical UI mode if the object is not detected or is not within the predetermined distance from the user;

changing the physical UI mode to the non-physical UI mode if a state of the object is changed to be beyond the predetermined distance of the user, and changing the non-physical UI mode to the physical UI mode if the state of the object is changed to be within the predetermined distance from the user; and

receiving a user command in the physical UI mode by physically contacting the object and in the non-physical UI mode by not physically contacting the object.