Mobile terminal and method for controlling the same

• US 10,379,622 B2
• Filed: 10/21/2016
• Issued: 08/13/2019
• Est. Priority Date: 12/07/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Park, Hyunji, Lee, Younghoon
• Primary Examiner(s): Zhou, Hong
• Application Number:
  US15/331,653
• Publication Number:
  US 20170160812A1
• Time in Patent Office: 1,026 Days
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/36 by graphic or iconic representation
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays

Abstract

Disclosed are a mobile terminal capable of controlling a head mounted display (HMD) and a controlling method thereof. The mobile terminal includes: a sensor; a communication unit capable of communicating with a head mounted display configured to display a virtual reality screen; and a controller capable of: controlling the head mounted display to display the virtual reality screen including first content; and when a preset gesture input corresponding to a user'"'"'s head or pupil movement is detected via the sensor, controlling the head mounted display to display locked second content on the virtual reality screen while a worn state of the head mounted display is maintained.

Claims

1. A mobile terminal, comprising:

a sensor;

a communication unit capable of communicating with a head mounted display configured to display a virtual reality screen; and

a controller capable of;

controlling the head mounted display to display the virtual reality screen including first content;

when a preset gesture input corresponding to a user'"'"'s head or pupil movement is detected via the sensor, controlling the head mounted display to display locked second content on the virtual reality screen while a worn state of the head mounted display is maintained;

changing the second content to a locked state when the head mounted display is disconnected from the mobile terminal; and

controlling the head mounted display to display different screens corresponding to the locked state according to an output state of the second content when the head mounted display is disconnected from the mobile terminal.

2. The mobile terminal of claim 1, wherein the controller is further capable of:

reconfiguring the virtual reality screen based on the first content and the second content when the preset gesture input is detected; and

controlling the head mounted display to display, on the second content, an object indicating that the second content is in a locked state.

3. The mobile terminal of claim 2, wherein the controller is further capable of controlling at least part of the first content to disappear or controlling a corresponding display change to appear when the second content is displayed.

4. The mobile terminal of claim 2, wherein the second content is displayed on a region differentiated from a region of the first content.

5. The mobile terminal of claim 1, wherein the controller is further capable of activating the sensor while maintaining the displayed state of the first content and causing the mobile terminal to enter an operation mode for releasing a locked content when a second input is detected while the first content is displayed.

6. The mobile terminal of claim 5, wherein the controller is further capable of detecting relative coordinates corresponding to the user'"'"'s head or pupil movement to detect the preset gesture input.

7. The mobile terminal of claim 5, wherein the controller is further capable of controlling the head mounted display to display:

information for guiding a gesture input for releasing the locked content on the virtual reality screen as the mobile terminal enters the operation mode; and

a line corresponding to the user'"'"'s head or pupil movement on the virtual reality screen.

8. The mobile terminal of claim 1, wherein the second content is related to the first content and is determined according to a type of the first content.

9. The mobile terminal of claim 1, wherein the controller is further capable of controlling the head mounted display to display:

a notification icon indicating a released state of the second content when the preset gesture input is detected; and

the second content after the notification icon is displayed.

10. The mobile terminal of claim 1, wherein the controller is further capable of controlling the head mounted display to display the second content differently based on a detached state of the head mounted display.

11. The mobile terminal of claim 10, wherein the controller is further capable of:

controlling the head mounted display to display the second content when the head mounted display is re-worn within a threshold amount of time and the detached state satisfies a first condition; and

changing the second content to a locked state when the detached state satisfies a second condition regardless of whether the head mounted display is re-worn within the threshold amount of time.

12. The mobile terminal of claim 11, wherein:

the first condition corresponds to whether the head mounted display has contacted a ground surface; and

the second condition corresponds to a specific positioning of the head mounted display on the ground surface.

13. The mobile terminal of claim 11, wherein the controller is further capable of controlling the head mounted display to:

no longer display the second content on the virtual reality screen when the second content is changed to the locked state;

or

changing the second content to replacement content when the second content is changed to the locked state.

14. The mobile terminal of claim 1, wherein the second content disappears from the virtual reality screen when the head mounted display is disconnected from the mobile terminal while the first content and the second content are displayed.

15. The mobile terminal of claim 1, wherein the second content is changed to replacement content when the head mounted display is disconnected from the mobile terminal while only the second content is displayed.

16. The mobile terminal of claim 1, wherein the controller is further capable of changing the second content to the locked state when a threshold length of time elapses after the head mounted display is disconnected from the mobile terminal.

17. The mobile terminal of claim 1, wherein the controller is further capable of changing the second content to the locked state in response to a first input when the head mounted display is disconnected from the mobile terminal.

18. A method of controlling a mobile terminal, comprising:

displaying a virtual reality screen on a head mounted display in communication with the mobile terminal, the virtual reality screen including first content;

tracking a user'"'"'s head or pupil movement via a sensor of the head mounted display;

changing at least part of the first content when a preset gesture input is detected in the user'"'"'s head or pupil movement;

displaying locked second content on the virtual reality screen while a worn state of the head mounted display is maintained;

changing the second content to a locked state when the head mounted display is disconnected from the mobile terminal; and

controlling the head mounted display to display different screens corresponding to the locked state according to an output state of the second content when the head mounted display is disconnected from the mobile terminal.

19. The method of claim 18, further comprising:

detecting that the head mounted display is detached while the second content is displayed; and

displaying the second content differently based on a detached state of the head mounted display.