Head mounted display device and method for controlling the same

• US 10,134,358 B2
• Filed: 11/18/2016
• Issued: 11/20/2018
• Est. Priority Date: 05/17/2016
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Jung, Heonjae, Kwon, Yunmi, Hong, Jihee
• Primary Examiner(s): Sharifi-Tafreshi, Koosha
• Application Number:
  US15/355,774
• Publication Number:
  US 20170337897A1
• Time in Patent Office: 732 Days
• Field of Search: 345 7, 345 8, 345633, 359630
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 2203/0383 Remote input, i.e. interface arrangements in which the signals generated by a pointing device are transmitted to a PC at a remote location, e.g. to a PC in a LAN
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones

Abstract

A head mounted display (HMD) device including a communication processor configured to communicate with a mobile terminal; a display configured to display a content received from the mobile terminal; and a controller configured to in response to an event generated on the mobile terminal, selectively display an object on the display corresponding to the event generated on the mobile terminal, based on a type of the displayed content.

Claims

1. A head mounted display (HMD) device, comprising:

a display configured to display a content received from a mobile terminal; and

a controller configured to;

in response to an event generated on the mobile terminal, selectively display an object on the display corresponding to the event generated on the mobile terminal, based on a type of the displayed content,

wherein the controller is further configured to restrict displaying of the object while the content is being displayed when the event generated on the mobile terminal corresponds to an application related to the content displayed on the display of the HMD device.

2. The HMD device of claim 1, wherein the controller is further configured to change an image of the object based on an execution degree of the content displayed on the display of the HMD device.

3. The HMD device of claim 2, wherein the execution degree of the content includes at least one of an execution time of the content, an execution frequency of the content, an execution state of the content, and an entry degree into a lower content from the content displayed on the display of the HMD device.

4. The HMD device of claim 1, wherein the controller is further configured to display the object on the display corresponding to the event generated on the mobile terminal when the event is generated from a preset application and satisfies a specific condition preset by a user input.

5. The HMD device of claim 1, wherein in response to a preset input to the displayed object being received, the controller is further configured to move the object on the display along with a movement of the HMD device, while the displayed content is also moved to correspond to the movement of the HMD device.

6. The HMD device of claim 1, wherein in response to a movement of the HMD device being sensed, the controller is further configured to move the object on the display to correspond to the movement of the HMD device together with the displayed content, and stop displaying the object when a predetermined time lapses.

7. The HMD device of claim 1, wherein the controller is further configured to:

in response to a user'"'"'s eye line being fixed to the object for a predetermined time, change an image of the object and display information related to the event on the changed image.

8. The HMD device of claim 1, wherein the controller is further configured to move the object to an edge region of the display in response to a first touch input applied to a touch screen of the mobile terminal.

9. The HMD device of claim 8, wherein the edge region is divided into a plurality of regions according to an application type, and

wherein the controller is further configured to move the object to a corresponding edge region according to an application of the event corresponding to the object.

10. The HMD device of claim 8, wherein when the object is moved to the corresponding edge region, the controller is further configured to change the object into a preset image.

11. The HMD device of claim 8, wherein the controller is further configured to display information about the event corresponding to the object on the content, in response to a user'"'"'s eye line being fixed to the edge region for a predetermined time.

12. The HMD device of claim 8, wherein in response to a plurality of touch points on the touch screen of the mobile terminal being dragged to a preset direction while the content is being displayed, the controller is further configured to enter an operation mode for checking events corresponding to objects which have moved to the edge region.

13. The HMD device of claim 12, wherein the controller is further configured to:

display the events corresponding to the objects which have moved to the edge region, on the content in an integrated manner, and

display information about an event occurrence time on each of the objects.

14. The HMD device of claim 1, wherein in response to a preset touch gesture applied to a touch screen of the mobile terminal, the controller is further configured to stop displaying the object on the display of the HMD device.

15. The HMD device of claim 1, wherein the controller is further configured to change a position of the object along a touch path of a touch gesture applied to a touch screen of the mobile terminal.

16. The HMD device of claim 1, wherein in response to the HMD device being moved in a first direction while the object is displayed, the controller is further configured to move the displayed content in a second direction opposite to the first direction, and

display information related to the event on the image of the object.

17. The HMD device of claim 16, wherein the controller is further configured to control a display degree of the information related to the event, based on a moving distance of the HMD device in the first direction.

19. The HMD device of claim 1, wherein the restriction of displaying the object is released and the object is displayed on the display of the HMD device when a preset condition is satisfied.

18. A method of controlling a head mounted display (HMD) device, the method comprising:

displaying, via a display of the HMD device, a content received from a mobile terminal communicating with the HMD device;

in response to an event generated on the mobile terminal, selectively displaying, via a display of the HMD device, an object corresponding to the event generated on the mobile terminal, based on a type of the displayed content; and

restricting displaying of the object while the content is being displayed when the event generated on the mobile terminal corresponds to an application related to the content displayed on the display of the HMD device.