HMD AND METHOD FOR CONTROLLING SAME

• US 20180321493A1
• Filed: 12/09/2015
• Published: 11/08/2018
• Est. Priority Date: 11/11/2015
• Status: Active Application

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: KIM, Sumi, BYUN, Joonwon, OH, Hyunju, NAM, Kyoungduck, KWON, Yongjin, KIM, Sangho, JEONG, Woochan, PARK, Sungjun
• Application Number:
  US15/773,230
• Publication Number:
  US 20180321493A1
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2203/04802 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H03K 17/955 using a capacitive detector
• CPC Class Codes: H03K 17/968 : using opto-electronic devices
• CPC Class Codes: H03K 2217/9651 : the moving element acting on a force, e.g. pressure sensitive element

Abstract

The present invention relates to a tethering type head mounted display (HMD) which is connected to a mobile terminal and a method for controlling the HMD. The HMD, which is connected to a mobile terminal, comprises: a communication unit for performing wired or wireless communication with the mobile terminal; a display unit for outputting image information; a detection unit for detecting a movement of the HMD; and a control unit for controlling the display unit so as to output image information which is controlled according to the result of detecting a movement of the HMD, wherein, when one of predetermined situations occurs, the control unit controls the display unit so as to output image information which is controlled according to a movement detected in the mobile terminal, and when the situation that has occurred ends, the control unit controls the display unit so as to output image information which is controlled according to a movement of the HMD.

Claims

1. An head mounted display (HMD) device connected to a mobile terminal, the HMD comprising:

a communication unit configured to perform wired or wireless communication with the mobile terminal;

a display unit configured to display image information;

a sensing unit configured to sense a movement of the HMD; and

a controller configured to control the display unit to display image information controlled according to a result of sensing the movement of the HMD,

wherein the controller controls the display unit to display image information controlled according to a movement sensed by the mobile terminal when any one of preset situations occurs, and controls the display unit to display image information controlled according to a movement of the HMD when the occurred situation is ended.

2. The HMD device of claim 1, wherein the display unit displays an image of a virtual space according to previously stored content, and

the controller controls the display unit to display an image corresponding to a specific region of the virtual space according to a result of sensing the movement of the HMD, and controls the display unit to display an image corresponding to another region of the virtual space, around an image of the virtual space displayed according to the movement of the HMD, according to a user input sensed from the mobile terminal when a specific situation is sensed, and controls the display unit to display an image of the virtual space controlled according to the movement of the HMD when the sensed specific situation is ended.

3. The HMD device of claim 2, wherein the controller displays a menu screen for allowing a user to select an input for controlling an image of a virtual space displayed on the display unit on the display unit when the specific situation is sensed, and

the menu screen comprises menus for selecting either one of a movement of a user'"'"'s head sensed through the HMD and a user input sensed through the mobile terminal or both the movement of the user'"'"'s head and the user input as an input for controlling an image of the virtual space.

4. The HMD device of claim 3, wherein the controller displays either one menu according to the movement of the user'"'"'s head sensed through the HMD or the user input sensed through the mobile terminal to be distinguished from the other menus, and controls an image of a virtual space displayed on the display unit according to a control method corresponding to the either one of the menus displayed in a distinguished manner.

5. The HMD device of claim 2, wherein when the specific situation is sensed, the controller displays information on devices that control an additional graphic object or the displayed image of the virtual space on the displayed image, and indicates that the HMD is in a state where an image corresponding to another region of the virtual space is displayed based on both the user'"'"'s head movement sensed by the HMD and the user input sensed by the mobile terminal.

6. The HMD device of claim 2, wherein the user'"'"'s input sensed through the mobile terminal is at least one of a drag input applied to a touch screen of the mobile terminal or an angular velocity or acceleration sensed by the mobile terminal.

7. The HMD device of claim 2, wherein the controller controls the display unit to display an image corresponding to another region of the virtual space according to the user'"'"'s head movement sensed by the HMD or the user input sensed through the mobile terminal, based on a specific region preset to correspond to a forward direction of the HMD among regions of the virtual space, and changes a preset specific region according to a user'"'"'s selection to correspond to the forward direction of the HMD.

8. The HMD device of claim 1, wherein the display unit displays image information of content previously stored in the mobile terminal, and

the controller displays image information controlled according to a result of sensing the movement of the HMD, executes a specific function of the mobile terminal according to a user'"'"'s selection, displays a screen related to the execution of the specific function controlled according to a user input sensed through the touch screen of the mobile terminal on the display unit, controls the mobile terminal to restrict a user'"'"'s control signal input when the image information of the content is displayed on the display unit, and controls the mobile terminal to release the restricted user'"'"'s control signal input when a specific user'"'"'s input is sensed.

9. The HMD device of claim 8, wherein the specific function is a function corresponding to an event occurred in the mobile terminal or a function selected according to a preset user'"'"'s input among functions executable in the mobile terminal.

10. The HMD device of claim 9, wherein, when the preset user'"'"'s input is sensed, the controller displays graphic objects corresponding to functions executable in the mobile terminal, respectively, on at least a part of the display unit, and

the specific function is a function corresponding to any one of the graphic objects selected by a user.

11. The HMD device of claim 9, wherein the touch screen of the mobile terminal is partitioned into a plurality of regions set to correspond to a plurality of different functions executable in the mobile terminal, respectively, and

the specific function is a function corresponding to any one of the plurality of regions in which the touch input is sensed.

12. The HMD device of claim 8, wherein the controller displays one point on the display unit corresponding to one point on the touch screen at which the touch input is sensed, on which a screen related to the execution of the specific function is displayed, in a distinguished manner, and determines that the touch input is applied to the one point displayed in a distinguished manner to control a function executed in the mobile terminal.

13. The HMD device of claim 12, wherein when a touch object that applies the touch input to the touch screen approaches within a predetermined distance from the touch screen, the controller senses the touch object, and displays a position of the sensed touch object on a screen related to the execution of the specific function.

14. The HMD device of claim 12, wherein the controller sets one region on the touch screen of the mobile terminal as a touch recognition region according to a user'"'"'s input, and sets each part of the touch recognition region to correspond to each part a region on the display unit displayed with a screen related to the execution of the specific function, and displays one point on the display unit displayed with a screen related to the execution of the specific function corresponding to one point in the touch recognition region in which the touch input is sensed, in a distinguished manner.

15. The HMD device of claim 14, wherein when a touch recognition region is set on the touch screen, the controller changes the shape of a screen related to the execution of the specific function displayed on the display unit according to the shape of the set touch recognition region.

16. The HMD device of claim 1, wherein the controller senses a case where a preset user'"'"'s touch input is sensed on the touch screen of the mobile terminal or a specific touch input gesture is sensed through the mobile terminal as an event that the preset situation has occurred, and senses a case where a specific function executed according to the preset user'"'"'s touch input or the specific touch input gesture is ended or the preset user'"'"'s touch input or the specific touch input gesture is sensed again as an event that the occurred situation is ended.

17. The HMD device of claim 16, wherein the mobile terminal operates in a doze mode when connected to the HMD, and

the doze mode is an operation state capable of sensing at least one of a touch input applied to the touch screen of the mobile terminal and a movement of the mobile terminal while a light emitting device of the touch screen of the mobile terminal is off.

18. The HMD device of claim 1, wherein the controller further senses a case where specific image information is displayed on the display unit or the remaining amount of power of the HMD is less than a preset level as an event that any one of the preset situations has occurred, and senses a case where the display of the specific image information is ended or the remaining amount of power of the HMD is above the preset level as an event that the occurred situation is ended.

19. The HMD device of claim 18, wherein the specific image information corresponds to a specific graphic object, and

the controller displays the specific image information corresponding to the specific graphic object on the display unit when a user gazes at one region on the display unit displayed with the specific graphic object for more than a predetermined period of time.

20. A method of controlling a head mounted display (HMD) connected to a mobile terminal, the method comprising:

displaying image information related to selected content on a display unit provided in the HMD;

sensing a user'"'"'s head movement through a sensor provided in the HMD;

controlling image information displayed on the display unit according to the sensed movement;

sensing an occurrence of a preset situation;

sensing a movement of the mobile terminal based on the occurred specific situation;

controlling image information displayed on the display unit according to the sensed movement of the mobile terminal; and

controlling image information displayed on the display unit based on a movement sensed through the HMD when an end of the preset situation is sensed.