Head-mounted display controlled by sightline, method for controlling same, and computer program for controlling same

• US 10,261,581 B2
• Filed: 09/17/2015
• Issued: 04/16/2019
• Est. Priority Date: 09/18/2014
• Status: Active Grant

• Current Assignee: FXGear, Inc.
• Original Assignee: FXGear, Inc.
• Inventors: Kim, Sung Wu, Choi, Kwang Jin
• Primary Examiner(s): Michaud, Robert J
• Application Number:
  US15/511,608
• Publication Number:
  US 20170255256A1
• Time in Patent Office: 1,307 Days
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2320/068 for control of viewing angle adjustment
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 3/2092 Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto suitable for both CRT and flat panel G09G5/003; specific for a CRT G09G1/165
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092

Abstract

A head-mounted display (HMD) may comprise: a sensor unit configured to measure the movement of a user wearing the HMD; a display unit configured to display an image; and a control unit configured to calculate the user'"'"'s sightline direction from a measurement value from the sensor unit and, when the user'"'"'s sightline direction satisfies a preset condition, control elements of the image other than the viewpoint using the user'"'"'s sightline as a input signal. When the user'"'"'s sightline direction satisfies the preset condition, the HMD controls the image, which is displayed on the HMD, using the user'"'"'s sightline as an input signal, thereby performing event processing and the like without any separate controller.

Claims

1. A head-mounted display, comprising:

a sensor unit configured to measure a motion of a user wearing the head-mounted display;

a display unit configured to comprise a screen, to display a part of a wide viewing angle image corresponding to the virtual space and comprising one or more objects mapped to specific coordinates in the virtual space as an image on the screen, to change a viewpoint of the image by moving the displayed part of the wide viewing angle image in response to the motion of a head of the user wearing the head-mounted display; and

a control processor configured to compute coordinates in the virtual space in which a line of sight direction of the user is from a measurement value of the sensor unit, and control an element other than the viewpoint of the image using the line of sight of the user as an input signal when the coordinates in the line of sight direction is located at the one or more objects,

wherein the control processor is configured to display a cursor indicating the line of sight direction of the user on the screen,

wherein the control processor is further configured to operate in a first mode and a second mode shifting between the first mode and the second mode, the first mode in which the image displayed on the display unit changes in the viewpoint of the image based on the cursor indicating the line of sight direction of the user, and the second mode in which the element other than the viewpoint of the image is controlled using the cursor indicating the line of sight of the user as the input signal.

2. The head-mounted display according to claim 1, wherein the control processor is configured to change at least one of a type, an output region, an output form, and a playback speed of the image displayed on the display unit using the line of sight of the user as the input signal.

3. A method for controlling a head-mounted display, comprising:

displaying a part of a wide viewing angle image corresponding to the virtual space and comprising one or more objects mapped to specific coordinates in the virtual space as an image on a screen on the head-mounted display;

measuring a motion of a user wearing the head-mounted display by a sensor unit;

changing a viewpoint of the image by moving the displayed part of the wide viewing angle image in response to the motion of a head of the user wearing the head-mounted display;

computing, by a control processor, coordinates in the virtual space in which a line of sight direction of the user is from a measurement value of the sensor unit; and

controlling, by the control processor, an element other than the viewpoint of the image using the line of sight of the user as an input signal when the coordinates in the line of sight direction is located at the one or more objects,

wherein the controlling an element other than the viewpoint of the image comprises;

displaying, by the control processor, a cursor indicating the line of sight direction of the user on the screen,

wherein the controlling an element other than the viewpoint of the image comprises;

operating, by the control processor, in a first mode and a second mode shifting between the first mode and the second mode, the first mode in which the image displayed on the display unit changes in the viewpoint of the image based on the cursor indicating the line of sight direction of the user, and the second mode in which the element other than the viewpoint of the image is controlled using the cursor indicating the line of sight of the user as the input signal.

4. The method for controlling a head-mounted display according to claim 3, wherein the controlling of an element other than viewpoint of the image comprises changing at least one of a type, an output region, an output form, and a playback speed of the image displayed on the head-mounted display using the line of sight of the user as the input signal.

5. A computer program product comprising a non-transitory computer-readable recording medium storing executable code, the code when executed causes the steps of:

displaying a part of a wide viewing angle image corresponding to a virtual space and comprising one or more objects mapped to specific coordinates in the virtual space as an image on a screen on a head-mounted display;

measuring a motion of a user wearing the head-mounted display by a sensor unit;

changing a viewpoint of the image by moving the displayed part of the wide viewing angle image in response to the motion of a head of the user wearing the head-mounted display;

computing, by a control processor, coordinates in the virtual space in which a line of sight direction of the user is from a measurement value of the sensor unit; and

controlling, by the control processor, an element other than the viewpoint of the image using the line of sight of the user as an input signal when the coordinates in the line of sight direction is located at the one or more objects,

wherein the controlling an element other than the viewpoint of the image comprises;

displaying, by the control processor, a cursor indicating the line of sight direction of the user on the screen,

wherein the controlling an element other than the viewpoint of the image comprises;

operating, by the control processor, in a first mode and a second mode shifting between the first mode and the second mode, the first mode in which the image displayed on the display unit changes in the viewpoint of the image based on the cursor indicating the line of sight direction of the user, and the second mode in which the element other than the viewpoint of the image is controlled using the cursor indicating the line of sight of the user as the input signal.