Virtual image display program, virtual image display apparatus, and virtual image display method

• US 10,406,432 B2
• Filed: 06/29/2016
• Issued: 09/10/2019
• Est. Priority Date: 09/16/2015
• Status: Active Grant

• Current Assignee: GREE Inc.
• Original Assignee: GREE Inc.
• Inventors: Watanabe, Masashi
• Primary Examiner(s): Skaarup, Jason
• Application Number:
  US15/196,410
• Publication Number:
  US 20170072305A1
• Time in Patent Office: 1,168 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/453 Help systems
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A game processing apparatus displays an image of virtual space on a display that displays an image by using a sensor for identifying the position and direction of a certain body part of a player and the display. A controller in the game processing apparatus displays the image of the virtual space on the display in accordance with the position and direction of the certain body part of the player, and, when an information providing condition regarding the position and direction of the certain body part of the player is satisfied, outputs to-be-provided information.

Claims

1. A computer program product embodied on a non-transitory computer-readable medium, comprising code executable by a virtual image display apparatus having at least a processor and a memory, the memory being configured to store an information providing condition of the virtual image display apparatus and being further configured to store to-be-provided information, to cause the virtual image display apparatus to carry out the following steps:

detecting, with a sensor operationally linked to the virtual image display apparatus, a movement of a body part of a player, the body part comprising at least one of a head of the player and an eye of the player, and the sensor being at least one of the set of;

a gyro sensor configured to measure movement of the head of the player, an acceleration sensor configured to measure movement of the head of the player, a geomagnetic sensor configured to measure movement of the head of the player and a line-of-sight sensor configured to measure movement of the eye of the player; and

determining, based on the movement of the body part of the player, a position and direction of the body part of a player;

displaying, on a display operationally linked to the virtual image display apparatus, in accordance with the position and direction of the body part of the player, an image of a virtual space including a first area and a second area; and

with the virtual image display apparatus, providing, when the information providing condition is satisfied, the to-be-provided information to the player by displaying the to-be-provided information in the second area;

wherein the information providing condition is a condition of a gaze position moving to the second area from the first area, the gaze position being specified by at least one of the body part of the player being in a specified position or the direction of the body part of the player being at least a specified direction.

2. The computer program product of claim 1, wherein

the virtual space includes a target object selectable by a gaze of the player,

wherein an area including the target object is recorded in the memory,

wherein the information providing condition includes a condition wherein a gaze position of the player, the gaze position being identified from the position and the direction of the body part of the player, becomes directed outside the area, and

further comprising providing, with the virtual image display apparatus, the to-be-provided information to the player when the gaze position of the player becomes directed outside the area.

3. The computer program product of claim 1, wherein

the virtual space includes a target object selectable by a gaze of the player,

wherein the information providing condition includes a condition wherein a gaze position of the player, the gaze position being identified from the position and direction of the body part of the player, moves outside the target object before the player completes selecting the target object, and

further comprising providing, with the virtual image display apparatus, the to-be-provided information to the player when the gaze position of the player moves outside the target object before the player completes selecting the target object.

4. The computer program product of claim 1, wherein

the information providing condition comprises a reference range regarding movement of the body part of the player,

wherein the information providing condition includes a condition wherein the movement of the body part of the player has gone outside the reference range, and

further comprising providing, with the virtual image display apparatus, the to-be-provided information to the player when the movement of the body part of the player becomes outside the reference range.

5. The computer program product of claim 1, wherein

the memory comprises a recording of a predetermined movement of the body part of the player,

wherein the information providing condition includes a condition wherein movement of the body part of the player corresponds to the predetermined movement recorded in the memory, and

further comprising providing, with the virtual image display apparatus, the to-be-provided information to the player when the movement of the body part of the player corresponds to the predetermined movement recorded in the memory.

6. The computer program product of claim 1, further comprising displaying, with the virtual image display apparatus, the to-be-provided information on an object in the virtual space.

7. The computer program product of claim 1, further comprising displaying, with the virtual image display apparatus, the to-be-provided information in a direction in which a moving body is moving, wherein the moving body is an object in the virtual space which moves in the virtual space.

8. A virtual image display apparatus that is configured to display an image of virtual space on a display, the image of virtual space of the display being based on the output of a detector for identifying a position and direction of a body part of a player, the body part comprising at least one of a head of the player and an eye of the player, the detector being at least one of the set of:

a gyro sensor configured to measure movement of the head of the player, an acceleration sensor configured to measure movement of the head of the player, a geomagnetic sensor configured to measure movement of the head of the player and a line-of-sight sensor configured to measure movement of the eye of the player, the detector configured to identify the position and direction of the body part of the player based on the movement of the body part of the player, the virtual image display apparatus comprising;

a memory, the memory comprising a recording of an information providing condition and to-be-provided information; and

a processor, the processor being configured to display the image of the virtual space on the display in accordance with the position and direction of the certain body part of the player, the virtual space including a first area and a second area;

the processor further being configured to output the to-be-provided information to the player when the information providing condition is satisfied by displaying the to-be-provided information in the second area;

wherein the information providing condition is a condition of a gaze position moving to the second area from the first area, the gaze position being specified by at least one of the body part of the player being in a specified position or the direction of the body part of the player being at least a specified direction.

9. A virtual image display method for displaying an image of virtual space on a display that displays an image by using a virtual image display apparatus, a detector for identifying a position and direction of a body part of a player, the body part comprising at least one of a head of the player and an eye of the player, the detector being at least one of the set of:

a gyro sensor configured to measure movement of the head of the player, an acceleration sensor configured to measure movement of the head of the player, a geomagnetic sensor configured to measure movement of the head of the player and a line-of-sight sensor configured to measure movement of the eye of the player, the detector configured to identify the position and direction of the body part of the player based on the movement of the body part of the player, and the display, the virtual image display apparatus including a processor and a memory, the memory comprising a recording of an information providing condition and to-be-provided information, the method comprising;

with the processor, displaying the image of the virtual space on the display in accordance with the position and direction of the certain body part of the player, the virtual space including a first area and a second area; and

with the processor, providing, when the information providing condition is satisfied, the to-be-provided information to the player by displaying the to-be-provided information in the second area;

wherein the information providing condition is a condition of a gaze position moving to the second area from the first area, the gaze position being specified by at least one of the body part of the player being in a specified position or the direction of the body part of the player being at least a specified direction.