Information processing device, storage medium storing information processing program, information processing system, and information processing method

• US 9,864,905 B2
• Filed: 01/20/2016
• Issued: 01/09/2018
• Est. Priority Date: 02/10/2015
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Tanaka, Wataru, Kitazono, Yusuke
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US15/001,504
• Publication Number:
  US 20160232674A1
• Time in Patent Office: 720 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An example information processing system includes a camera. The information processing system calculates, based on a camera image obtained by the camera, a position or a direction of an object (e.g., a hand of a user) included in the camera image. The information processing system instructs an output device to produce an output in accordance with the position or the direction of the object. The information processing system displays, on a display device, a guide image including a range image representing a range and an indicator image whose position or direction changes in accordance with a change in the output within the range.

Claims

1. An information processing system comprising a camera, wherein one or more processor of the information processing system is configured to provide execution including:

calculating, based on a camera image obtained by the camera, a position or a direction of an object included in the camera image;

instructing an output device to produce an output in accordance with the position or the direction of the object; and

displaying, on a display device, a guide image including a range image representing a range and an indicator image whose position or direction changes in accordance with a change in the output within the range,

wherein the processor is configured to display, on the display device, the guide image including a predetermined device reference image so that a positional relationship between the indicator image and the predetermined device reference image corresponds to a positional relationship between the object and the camera.

2. The information processing system according to claim 1, wherein the camera is configured to capture an image in a side surface direction of a housing of the information processing system.

3. The information processing system according to claim 1, wherein the indicator image is an image representing the object.

4. The information processing system according to claim 1, wherein when a position of the object cannot be calculated, the processor changes a display mode of the guide image, as compared with that when the position of the object is calculated.

5. The information processing system according to claim 1, wherein when the calculated position of the object is further away toward an un-calculatable area from a predetermined reference position by a predetermined distance or more, the processor changes a display mode of the guide image, as compared with that when the calculated position is within the predetermined distance.

6. The information processing system according to claim 1, wherein the processor is configured to calculate, based on the camera image, a position of the object with respect to an image-capturing direction of the camera.

7. The information processing system according to claim 1, wherein:

the processor is configured to control an output to the output device so that the output is in accordance with a relationship between the calculated position of the object and a predetermined reference position; and

the processor is configured to display, on the display device, the guide image including a process reference image placed at a position corresponding to the reference position within the range represented by the range image.

11. The information processing system according to claim 1, wherein the predetermined device reference image is a camera reference image so that a positional relationship between the indicator image and the camera reference image corresponds to the positional relationship between the object and the camera.

8. A non-transitory computer-readable storage medium storing an information processing program to be performed by a computer of an information processing system comprising a camera, the information processing program instructing the computer to execute:

calculating, based on a camera image obtained by the camera, a position or a direction of an object included in the camera image;

instructing an output device to produce an output in accordance with the position or the direction of the object; and

displaying, on a display device, a guide image including a range image representing a range and an indicator image whose position or direction changes in accordance with a change in the output within the range,

wherein the guide image includes a predetermined device reference image so that a positional relationship between the indicator image and the predetermined device reference image corresponds to a positional relationship between the object and the camera.

12. The non-transitory computer-readable storage medium according to claim 8, wherein the predetermined device reference image is a camera reference image so that a positional relationship between the indicator image and the camera reference image corresponds to the positional relationship between the object and the camera.

9. An information processing device comprising:

a camera, and

one or more processors at least configured to;

calculate, based on a camera image obtained by the camera, a position or a direction of an object included in the camera image;

instruct an output device to produce an output in accordance with the position or the direction of the object; and

display, on a display device, a guide image including a range image representing a range and an indicator image whose position or direction changes in accordance with a change in the output within the range,

wherein the guide image includes a predetermined device reference image so that a positional relationship between the indicator image and the predetermined device reference image corresponds to a positional relationship between the object and the camera.

13. The information processing device according to claim 9, wherein the predetermined device reference image is a camera reference image so that a positional relationship between the indicator image and the camera reference image corresponds to the positional relationship between the object and the camera.

10. An information processing method to be performed on an information processing system comprising a camera, the method comprising:

calculating, based on a camera image obtained by the camera, a position or a direction of an object included in the camera image;

instructing an output device to produce an output in accordance with the position or the direction of the object; and

displaying, on a display device, a guide image including a range image representing a range and an indicator image whose position or direction changes in accordance with a change in the output within range;

wherein the guide image includes a predetermined device reference image so that a positional relationship between the indicator image and the predetermined device reference image corresponds to a positional relationship between the object and the camera.

14. The method according to claim 10, wherein the predetermined device reference image is a camera reference image so that the positional relationship between the indicator image and the camera reference image corresponds to a positional relationship between the object and the camera.