Storage medium having stored thereon information processing program, information processing apparatus, information processing system, and information processing method

• US 9,492,743 B2
• Filed: 02/22/2012
• Issued: 11/15/2016
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Hayashi, Yugo, Sumaki, Kazuya, Ohta, Keizo, Ito, Jun
• Primary Examiner(s): Thompson, James A
• Assistant Examiner(s): Guo, Xilin
• Application Number:
  US13/402,026
• Publication Number:
  US 20120229455A1
• Time in Patent Office: 1,728 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/816 Athletics, e.g. track-and-field sports
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/8005 Athletics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 : Rotation of whole images or parts thereof

Abstract

On the basis of data based on an attitude and/or a motion of a portable display apparatus body, at least a direction of rotation and an amount of rotation of the portable display apparatus about a predetermined direction in real space are calculated, and a corrected amount of rotation is calculated by correcting the calculated amount of rotation by a predetermined factor. Then, an operation indication direction based on the direction of rotation and the corrected amount of rotation is calculated with respect to a direction that corresponds to the predetermined direction and is set in a virtual world, and predetermined processing is performed on the virtual world. A first image including at least a part of the virtual world is displayed on the portable display apparatus.

Claims

1. A non-transitory computer-readable storage medium having stored thereon an information processing program to be executed by a computer of an information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing program causing the computer to perform features comprising:

calculating, on the basis of the attitude and/or motion data output from the portable display apparatus, at least a direction of rotation and an amount of rotation of the portable display apparatus about a predetermined direction in real space;

calculating a corrected amount of rotation by correcting the calculated amount of rotation by a predetermined factor,

calculating, with respect to a direction that corresponds to the predetermined direction and is set in a virtual world, an operation indication direction based on the direction of rotation and the corrected amount of rotation;

performing predetermined processing on the virtual world in accordance with the operation indication direction;

generating a first image including at least a part of the virtual world subjected to the processing; and

displaying the first image on the portable display apparatus.

2. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

in accordance with the operation indication direction, a process is performed of controlling an attitude of a first virtual camera set in the virtual world, and

an image of the virtual world viewed from the first virtual camera is generated as the first image.

3. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 2, wherein

the attitude of the first virtual camera is controlled such that the operation indication direction coincides with a direction of a line of sight of the first virtual camera.

4. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

using as the predetermined direction a horizontal direction in real space perpendicular to a perspective direction of a display screen of the portable display apparatus, a direction of rotation and an amount of rotation of the perspective direction are calculated, and

the operation indication direction is calculated so as to rotate about a horizontal direction in the virtual world on the basis of the direction of rotation and the corrected amount of rotation of the portable display apparatus.

5. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 4, wherein

the direction of rotation and the amount of rotation of the portable display apparatus about each of the predetermined direction, a direction of gravity in real space, and a horizontal direction in real space perpendicular to the predetermined direction are calculated,

the corrected amount of rotation is calculated from the calculated amount of rotation about the predetermined direction, and

the operation indication direction is calculated so as to:

rotate about a horizontal direction in the virtual world on the basis of the direction of rotation and the corrected amount of rotation about the predetermined direction;

rotate about a direction of gravity in the virtual world on the basis of the direction of rotation and the amount of rotation about the direction of gravity in real space; and

rotate about a horizontal direction in the virtual world on the basis of the direction of rotation and the amount of rotation about the horizontal direction in real space perpendicular to the predetermined direction.

6. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 4, wherein

the corrected amount of rotation is calculated such that, by making a magnification correction on the calculated amount of up-down rotation about the predetermined direction by a predetermined factor, the amount of up-down rotation that is less than 90°

is corrected to substantially 90°

.

7. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

a direction of rotation and an amount of rotation of a perspective direction of a display screen of the portable display apparatus are calculated, using as the predetermined direction a direction defined in the portable display apparatus body.

8. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the corrected amount of rotation is calculated by correcting by the predetermined factor the calculated amount of rotation, of a perspective direction of a display screen of the portable display apparatus, about the predetermined direction, and offsetting the perspective direction by a predetermined amount about the predetermined direction.

9. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 4, wherein

the corrected amount of rotation is calculated by correcting by the predetermined factor the calculated amount of rotation of the perspective direction about the predetermined direction, and offsetting the perspective direction in an elevation direction or a depression direction and by a predetermined amount about the predetermined direction.

10. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the corrected amount of rotation is calculated by making a magnification correction on the calculated amount of rotation by a predetermined factor.

11. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the corrected amount of rotation is calculated by linearly transforming the calculated amount of rotation by a predetermined factor.

12. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the corrected amount of rotation is calculated by nonlinearly transforming the calculated amount of rotation by a predetermined factor.

13. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, wherein

the calculated amount of rotation is transformed into the corrected amount of rotation using a nonlinear function in which the predetermined factor incrementally increases in an elevation direction and a depression direction with respect to a predetermined reference direction in real space.

14. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, wherein

the calculated amount of rotation is transformed into the corrected amount of rotation using a nonlinear function in which the predetermined factor decrementally decreases in an elevation direction and a depression direction with respect to a predetermined reference direction in real space.

15. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

when included in a first range, the calculated amount of rotation is transformed into the corrected amount of rotation using a first function, and

when included in a second range, the calculated amount of rotation is transformed into the corrected amount of rotation using a second function.

16. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 15, wherein

the first range is set symmetrically about a predetermined reference direction in real space, and the predetermined factor indicated by the first function is set to be smaller than the predetermined factor indicated by the second function.

17. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 13, wherein

the predetermined reference direction is such that when the amount of rotation indicating the predetermined reference direction is corrected to the corrected amount of rotation, a direction obtained by the corrected amount of rotation is a horizontal direction in the virtual world.

18. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

when the corrected amount of rotation indicates an amount of rotation beyond a vertically up direction in the virtual world, the corrected amount of rotation is calculated so as to be limited to the vertically up direction, and

when the corrected amount of rotation indicates an amount of rotation beyond a vertically down direction in the virtual world, the corrected amount of rotation is calculated so as to be limited to the vertically down direction.

19. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

image data indicating the first image is output to the portable display apparatus, and

the portable display apparatus performs features comprising:

acquiring the image data output from the information processing apparatus, and

displaying the first image indicated by the acquired image data.

20. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 19, the information processing program further causing the computer to perform features comprising:

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus,

the compression image data output from the information processing apparatus is acquired,

the portable display apparatus further performs features comprising decompressing the compression image data to obtain the image data indicating the first image, and

displaying the first image indicated by the acquired and decompressed image data.

21. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

besides the first image, a second image representing the virtual world is further displayed on another display apparatus connected to the information processing apparatus.

22. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, the information processing program further causing the computer to execute

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus, and, besides the compression image data, image data indicating the second image is output to said another display apparatus without being compressed, and

the portable display apparatus performs features comprising:

acquiring the compression image data output from the information processing apparatus;

decompressing the compression image data to obtain the image data indicating the first image; and

displaying the first image indicated by the acquired and decompressed image data.

23. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, wherein

a point of view for generating the second image is set at a position different from a position of a point of view for generating the first image.

24. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, wherein

a point of view for generating the second image is set at a position in the virtual world including a point of view for generating the first image, and a range wider than a range of the virtual world represented by the first image is displayed as the second image on said another display apparatus.

25. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, wherein

a direction of a line of sight from a point of view for generating the second image is set in a vertical direction in the virtual world, and the point of view for generating the second image is set at a position of viewing from a bird'"'"'s-eye view a point of view for generating the first image.

26. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the portable display apparatus includes at least one of a gyro sensor and an acceleration sensor, each of which outputs data based on the attitude and/or the motion of the portable display apparatus body, and

on the basis of the data output from the at least one of the gym sensor and the acceleration sensor, the direction of rotation and the amount of rotation of the portable display apparatus are calculated.

27. An information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing apparatus comprising:

a rotation information calculation unit configured to calculate, on the basis of the attitude and/or motion data output from the portable display apparatus, at least a direction of rotation and an amount of rotation of the portable display apparatus about a predetermined direction in real space;

a correction unit configured to calculate a corrected amount of rotation by correcting by a predetermined factor the amount of rotation calculated by the rotation information calculation unit;

an operation indication direction calculation unit configured to calculate, with respect to a direction that corresponds to the predetermined direction and is set in a virtual world, an operation indication direction based on the direction of rotation and the corrected amount of rotation;

an operation indication direction processing unit configured to perform predetermined processing on the virtual world in accordance with the operation indication direction;

a first image generation unit configured to generate a first image including at least a part of the virtual world subjected to the processing performed by the operation indication direction processing unit; and

a display control unit configured to display the first image on the portable display apparatus.

28. An information processing system including a plurality of apparatuses configured to communicate with each other, the information processing system capable of displaying an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing system comprising:

a rotation information calculation unit configured to calculate, on the basis of the attitude and/or motion data output from the portable display apparatus, at least a direction of rotation and an amount of rotation of the portable display apparatus about a predetermined direction in real space;

a correction unit configured to calculate a corrected amount of rotation by correcting by a predetermined factor the amount of rotation calculated by the rotation information calculation unit;

an operation indication direction calculation unit configured to calculate, with respect to a direction that corresponds to the predetermined direction and is set in a virtual world, an operation indication direction based on the direction of rotation and the corrected amount of rotation;

an operation indication direction processing unit configured to perform predetermined processing on the virtual world in accordance with the operation indication direction;

a first image generation unit configured to generate a first image including at least a part of the virtual world subjected to the processing performed by the operation indication direction processing unit; and

a display control unit configured to display the first image on the portable display apparatus.

29. An information processing method performed by a processor or a corporation of a plurality of processors included in an information processing system including at least one information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing method comprising:

calculating, on the basis of the attitude and/or motion data output from the portable display apparatus, at least a direction of rotation and an amount of rotation of the portable display apparatus about a predetermined direction in real space;

calculating a corrected amount of rotation by correcting the calculated amount of rotation by a predetermined factor,

calculating, with respect to a direction that corresponds to the predetermined direction and is set in a virtual world, an operation indication direction based on the direction of rotation and the corrected amount of rotation;

performing predetermined processing on the virtual world in accordance with the operation indication direction;

generating a first image including at least a part of the virtual world subjected to the processing; and

displaying the first image on the portable display apparatus.