Storage medium having stored thereon information processing program, information processing apparatus, information processing system, and information processing method

• US 9,526,981 B2
• Filed: 02/21/2012
• Issued: 12/27/2016
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Hayashi, Yugo, Sumaki, Kazuya
• Primary Examiner(s): Thompson, James A
• Assistant Examiner(s): Guo, Xilin
• Application Number:
  US13/400,944
• Publication Number:
  US 20120229453A1
• Time in Patent Office: 1,771 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/816 Athletics, e.g. track-and-field sports
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/8005 Athletics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 : Rotation of whole images or parts thereof

Abstract

On the basis of data output from a portable display apparatus, a direction of rotation of the portable display apparatus about a predetermined direction in real space is calculated. Attitudes of an object, placed in a virtual world, and a first virtual camera, for generating an image of the virtual world including at least the object, are controlled such that the object and the first virtual camera rotate in the direction of rotation about a direction that corresponds to the predetermined direction and is set in the virtual world. Then, a first image representing the virtual world viewed from the first virtual camera is displayed on the portable display apparatus.

Claims

1. A non-transitory computer-readable storage medium having stored thereon an information processing program to be executed by a computer of an information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing program causing the computer to perform functionality comprising:

calculating, on the basis of the attitude and/or motion data output from the portable display apparatus, a direction of rotation of the portable display apparatus about a predetermined direction in real space;

controlling an attitude of an object, placed in a virtual world, such that the object rotates in the direction of rotation about a direction that corresponds to the predetermined direction and is set in the virtual world;

controlling an attitude of a first virtual camera, for generating an image of the virtual world including at least the object, such that the first virtual camera rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world; and

displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

2. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

an amount of rotation of the portable display apparatus about the predetermined direction in real space is further calculated on the basis of the attitude and/or motion data output from the portable display apparatus,

the attitude of the object is controlled such that the object rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world, and by an amount of rotation based on the amount of rotation of the portable display apparatus, and

the attitude of the first virtual camera is controlled such that the first virtual camera rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world, and by an amount of rotation based on the amount of rotation of the portable display apparatus.

3. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 2, wherein

the attitude of the first virtual camera is controlled such that the first virtual camera rotates in the direction of rotation by the same amount of rotation as the amount of rotation of the portable display apparatus.

4. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, the information processing program further causing the computer to perform functionality comprising:

calculating, on the basis of the attitude and/or motion data output from the portable display apparatus, a direction of gravity relative to the portable display apparatus, wherein

the direction of rotation and the amount of rotation of the portable display apparatus are calculated with respect to the direction of gravity, and

the attitude of the first virtual camera is controlled such that when the first image is displayed on the portable display apparatus, the direction of gravity relative to the portable display apparatus displaying the first image coincides with a direction of gravity in the virtual world displayed as the first image.

5. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

the object is controlled so as to rotate in the direction of rotation and by the same amount of rotation as the amount of rotation of the portable display apparatus, and to have such an attitude and position that a positional relationship between the object and the first virtual camera is fixed.

6. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

using as the predetermined direction a predetermined axis provided in the portable display apparatus, the direction of rotation and the amount of rotation of the portable display apparatus about the predetermined axis are calculated.

7. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 6, wherein

using as the predetermined axis a perspective direction of a display screen that displays the first image on the portable display apparatus, the direction of rotation and the amount of rotation of portable display apparatus about the perspective direction are calculated, and

the attitude of the first virtual camera is controlled such that, using a direction of a line of sight of the first virtual camera as a direction corresponding to the perspective direction, the first virtual camera rotates about the direction of the line of sight, in the direction of rotation and by the same amount of rotation as the amount of rotation of the portable display apparatus.

8. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 7, wherein

further using, as the predetermined axis, each of a height direction and a width direction of the portable display apparatus that are orthogonal to the perspective direction and are orthogonal to each other, the direction of rotation and the amount of rotation of the portable display apparatus about each of the height direction and the width direction are calculated, and

the attitude of the first virtual camera is controlled such that:

the first virtual camera rotates about a height direction orthogonal to the direction of the line of sight of the first virtual camera, in the same direction of rotation and by the same amount of rotation as the direction of rotation and the amount of rotation about the height direction of the portable display apparatus; and

the first virtual camera rotates about a width direction orthogonal to the direction of the line of sight of the first virtual camera and orthogonal to the height direction orthogonal to the direction of the line of sight of the first virtual camera, in the same direction of rotation and by the same amount of rotation as the direction of rotation and the amount of rotation of the portable display apparatus about the width direction of the portable display apparatus.

9. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

the attitude of the first virtual camera in the virtual world is controlled so as to be the same as the attitude of the portable display apparatus in real space.

10. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

image data indicating the first image is output to the portable display apparatus, and the portable display apparatus performs functionality comprising:

acquiring the image data output from the information processing apparatus, and displaying the first image indicated by the acquired image data.

11. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 10, the information processing program further causing the computer to perform functionality comprising:

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus,

the compression image data output from the information processing apparatus is acquired,

the portable display apparatus further performs functionality comprising decompressing the compression image data to obtain the image data indicating the first image, and

displaying the first image indicated by the acquired and decompressed image data.

12. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

besides the first image, a second image representing the virtual world viewed from a second virtual camera is further displayed on another display apparatus connected to the information processing apparatus.

13. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, the information processing program further causing the computer to perform functionality comprising:

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus, and, besides the compression image data, image data indicating the second image is output to said another display apparatus without being compressed, and the portable display apparatus performs functionality comprising;

acquiring the compression image data output from the information processing apparatus;

decompressing the compression image data to obtain the image data indicating the first image; and

displaying the first image indicated by the acquired and decompressed image data.

14. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, the information processing program further causing the computer to perform functionality comprising:

setting, on the basis of a position of the object in the virtual world, the second virtual camera, for generating an image of the virtual world, at a position different from a position of the first virtual camera such that the object is included in the second image.

15. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 14, wherein

the second virtual camera is set at a position further away from the object than the first virtual camera is from the object, and

a range wider than a range of the virtual world represented by the first image is displayed as the second image on said another display apparatus.

16. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 14, wherein

the second virtual camera is set at a position of viewing the object from a bird'"'"'s-eye view in the virtual world, and

an image obtained by viewing from a bird'"'"'s-eye view the object placed in the virtual world is displayed as the second image on said another display apparatus.

17. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the portable display apparatus includes at least one of a gyro sensor and an acceleration sensor, each of which outputs data based on the attitude and/or the motion of the portable display apparatus body, and

on the basis of the data output from the at least one of the gyro sensor and the acceleration sensor, the direction of rotation and an amount of rotation of the portable display apparatus are calculated.

18. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein the attitude of the object is controlled to correspond to the predetermined direction while the predetermined direction is within a specified range where the attitude of the first virtual camera is controlled to correspond to the predetermined direction regardless of whether the predetermined direction is within the specified range.

19. An information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing apparatus comprising at east one processor, the apparatus configured to:

calculate, on the basis of the attitude and/or motion data output from the portable display apparatus, a direction of rotation of the portable display apparatus about a predetermined direction in real space;

control an attitude of an object, placed in a virtual world, such that the object rotates in the direction of rotation about a direction that corresponds to the predetermined direction and is set in the virtual world;

control an attitude of a first virtual camera, for generating an image of the virtual world including at least the object, such that the first virtual camera rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world; and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

20. An information processing system including a plurality of apparatuses configured to communicate with each other, the information processing system configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing system comprising:

a processing system having at least one processor, the processing system configured to;

calculate, on the basis of the attitude and/or motion data output from the portable display apparatus, a direction of rotation of the portable display apparatus about a predetermined direction in real space,

control an attitude of an object, placed in a virtual world, such that the object rotates in the direction of rotation about a direction that corresponds to the predetermined direction and is set in the virtual world,

control an attitude of a first virtual camera, for generating an image of the virtual world including at least the object, such that the first virtual camera rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world, and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

21. An information processing method performed by a processor or a corporation of a plurality of processors included in an information processing system including at least one information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing method comprising:

calculating, on the basis of the attitude and/or motion data output from the portable display apparatus, a direction of rotation of the portable display apparatus about a predetermined direction in real space;

controlling an attitude of an object, placed in a virtual world, such that the object rotates in the direction of rotation about a direction that corresponds to the predetermined direction and is set in the virtual world;

controlling an attitude of a first virtual camera, for generating an image of the virtual world including at least the object, such that the first virtual camera rotates in the direction of rotation about the direction that corresponds to the predetermined direction and is set in the virtual world; and

displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.