Storage medium having stored thereon information processing program, information processing apparatus, information processing system, and information processing method

• US 9,345,962 B2
• Filed: 01/05/2012
• Issued: 05/24/2016
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Hayashi, Yugo, Minato, Keiichi
• Primary Examiner(s): Liu, Zhengxi
• Assistant Examiner(s): Gray, Ryan M
• Application Number:
  US13/343,913
• Publication Number:
  US 20120229512A1
• Time in Patent Office: 1,601 Days
• Field of Search: 345/158, 345/173
• US Class Current: 1/1
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/816 Athletics, e.g. track-and-field sports
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/8005 Athletics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 : Rotation of whole images or parts thereof

Abstract

An image is displayed on a portable display apparatus that outputs at least body state data based on an attitude and/or a motion of the portable display apparatus body and touch position data based on a touch position on a touch panel provided in the portable display apparatus. On the basis of the body state data and the touch position data, an attitude and/or a position of a first virtual camera for generating an image of a virtual world is controlled. A first image representing the virtual world viewed from the first virtual camera is displayed on the portable display apparatus.

Claims

1. A non-transitory computer-readable storage medium comprising an information processing program to be executed by a computer of an information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least body state data based on an attitude and/or a motion of the portable display apparatus body and touch position data based on a touch position on a touch panel provided in the portable display apparatus, the information processing program causing the computer to:

calculate the attitude and/or the motion of the portable display apparatus on the basis of the body state data;

independently control an attitude of a first virtual camera for generating an image of a virtual world on the basis of the attitude and/or the motion of the portable display apparatus such that the first virtual camera rotates relative to the virtual world;

independently control a position of the first virtual camera for generating the image of the virtual world to move in a direction relative to a direction of input on the touch panel, the position of the first virtual camera being controlled on the basis of the touch position indicated by the touch position data such that the first virtual camera moves relative to the virtual world; and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

2. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the attitude of the first virtual camera is controlled on the basis of the attitude and/or the motion of the portable display apparatus such that the first virtual camera rotates relative to an object placed in the virtual world, and the position of the first virtual camera is controlled, on the basis of the touch position indicated by the touch position data, such that the first virtual camera moves relative to the object.

3. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, the information processing program further causing the computer to:

control, on the basis of the calculated attitude and/or motion of the portable display apparatus and the touch position indicated by the touch position data, an action of a player object placed in the virtual world, wherein

the first virtual camera is placed at a position of generating an image of the virtual world, the image including at least the player object.

4. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein the first virtual camera is placed at a position of looking down from directly above the player object in the virtual world.

5. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

an attitude of the player object is controlled on the basis of the attitude and/or the motion of the portable display apparatus such that the player object rotates relative to the virtual world about a vertical direction in the virtual world, in the same direction of rotation and by the same angle of rotation as a direction of rotation and an angle of rotation of the first virtual camera, and

a position of the player object is controlled, on the basis of the touch position indicated by the touch position data, such that the player object moves relative to the virtual world in the same moving direction and at the same moving distance as a moving direction and a moving distance of the first virtual camera.

6. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, the information processing program further causing the computer to:

acquire data based on a load applied to a load detection device, wherein

a position of the player object is controlled on the basis of the acquired data such that the player object moves relative to the virtual world.

7. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, the information processing program further causing the computer to:

calculate, on the basis of the touch position data, a displacement of the touch position input to the touch panel, wherein

the first virtual camera is controlled on the basis of the displacement of the touch position so as to move relative to the virtual world.

8. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 7, wherein

a moving direction in which the first virtual camera is to be caused to move is calculated on the basis of a direction of the displacement of the touch position, and the first virtual camera is controlled so as to move relative to the virtual world in the moving direction.

9. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 8, wherein

a direction opposite to the direction of the displacement of the touch position is calculated as the moving direction.

10. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 9, wherein

a direction opposite to a direction that corresponds to the direction of the displacement and is included in the virtual world displayed so as to overlap the displacement of the touch position is calculated as the moving direction.

11. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 7, wherein

a moving distance at which the first virtual camera is to be caused to move is calculated on the basis of an amount of the displacement of the touch position, and the first virtual camera is controlled so as to move relative to the virtual world at the moving distance.

12. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 11, wherein

a distance corresponding to the amount of the displacement and included in the virtual world displayed so as to overlap the displacement of the touch position is calculated as the moving distance.

13. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

at least an attitude and/or a motion of the portable display apparatus that are obtained by rotating the portable display apparatus about a perspective direction of, and perpendicular to, a display screen of the portable display apparatus are calculated, and

the attitude of the first virtual camera is controlled in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the perspective direction.

14. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 13, wherein

the attitude of the first virtual camera is controlled such that in accordance with a direction of rotation in which the portable display apparatus rotates about the perspective direction, the first virtual camera rotates relative to the virtual world about a direction of a line of sight of the first virtual camera and in the direction of rotation.

15. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 13, wherein

the attitude of the first virtual camera is controlled such that the first virtual camera rotates relative to the virtual world, in accordance with an angle by which a facing direction of the portable display apparatus body changes about the perspective direction.

16. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 15, wherein

the attitude of the first virtual camera is controlled such that the first virtual camera rotates relative to the virtual world by the same angle as the angle by which the facing direction of the portable display apparatus body changes.

17. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 13, wherein

the attitude of the first virtual camera is controlled such that in accordance with a direction of rotation and an angle of rotation in and by which the portable display apparatus rotates about the perspective direction, the first virtual camera rotates relative to the virtual world about a direction of a line of sight of the first virtual camera in the direction of rotation and by the angle of rotation;

and

when the touch position data indicates an operation of dragging the touch panel, the position of the first virtual camera is controlled such that in accordance with a moving direction and a moving distance of the touch position in the virtual world displayed on the display screen so as to overlap the touch position indicated by the touch position data, the first virtual camera moves relative to the virtual world along a plane perpendicular to the direction of the line of sight in a direction opposite to the moving direction and at the moving distance.

18. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the portable display apparatus includes at least one of a gyro sensor and an acceleration sensor, and

the attitude and/or the motion of the portable display apparatus are calculated on the basis of data output from the at least one of the gyro sensor and the acceleration sensor.

19. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

image data indicating the first image is output to the portable display apparatus, and

the portable display apparatus configured to:

acquire the image data output from the information processing apparatus, and

display the first image, on a display screen of the portable display apparatus, indicated by the acquired image data.

20. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 19, the information processing program further causing the computer to:

generate compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus,

the compression image data output from the information processing apparatus is acquired by the portable display apparatus,

the portable display apparatus is further configured to decompress the compression image data to obtain the image data indicating the first image, and

the display screen of the portable display apparatus displays the first image indicated by the acquired and decompressed image data.

21. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

besides the first image, a second image representing the virtual world viewed from a second virtual camera is further displayed on another display apparatus connected to the information processing apparatus.

22. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, the information processing program further causing the computer to:

generate compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus, and, besides the compression image data, image data indicating the second image is output to said another display apparatus without being compressed, and

the portable display apparatus configured to;

acquire the compression image data output from the information processing apparatus; and

decompress the compression image data to obtain the image data indicating the first image, wherein

a display screen of the portable display apparatus displays the first image indicated by the acquired and decompressed image data.

23. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 21, the information processing program further causing the computer to:

set the second virtual camera at a position different from the position of the first virtual camera, and control an attitude and/or a position of the second virtual camera.

24. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 23, wherein

a direction of a line of sight of the second virtual camera is set to be parallel to a direction of a line of sight of the first virtual camera, and the second virtual camera is set behind the first virtual camera, and

a range wider than a range of the virtual world represented by the first image is displayed as the second image on said another display apparatus.

25. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 23, wherein

a direction of a line of sight of the first virtual camera is set to a vertical direction in the virtual world, and

a direction of a line of sight of the second virtual camera is set to the vertical direction in the virtual world.

26. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the first virtual camera is configured to move in a horizontal direction along a first axis relative to a horizontal direction of input on the touch panel, and

the first virtual camera is configured to move in a vertical direction along a second axis relative to a vertical direction of input on the touch panel.

27. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein the information processing apparatus is separately housed from the portable display apparatus.

28. An information processing apparatus comprising at least one processor and capable of displaying an image on a portable display apparatus that outputs at least body state data based on an attitude and/or a motion of the portable display apparatus body and touch position data based on a touch position on a touch panel provided in the portable display apparatus, the information processing apparatus configured to:

calculate the attitude and/or the motion of the portable display apparatus on the basis of the body state data;

independently control an attitude of a first virtual camera for generating an image of a virtual world on the basis of the attitude and/or the motion of the portable display apparatus such that the first virtual camera rotates relative to the virtual world;

independently control a position of the first virtual camera for generating the image of the virtual world to move in a direction relative to a direction of input on the touch panel, the position of the first virtual camera being controlled on the basis of the touch position indicated by the touch position data such that the first virtual camera moves relative to the virtual world; and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

29. An information processing system comprising a plurality of apparatuses configured to communicate with each other, the information processing system capable of displaying an image on a portable display apparatus that outputs at least body state data based on an attitude and/or a motion of the portable display apparatus body and touch position data based on a touch position on a touch panel provided in the portable display apparatus, the information processing system configured to:

calculate the attitude and/or the motion of the portable display apparatus on the basis of the body state data;

independently control an attitude of a first virtual camera for generating an image of a virtual world on the basis of the attitude and/or the motion of the portable display apparatus such that the first virtual camera rotates relative to the virtual world;

independently control a position of the first virtual camera for generating the image of the virtual world to move in a direction relative to a direction of input on the touch panel, the position of the first virtual camera being controlled on the basis of the touch position indicated by the touch position data such that the first virtual camera moves relative to the virtual world; and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.

30. An image processing method performed by a processor or a cooperation of a plurality of processors included in an image processing system including at least one information processing apparatus capable of displaying an image on a portable display apparatus that outputs at least body state data based on an attitude and/or a motion of the portable display apparatus body and touch position data based on a touch position on a touch panel provided in the portable display apparatus, the information processing method comprising:

calculating the attitude and/or the motion of the portable display apparatus on the basis of the body state data;

independently controlling an attitude of a first virtual camera for generating an image of a virtual world on the basis of the attitude and/or the motion of the portable display apparatus such that the first virtual camera rotates relative to the virtual world;

independently controlling a position of the first virtual camera for generating the image of the virtual world to move in a direction relative to a direction of input on the touch panel, the position of the first virtual camera being controlled on the basis of the touch position indicated by the touch position data such that the first virtual camera moves relative to the virtual world; and

displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera.