Storage medium having stored thereon information processing program, information processing apparatus, information processing system, and information processing method

• US 9,522,323 B2
• Filed: 01/31/2012
• Issued: 12/20/2016
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Hayashi, Yugo, Sumaki, Kazuya
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): Guo, Xilin
• Application Number:
  US13/362,289
• Publication Number:
  US 20120229513A1
• Time in Patent Office: 1,785 Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/20 Input arrangements for video game devices
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/816 Athletics, e.g. track-and-field sports
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/8005 Athletics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 : Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 11/60 : Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/20 : Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: G06T 3/60 : Rotation of whole images or parts thereof

Abstract

On the basis of data output from a portable display apparatus, it is determined whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range. When it has been determined that the direction of the predetermined axis is included in the predetermined range, an action of an object placed in a virtual world is controlled in accordance with the direction of the predetermined axis. Further, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera for generating an image of the virtual world is controlled in accordance with the direction of the predetermined axis. Then, a first image representing the virtual world viewed from the first virtual camera is displayed on the portable display apparatus.

Claims

1. A non-transitory computer-readable storage medium having stored thereon an information processing program to be executed by a computer of an information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing program causing the computer to perform functionality comprising:

determining, on the basis of the attitude and/or motion data output from the portable display apparatus, whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range;

controlling, when the direction of the predetermined axis is included in the predetermined range, an action of an object placed in a virtual world in accordance with the direction of the predetermined axis;

controlling, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera, for generating an image of the virtual world, in accordance with the direction of the predetermined axis; and

displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera,

wherein a direction of the object corresponds to the attitude and/or motion of the portable display device until the predetermined axis is outside the predetermined range, where a direction of the virtual camera continues to correspond to the attitude and/or motion of the portable display device even when the predetermined axis is outside the predetermined range.

2. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the determination of whether or not the direction of the predetermined axis is included in the predetermined range includes

determining, on the basis of the data output from the portable display apparatus, whether or not the direction of the predetermined axis in a left-right direction in real space is included in a first range set in the left-right direction, and

when it has been determined that the direction of the predetermined axis in the left-right direction in real space is included in the first range, the action of the object is controlled such that on the basis of the direction of the predetermined axis in the left-right direction in real space, the object rotates and/or moves in a left-right direction in the virtual world.

3. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 2, wherein

the determination of whether or not the direction of the predetermined axis is included in the predetermined range includes

determining, on the basis of the data output from the portable display apparatus, whether or not the direction of the predetermined axis in an up-down direction in real space is included in a second range set in the up-down direction, and

when it has been determined that the direction of the predetermined axis in the up-down direction in real space is included in the second range, the action of the object is controlled such that on the basis of the direction of the predetermined axis in the up-down direction in real space, the object rotates and/or moves in an up-down direction in the virtual world.

4. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

the first range has an angular range of a predetermined size, and

the second range has an angular range of a size different from the size of the angular range of the first range.

5. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 3, wherein

the first range is a range having the same angular ranges in a left-right direction about a reference direction in real space, and

the second range is a range having angular ranges different from each other in an up-down direction with respect to a horizontal direction in real space.

6. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

when it has been determined that the direction of the predetermined axis is included in the predetermined range, an attitude of the first virtual camera is controlled such that a direction of a line of sight of the first virtual camera is the same as a direction of the object in the virtual world.

7. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the action of the object is controlled on the basis of a change in the direction of the predetermined axis, and

the action of the first virtual camera is controlled on the basis of the change in the direction of the predetermined axis.

8. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 7, wherein

the action of the object is controlled such that the greater the change in the direction of the predetermined axis, the greater the object moves, and

the action of the first virtual camera is controlled such that the greater the change in the direction of the predetermined axis, the greater the first virtual camera moves.

9. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the action of the object is controlled such that the object rotationally moves in accordance with an angle of the predetermined axis, and

the action of the first virtual camera is controlled such that the first virtual camera rotationally moves in accordance with the angle of the predetermined axis.

10. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, the information processing program further causing the computer to perform functionality comprising:

calculating an attitude and/or a motion of the portable display apparatus on the basis of the data output from the portable display apparatus, wherein

on the basis of the attitude and/or the motion of the portable display apparatus, it is determined whether or not the direction of the predetermined axis is included in the predetermined range.

11. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 10, wherein

the attitude and/or the motion of the portable display apparatus are calculated with respect to a predetermined direction in real space,

on the basis of the attitude and/or the motion of the portable display apparatus with respect to the predetermined direction in real space, the action of the object is controlled with respect to a direction that corresponds to the predetermined direction and is set in the virtual world, and

on the basis of the attitude and/or the motion of the portable display apparatus with respect to the predetermined direction in real space, the action of the first virtual camera is controlled with respect to the direction that corresponds to the predetermined direction and is set in the virtual world.

12. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 11, wherein

the attitude and/or the motion of the portable display apparatus are calculated with respect to a direction of gravity in real space, using the direction of gravity as the predetermined direction,

on the basis of the attitude and/or the motion of the portable display apparatus with respect to the direction of gravity in real space, the action of the object is controlled with respect to a direction of gravity set in the virtual world, and

on the basis of the attitude and/or the motion of the portable display apparatus with respect to the direction of gravity in real space, the action of the first virtual camera is controlled with respect to the direction of gravity set in the virtual world.

13. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, wherein

at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space are calculated,

at least a range about the direction of gravity in real space is set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, it is determined whether or not the direction of the predetermined axis is included in the predetermined range,

on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, the action of the object is controlled such that the object rotates about the direction of gravity set in the virtual world, and

on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the direction of gravity in real space, the action of the first virtual camera is controlled such that the first virtual camera rotates about the direction of gravity set in the virtual world.

14. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 12, wherein

at least the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about a horizontal direction perpendicular to the direction of gravity in real space are calculated,

at least a range about the horizontal direction in real space is set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, it is determined whether or not the direction of the predetermined axis is included in the predetermined range,

on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, the action of the object is controlled such that the object swings upward and downward about a horizontal direction that corresponds to the horizontal direction in real space and is set in the virtual world, and

on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by swinging the portable display apparatus upward and downward about the horizontal direction in real space, the action of the first virtual camera is controlled such that the first virtual camera swings upward and downward about the horizontal direction that corresponds to the horizontal direction in real space and is set in the virtual world.

15. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 10, wherein

at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of a first axis and a second axis that are orthogonal to a perspective direction of, and perpendicular to, a display screen of the portable display apparatus are calculated, the first image displayed on the display screen,

each of a range about the first axis and a range about the second axis is set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the first axis and the second axis, it is determined whether or not the direction of the predetermined axis is included in each predetermined range,

when it has been determined that the direction of the predetermined axis is included in the predetermined range about the first axis, the action of the object is controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the first axis, the object rotates about an axis that corresponds to the first axis and is set in the virtual world,

when it has been determined that the direction of the predetermined axis is included in the predetermined range about the second axis, the action of the object is controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the second axis, the object rotates about an axis that corresponds to the second axis and is set in the virtual world, and

the action of the first virtual camera is controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the first axis and the second axis, the first virtual camera rotates about each of axes that correspond to the first axis and the second axis and are orthogonal to a direction of a line of sight of the first virtual camera.

16. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 15, wherein

at least the attitude and/or the motion of the portable display apparatus that are obtained by rotating the display apparatus about each of an axis along a width direction of the display screen and an axis along a height direction of the display screen are calculated, each axis being orthogonal to the perspective direction,

each of a range about the axis along the width direction and a range about the axis along the height direction is set as the predetermined range, and on the basis of the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about each of the axis along the width direction and the axis along the height direction, it is determined whether or not the direction of the predetermined axis is included in each predetermined range,

when it has been determined that the direction of the predetermined axis is included in the predetermined range about the axis along the width direction, the action of the object is controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the width direction, the object rotates about a horizontal axis that corresponds to the axis along the width direction and is set in the virtual world,

when it has been determined that the direction of the predetermined axis is included in the predetermined range about the axis along the height direction, the action of the object is controlled such that in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by the portable display apparatus about the axis along the height direction, the object rotates about a vertical axis that corresponds to the axis along the height direction and is set in the virtual world, and

the action of the first virtual camera is controlled such that:

in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the width direction, the first virtual camera rotates about a horizontal axis that is orthogonal to a direction of a line of sight of the first virtual camera and is included in the virtual world; and

in accordance with the attitude and/or the motion of the portable display apparatus that are obtained by rotating the portable display apparatus about the axis along the height direction, the first virtual camera rotates about a vertical axis in the virtual world.

17. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 10, wherein

the calculation of the attitude and/or the motion of the portable display apparatus includes:

setting a reference direction of the portable display apparatus in real space; and

calculating, on the basis of the data output from the portable display apparatus, a difference in left-right angle about a vertical direction in real space between a current direction of the predetermined axis and the reference direction, and a difference in up-down angle about a horizontal direction in real space between the current direction of the predetermined axis and the horizontal direction,

the determination of whether or not the direction of the predetermined axis is included in the predetermined range includes;

determining whether or not the difference in left-right angle is included in a first range set about the vertical direction in real space; and

determining whether or not the difference in up-down angle is included in a second range set about the horizontal direction in real space,

when it has been determined that the difference in left-right angle is included in the first range, the action of the object is controlled such that the object rotates and/or moves in a left-right direction in the virtual world on the basis of the difference in left-right angle,

when it has been determined that the difference in up-down angle is included in the second range, the action of the object is controlled such that the object rotates and/or moves in an up-down direction in the virtual world on the basis of the difference in up-down angle, and

the action of the first virtual camera is controlled such that;

the first virtual camera rotates and/or moves in a left-right direction in the virtual world on the basis of the difference in left-right angle; and

the first virtual camera rotates and/or moves, on the basis of the difference in up-down angle, in an up-down direction and about a left-right direction that is orthogonal to a direction of a line of sight of the first virtual camera and is included in the virtual world.

18. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein image data indicating the first image is output to the portable display apparatus, and

the portable display apparatus configured to:

acquire the image data output from the information processing apparatus, and

display the first image indicated by the acquired image data.

19. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 18, the information processing program further causing the computer to perform functionality comprising:

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus,

the portable display apparatus is configured to;

acquire the compression image data output from the information processing apparatus,

decompress the compression image data to obtain the image data indicating the first image, and

display the first image indicated by the image data that has been acquired and has been decompressed.

20. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

besides the first image, a second image representing the virtual world viewed from a second virtual camera is further displayed on another display apparatus connected to the information processing apparatus.

21. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 20, the information processing program further causing the computer to perform functionality comprising:

generating compression image data by compressing the image data indicating the first image, wherein

the generated compression image data is output to the portable display apparatus, and, besides the compression image data, image data indicating the second image is output to said another display apparatus without being compressed, and

the portable display apparatus configured to;

acquire the compression image data output from the information processing apparatus;

decompress the compression image data to obtain the image data indicating the first image; and

display the first image indicated by the image data that has been acquired and has been decompressed.

22. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 20, the information processing program further causing the computer to perform functionality comprising:

setting, on the basis of a position of the object in the virtual world, the second virtual camera, for generating an image of the virtual world, at a position different from a position of the first virtual camera such that the object is included in the second image.

23. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 22, wherein

the second virtual camera is set at a position further away from the object than the first virtual camera is from the object, and

a range wider than a range of the virtual world represented by the first image is displayed as the second image on said another display apparatus.

24. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 22, wherein

the second virtual camera is set at a position of viewing the object from a bird'"'"'s-eye view in the virtual world, and

an image obtained by viewing from a bird'"'"'s-eye view the object placed in the virtual world is displayed as the second image on said another display apparatus.

25. The non-transitory computer-readable storage medium having stored thereon the information processing program according to claim 1, wherein

the portable display apparatus includes at least one of a gyro sensor and an acceleration sensor, and

on the basis of data output from the at least one of the gyro sensor and the acceleration sensor, it is determined whether or not the direction of the predetermined axis is included in the predetermined range.

26. An information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing apparatus comprising at least one processor, the apparatus configured to:

determine, on the basis of the attitude and/or motion data output from the portable display apparatus, whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range;

control, when the direction of the predetermined axis is included in the predetermined range, an action of an object placed in a virtual world in accordance with the direction of the predetermined axis;

control, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera, for generating an image of the virtual world, in accordance with the direction of the predetermined axis; and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera,

wherein a direction of the object corresponds to the attitude and/or motion of the portable display device until the predetermined axis is outside the predetermined range, where a direction of the virtual camera continues to correspond to the attitude and/or motion of the portable display device even when the predetermined axis is outside the predetermined range.

27. An information processing system including a plurality of apparatuses configured to communicate with each other, the information processing system configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing system comprising:

a processing system having at least one processor, the processing system configured to;

determine, on the basis of the attitude and/or motion data output from the portable display apparatus, whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range,

control, when the direction of the predetermined axis is included in the predetermined range, an action of an object placed in a virtual world in accordance with the direction of the predetermined axis,

control, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera, for generating an image of the virtual world, in accordance with the direction of the predetermined axis, and

display on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera,

wherein a direction of the object corresponds to the attitude and/or motion of the portable display device until the predetermined axis is outside the predetermined range, where a direction of the virtual camera continues to correspond to the attitude and/or motion of the portable display device even when the predetermined axis is outside the predetermined range.

28. An information processing method performed by a processor or a corporation of a plurality of processors included in an information processing system including at least one information processing apparatus configured to display an image on a portable display apparatus that outputs at least data based on an attitude and/or a motion of the portable display apparatus body, the information processing method comprising:

determining, on the basis of the attitude and/or motion data output from the portable display apparatus, whether or not a direction of a predetermined axis set in the portable display apparatus is included in a predetermined range;

controlling, when the direction of the predetermined axis is included in the predetermined range, an action of an object placed in a virtual world in accordance with the direction of the predetermined axis;

controlling, when the direction of the predetermined axis is included in the predetermined range or present outside the predetermined range, an action of a first virtual camera, for generating an image of the virtual world, in accordance with the direction of the predetermined axis; and

displaying on the portable display apparatus a first image representing the virtual world viewed from the first virtual camera,

wherein a direction of the object corresponds to the attitude and/or motion of the portable display device until the predetermined axis is outside the predetermined range, where a direction of the virtual camera continues to correspond to the attitude and/or motion of the portable display device even when the predetermined axis is outside the predetermined range.