Computer-readable storage medium, information processing system, and information processing method

• US 9,561,443 B2
• Filed: 10/27/2011
• Issued: 02/07/2017
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Matsunaga, Hiroshi, Sumaki, Kazuya
• Primary Examiner(s): Zhou, Hong
• Application Number:
  US13/283,032
• Publication Number:
  US 20120229382A1
• Time in Patent Office: 1,930 Days
• Field of Search: 345156-173, 345/659, 345/419, 463/30, 463/31, 463/32, 463/33, 463/34, 463/36, 463/37, 463/38, 463/39
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/323 between game devices with different hardware characteristics, e.g. hand-held game devices connectable to game consoles or arcade machines
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/307 for displaying an additional window with a view from the top of the game field, e.g. radar screen
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6684 : by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball

Abstract

A movement direction of an object arranged in a virtual world is set based on attitude data. Further, the object is moved in the movement direction in the virtual world in accordance with data based on a load applied to a load detection device. An image showing the virtual world including at least the object or an image showing the virtual world viewed from the object is displayed as a first image on a portable display device.

Claims

1. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a portable display device which outputs at least attitude data based on an attitude of a body of the portable display device, wherein the information processing program causes the computer to:

obtain load data based on a load applied to a load detection device;

calculate an attitude of the portable display device based on the attitude data;

set a movement direction of an object arranged in a virtual world based on the calculated attitude of the portable display device;

calculate a center-of-gravity position of the load applied to the load detection device based on the obtained load data;

cause the object to move in the movement direction by a first action based on a change in a total load applied to the load detection device when the total load applied to the load detection device is equal to or greater than a predetermined value;

cause the object to move in the movement direction by a second action based on the calculated center-of-gravity position when the total load applied to the load detection device is less than the predetermined value; and

display, on the portable display device, as a first image, an image showing the virtual world including at least the object or an image showing the virtual world viewed from the object.

2. The non-transitory computer-readable storage medium according to claim 1, wherein

an attitude of the portable display device is calculated with respect to a predetermined direction in a real space, and

based on the attitude of the portable display device with respect to the predetermined direction in the real space, a movement direction of the object with respect to a direction which is set in the virtual world and corresponds to the predetermined direction is set.

3. The non-transitory computer-readable storage medium according to claim 2, wherein

with a direction of gravity in the real space being the predetermined direction, an attitude of the portable display device with respect to the direction of gravity is calculated, and

based on the attitude of the portable display device with respect to the direction of gravity in the real space, a movement direction of the object is set with respect to a direction of gravity set in the virtual world.

4. The non-transitory computer-readable storage medium according to claim 3, wherein

at least an attitude in which the portable display device rotates around the direction of gravity in the real space is calculated, and

the movement direction is caused to rotate around the direction of gravity set in the virtual world, based on the attitude in which the portable display device rotates around the direction of gravity in the real space, thereby setting the movement direction.

5. The non-transitory computer-readable storage medium according to claim 3, wherein

at least an attitude in which the portable display device swings up and down around a horizontal direction perpendicular to the direction of gravity in the real space is calculated, and

the movement direction is caused to swing up and down around a horizontal direction which is set in the virtual world and corresponds to the horizontal direction in the real space, based on the attitude in which the portable display device swings up and down around the horizontal direction in the real space, thereby setting the movement direction.

6. The non-transitory computer-readable storage medium according to claim 1, wherein

at least an attitude in which the portable display device rotates around two axes orthogonal to a depth direction of a display screen of the portable display device is calculated, the depth direction being perpendicular to the display screen on which the first image is displayed, and

the movement direction is caused to rotate around two axes orthogonal to the movement direction, based on the attitude in which the portable display device rotates around the two axes, thereby setting the movement direction.

7. The non-transitory computer-readable storage medium according to claim 6, wherein

at least an attitude in which the portable display device rotates around an axis along a horizontal direction of a display screen of the portable display device and an axis along a vertical direction of the display screen is calculated, the axes being orthogonal to the depth direction, and

the movement direction is caused to rotate around an axis of the horizontal direction in the virtual world, which is orthogonal to the movement direction, in accordance with the attitude in which the portable display device rotates around the axis along the horizontal direction of the display screen, and the movement direction is caused to rotate around an axis of the vertical direction in the virtual world, in accordance with the attitude in which the portable display device rotates around the axis along the vertical direction of the display screen, thereby setting the movement direction.

8. The non-transitory computer-readable storage medium according to claim 6, wherein

at least an orientation of the depth direction with respect to the direction of gravity in the real space is calculated, and

the movement direction is set such that the positional relation between the direction of gravity set in the virtual world and the movement direction is substantially equal to the positional relation between the direction of gravity in the real space and the depth direction.

9. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to:

set a reference direction of the portable display device in the real space;

calculate an angular difference, around a predetermined axis in the real space, between the direction of the portable display device at a current time and the reference direction, based on data outputted from the portable display device;

set a reference direction of the object in the virtual world; and

cause the movement direction of the object to rotate from the reference direction of the object around a predetermined axis in the virtual world, the predetermined axis corresponding to the predetermined axis in the real space, based on the calculated angular difference, thereby setting the movement direction of the object.

10. The non-transitory computer-readable storage medium according to claim 9, wherein the information processing program further causes the computer to:

rotationally displace the set reference direction, around the predetermined axis in the virtual world, by an amount of displacement according to the calculated angular difference, and

cause the movement direction of the object to rotate from the displaced reference direction, around the predetermined axis in the virtual world, based on the calculated angular difference, thereby setting the movement direction of the object.

11. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to:

control a first virtual camera for generating an image of the virtual world such that the object in the virtual world, which is viewed from behind, is included in the first image, and

display, on the portable display device, as the first image, an image showing the virtual world viewed from the first virtual camera.

12. The non-transitory computer-readable storage medium according to claim 11, wherein

the attitude of the first virtual camera is controlled such that a line-of-sight direction of the first virtual camera is equal to the movement direction.

13. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to:

arrange a first virtual camera for generating an image of the virtual world, at a subjective view point in which the object views the virtual world, and

display, on the portable display device, as the first image, an image showing the virtual world viewed from the first virtual camera.

14. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to:

display, separately from the first image, a second image showing the virtual world on another display device connected to the information processing apparatus.

15. The non-transitory computer-readable storage medium according to claim 14, wherein the information processing program further causes the computer to:

display, on the other display device, as the second image, an image of the virtual world including the object, in which the virtual world is viewed from a viewpoint different from the viewpoint of the virtual world of the first image.

16. The non-transitory computer-readable storage medium according to claim 15, wherein

the viewpoint of the virtual world of the second image is set at a position apart from the object by a distance longer than the distance between the object and the viewpoint of the virtual world of the first image, and

a range of the virtual world which is larger than the range of the virtual world shown in the first image is displayed on the other display device as the second image.

17. The non-transitory computer-readable storage medium according to claim 15, wherein the information processing program further causes the computer to:

set a viewpoint for generating the second image, at a position in the virtual world, from which the object is viewed in a manner of bird'"'"'s eye viewing, and

display, on the other display device, as the second image, an image in which the object arranged in the virtual world is viewed in a manner of bird'"'"'s eye viewing.

18. The non-transitory computer-readable storage medium according to claim 14, wherein

a first virtual camera for generating the first image and a second virtual camera for generating the second image are set, and a line-of-sight direction of the second virtual camera in the virtual world is caused to change in accordance with a change in a line-of-sight direction of the first virtual camera in the virtual world.

19. The non-transitory computer-readable storage medium according to claim 18, wherein

the first virtual camera is arranged such that a forward direction of the object from behind the object in the virtual world is an imaging direction of the first virtual camera, and the second virtual camera is arranged such that a position in the forward direction of the object in the virtual world is a point of view of the second virtual camera, and the object is displayed at the same position in the second image, and the positions and/or the attitudes of the first virtual camera and the second virtual camera are controlled in accordance with an action of the object.

20. The non-transitory computer-readable storage medium according claim 1, wherein

the object is caused to move in the movement direction, based on reciprocating motion of the calculated center-of-gravity position.

21. The non-transitory computer-readable storage medium according to claim 20, wherein

the object is caused to move in the movement direction at a movement speed according to a first action performed by the object when the center-of-gravity position, in its reciprocating motion, shifts from one side of a predetermined boundary to the other side, and the object is caused to move in the movement direction at a movement speed according to a second action performed by the object when the center-of-gravity position shifts from the other side of the predetermined boundary to the one side.

22. The non-transitory computer-readable storage medium according to claim 1, wherein

a movement speed of the object is set, based on a distance from a preset reference position to the calculated center-of-gravity position, and a forward movement or a backward movement of the object is set based on the direction from the reference position toward the center-of-gravity position, and then the object is caused to move in the movement direction.

23. The non-transitory computer-readable storage medium according to claim 1, wherein

the object is caused to move in the movement direction at a movement speed according to a first operation performed by the object, at a time point when the load applied to the load detection device increases to be equal to or higher than a predetermined value.

24. The non-transitory computer-readable storage medium according to claim 1, wherein

the object is caused to move in the movement direction when the load applied to the load detection device, which has once increased to be equal to or higher than a predetermined value, decreases to be equal to or lower than a predetermined value.

25. The non-transitory computer-readable storage medium according to claim 24, wherein

the object is caused to move in the movement direction at a movement speed based on a length of time from when the load increases to be equal to or higher than the predetermined value to when the load decreases to be equal to or lower than the predetermined value.

26. The non-transitory computer-readable storage medium according to claim 1, wherein

the portable display device includes at least one of a gyro sensor and an acceleration sensor, and

an attitude of the portable display device is calculated based on data outputted from at least one of the gyro sensor and the acceleration sensor.

27. The non-transitory computer-readable storage medium according to claim 1, wherein

image data representing the first image is output to the portable display device,

image data outputted from the information processing apparatus is obtained, and

the first image represented by the image data obtained by the image data obtaining section is displayed.

28. The non-transitory computer-readable storage medium according to claim 27, wherein the information processing program further causes the computer to:

generate compressed image data by compressing the image data representing the first image, and

output the generated compressed image data to the portable display device,

the portable display device configured to;

obtain the compressed image data outputted from the information processing apparatus,

decompress the compressed image data to obtain image data representing the first display image, and

display the first image represented by the obtained and decompressed image data.

29. The non-transitory computer-readable storage medium according to claim 1, wherein the attitude of the body of the portable display device is calculated based on a direction in which a back surface of the portable display device faces in real space.

30. The non-transitory computer-readable storage medium according to claim 1, wherein the attitude of the body of the portable display device is calculated based on a direction in which a back surface of the portable display device faces in real space at a specified point in time.

31. The non-transitory computer-readable storage medium according to claim 1, wherein when the total load applied to the load detection device is less than the predetermined value, the second action or a third action is determined based on the calculated center-of-gravity, and the object is caused to move in the movement direction based on the determined action.

32. The non-transitory computer-readable storage medium according to claim 1, wherein the first action corresponds to a dolphin kick action.

33. The non-transitory computer-readable storage medium according to claim 1, wherein the second action corresponds to a flutter kick action.

34. An information processing system including a portable display device, a load detection device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the load detection device is configured to output, to the information processing apparatus, load data based on a load applied to the load detection device,

the portable display device is configured to output, to the information processing apparatus, at least attitude data according to an attitude of the portable display device,

the information processing apparatus is configured to:

obtain the load data outputted from the load detection device;

calculate an attitude of the portable display device based on the attitude data;

set a movement direction of an object arranged in a virtual world based on the calculated attitude of the portable display device;

calculate a center-of-gravity position of the load applied to the load detection device based on the obtained load data;

cause the object to move in the movement direction by a first action based on a change in a total load applied to the load detection device when the total load applied to the load detection device is equal to or greater than a predetermined value;

cause the object to move in the movement direction by a second action based on the calculated center-of-gravity position when the total load applied to the load detection device is less than the predetermined value; and

output, to the portable display device, as a first image, an image showing the virtual world including at least the object or an image showing the virtual world viewed from the object, and

the portable display device is configured to;

obtain data of the first image outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

35. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining load data based on a load applied to a load detection device;

calculating an attitude of the portable display device based on the attitude data;

setting a movement direction of an object arranged in a virtual world based on the calculated attitude of the portable display device;

calculating a center-of-gravity position of the load applied to the load detection device based on the obtained load data;

causing the object to move in the movement direction by a first action based on a change in a total load applied to the load detection device when the total load applied to the load detection device is equal to or greater than a predetermined value;

causing the object to move in the movement direction by a second action based on the calculated center-of-gravity position when the total load applied to the load detection device is less than the predetermined value; and

displaying, on the portable display device, as a first image, an image showing the virtual world including at least the object or an image showing the virtual world viewed from the object.

36. An information processing system including a portable display device and a load detection device, wherein

the load detection device is configured to output, to the portable display device, load data based on a load applied to the load detection device, and

the portable display device is configured to:

generate attitude data in accordance with an attitude of the portable display device;

obtain the load data outputted from the load detection device;

calculate the attitude of the portable display device based on the attitude data;

set a movement direction of an object arranged in the virtual world based on the calculated attitude of the portable display device;

calculate a center-of-gravity position of the load applied to the load detection device based on the obtained load data;

cause the object to move in the movement direction by a first action based on a change in a total load applied to the load detection device when the total load applied to the load detection device is equal to or greater than a predetermined value;

cause the object to move in the movement direction by a second action based on the calculated center-of-gravity position when the total load applied to the load detection device is less than the predetermined value;

generate, as a first image, an image showing the virtual world including at least the object or an image showing the virtual world viewed from the object; and

display the generated first image.

37. An information processing system, comprising:

a processing system having at least one processor, the at least one processor controlling the processing system to;

obtain load data based on a load applied to a load detection device from a user standing on the load detection device, the load detection device operatively coupled to the information processing system;

calculate a center-of-gravity position of the user based on the obtained load data;

obtain attitude data output from a portable display device, the attitude data corresponding to an attitude of a body of the portable display device;

calculate a direction and the attitude of the body of the portable display device based on the obtained attitude data;

set a movement direction of an object in a virtual world based on the calculated direction and the calculated attitude of the body of the portable display device;

cause the object to move in the movement direction by a first action based on a change in a total load applied to the load detection device when the total load applied to the load detection device is equal to or greater than a predetermined value;

cause the object to move in the movement direction by a second action based on the calculated center-of-gravity position when the total load applied to the load detection device is less than the predetermined value;

render a first image from a first virtual camera showing the virtual world including the object that moves in the movement direction;

render a second image from a second virtual camera showing the virtual world and having a perspective different from the first virtual camera; and

output the first image to the portable display device and output the second image to a display device different from the portable display device.

38. The information processing system according to claim 37, wherein the first and second image are rendered using the processing system of the information processing system and transmitted to the portable display device and the display device, respectively, using wireless communication.