Computer-readable storage medium, information processing system, and information processing method for operating objects in a virtual world based on orientation data related to an orientation of a device

• US 9,539,511 B2
• Filed: 10/27/2011
• Issued: 01/10/2017
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Matsunaga, Hiroshi, Minato, Keiichi
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US13/283,072
• Publication Number:
  US 20120229448A1
• Time in Patent Office: 1,902 Days
• Field of Search: 345/655, 345/659
• US Class Current: 1/1
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/323 between game devices with different hardware characteristics, e.g. hand-held game devices connectable to game consoles or arcade machines
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1056 involving pressure sensitive buttons
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball

Abstract

It is determined, based on attitude data, whether or not a direction of a predetermined axis set in a portable display device is within a first range. When the result of the determination is that the direction of the predetermined axis is within the first range, an action of a first object arranged in a virtual world is controlled based on the direction of the predetermined axis. When it is determined that the direction of the predetermined axis is outside the first range, at least an action of a second object arranged in the virtual world is controlled based on the direction of the predetermined axis. Then, a first image showing the virtual world including at least the first object and the second object is displayed on the portable display device.

Claims

1. A non-transitory computer-readable storage medium having stored therein an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a portable display device which outputs at least attitude data based on an attitude of a body of the portable display device, wherein the information processing program causes the computer to perform functionality comprising:

calculating an attitude of the portable display device with respect to a predetermined direction in real space;

determining, based on the calculated attitude of the portable display device, whether or not a direction of a predetermined axis is within a first range which is set with respect to the predetermined direction in the real space, based on the attitude of the portable display device with respect to the predetermined direction in the real space;

controlling an action of a first object arranged in a virtual world, with reference to a predetermined direction which is set in the virtual world and corresponds to the predetermined direction in the real space, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is within the first range;

controlling at least an action of a second object arranged in the virtual world, with reference to the predetermined direction set in the virtual world, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is outside the first range; and

displaying, on the portable display device, a first image showing the virtual world including at least the first object and the second object.

2. The non-transitory computer-readable storage medium according to claim 1, wherein

the action of the first object is controlled based on a change in the direction of the predetermined axis, and

at least the action of the second object is controlled based on the change in the direction of the predetermined axis.

3. The non-transitory computer-readable storage medium according to claim 2, wherein

the action of the first object is controlled such that the greater the change in the direction of the predetermined axis is, the greater the movement of the first object is, and

the action of the second object is controlled such that the greater the change in the direction of the predetermined axis is, the greater the movement of the second object is.

4. The non-transitory computer-readable storage medium according to claim 1, wherein

the action of the first object is controlled by causing the first object to rotate in accordance with an angle of the predetermined axis, and

the action of the second object is controlled by causing the second object to rotate in accordance with the angle of the predetermined axis.

5. The non-transitory computer-readable storage medium according to claim 1, wherein

the action of the second object is controlled in accordance with the direction of the predetermined axis, and the action of the first object is controlled in accordance with the action of the second object.

6. The non-transitory computer-readable storage medium according to claim 5, wherein

the action of the second object is controlled in accordance with the direction of the predetermined axis, and the action of the first object is controlled such that the first object performs the same action as the second object.

7. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising:

setting, as the first range, a range within a first angle, and setting, as a second range, a range which is within a second angle larger than the first angle and is outside the first range, and determining whether or not the direction of the predetermined axis is within the second range,

controlling the action of the second object by causing the second object to rotate at a first angular velocity in accordance with the direction of the predetermined axis, when the result of the determination is that the direction of the predetermined axis is within the second range, and

controlling the action of the second object by causing the second object to rotate at a second angular velocity which is lower than the first angular velocity, in accordance with the direction of the predetermined axis, when the result of the determination is that the direction of the predetermined axis is neither in the first range nor in the second range.

8. The non-transitory computer-readable storage medium according to claim 7, wherein

a reference direction is set with respect to the direction of the predetermined axis, and a range within the first angle centering around the reference direction is set as the first range, and a range which is within the second angle centering around the reference direction and is outside the first range is set as the second range.

9. The non-transitory computer-readable storage medium according to claim 1, wherein

a reference direction is set with respect to the direction of the predetermined axis, and a range within the first angle centering around the reference direction is set as the first range.

10. The non-transitory computer-readable storage medium according to claim 9, wherein

the reference direction is set on a predetermined horizontal plane, and a range within the first angle on the horizontal plane centering around the reference direction is set as the first range.

11. The non-transitory computer-readable storage medium according to claim 1, wherein

a second image showing the virtual world on another display device connected to the information processing apparatus is displayed separately from the first image.

12. The non-transitory computer-readable storage medium according to claim 11, wherein

an image of the virtual world including the first object and the second object is displayed on the other display device as the second image, in which the virtual world is viewed from a viewpoint different from the viewpoint of the virtual world of the first image.

13. The non-transitory computer-readable storage medium according to claim 12, wherein

a viewpoint of the virtual world of the second image is set in a position apart from the first object and the second object by a distance longer than the distance from the viewpoint of the virtual world of the first image to the first object and the second object, and

a range of the virtual world is displayed on the other display device as the second image, which is larger than the range of the virtual world shown by the first image.

14. The non-transitory computer-readable storage medium according to claim 12, wherein

a viewpoint for generating the second image is set in a position in the virtual world from which the first object and the second object are viewed in a manner of bird'"'"'s eye viewing, and

an image in which the first object and the second object arranged in the virtual world are viewed in a manner of bird'"'"'s eye viewing is displayed on the other display device as the second image.

15. The non-transitory computer-readable storage medium according to claim 11, wherein

a first virtual camera for generating the first image and a second virtual camera for generating the second image are set, and a line-of-sight direction of the second virtual camera in the virtual world is caused to change in accordance with a change in a line-of-sight direction of the first virtual camera in the virtual world.

16. The non-transitory computer-readable storage medium according to claim 15, wherein

the first virtual camera is arranged such that a forward direction of the second object from behind the second object in the virtual world is an imaging direction of the first virtual camera, the second virtual camera is arranged such that a position in the forward direction of the second object in the virtual world is a point of view of the second virtual camera, and the second object is displayed in the second image at a constant position with respect to the point of view, and the positions and/or the attitudes of the first virtual camera and the second virtual camera are controlled in accordance with the action of the second object.

17. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising:

calculating, with a direction of gravity in the real space being the predetermined direction, an attitude of the portable display device with respect to the direction of gravity,

setting, with the direction of gravity in the real space being the predetermined direction in the real space, the first range with respect to the direction of gravity, and determining whether or not the direction of the predetermined axis is within the first range, based on the attitude of the portable display device with respect to the direction of gravity in the real space,

when the result of the determination is that the direction of the predetermined axis is within the first range, controlling the action of the first object with reference to the direction of gravity set in the virtual world, based on the direction of the predetermined axis with respect to the direction of gravity in the real space, and

when the result of the determination is that the direction of the predetermined axis is outside the first range, controlling the action of the second object with reference to the direction of gravity set in the virtual world, based on the direction of the predetermined axis with respect to the direction of gravity in the real space.

18. The non-transitory computer-readable storage medium according to claim 17, wherein the information processing program further causes the computer to perform functionality comprising:

calculating at least an attitude in which the portable display device rotates around the direction of gravity in the real space,

setting the first range in a horizontal direction with respect to the direction of gravity in the real space, and determining whether or not the direction of the predetermined axis which is projected onto a horizontal plane in the real space is within the first range,

when the result of the determination is that the direction of the predetermined axis projected onto the horizontal plane in the real space is within the first range, controlling the action of the first object by causing the first object to rotate and/or move along the horizontal direction set in the virtual world, based on the direction of the predetermined axis projected onto the horizontal plane, and

when the result of the determination is that the direction of the predetermined axis projected onto the horizontal plane in the real space is outside the first range, controlling the action of the second object by causing the second object to rotate and/or move along the horizontal direction set in the virtual world, based on the direction of the predetermined axis projected onto the horizontal direction.

19. The non-transitory computer-readable storage medium according to claim 18, wherein the information processing program further causes the computer to perform functionality comprising:

further calculating an attitude in which the portable display device swings up and down around a horizontal direction perpendicular to the direction of gravity in the real space, and

controlling the action of the first object by causing the first object to rotate and/or move around a horizontal direction which is set in the virtual world and corresponds to the horizontal direction in the real space, based on the attitude in which the portable display device swings up and down around the horizontal direction in the real space, regardless of whether the result of the determination is positive or negative.

20. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising:

calculating at least an attitude in which the portable display device rotates around two axes orthogonal to the predetermined axis, the predetermined axis being a depth direction of a display screen of the portable display device, the depth direction being perpendicular to the display screen on which the first image is displayed,

setting the first range around one of the two axes, and determining whether or not the depth direction rotated around the one axis is within the first range,

when the result of the determination is that the depth direction is within the first range, causing the first object to rotate and/or move around an axis of a vertical direction in the virtual world, based on the depth direction rotated around the one axis,

when the result of the determination is that the depth direction is outside the first range, causing the second object to rotate or move around the axis of the vertical direction in the virtual world, based on the depth direction which has rotated around the one axis, and

causing the first object to rotate and/or move around the horizontal direction in the virtual world, based on the depth direction rotated around the other axis, regardless of whether the result of the determination is positive or negative.

21. The non-transitory computer-readable storage medium according to claim 20, wherein the information processing program further causes the computer to perform functionality comprising:

calculating at least an attitude in which the portable display device rotates around an axis along a horizontal direction of a display screen of the portable display device and an axis along a vertical direction of the display screen, the axes being orthogonal to the depth direction,

setting the first range around the axis along the vertical direction of the display screen, and determining whether or not the depth direction rotated around the axis along the vertical direction is within the first range,

when the result of the determination is that the depth direction is within the first range, causing the first object to rotate and/or move around the axis of the vertical direction in the virtual world, based on the depth direction rotated around the axis of the vertical direction of the display screen,

when the result of the determination is that the depth direction is outside the first range, causing the second object to rotate and/or move around the axis of the vertical direction in the virtual world, based on the depth direction rotated around the axis of the vertical direction of the display screen, and

causing the first object to rotate and/or move around the horizontal direction in the virtual world, based on the depth direction rotated around the axis of the horizontal direction of the display screen, regardless of whether the result of the determination is positive or negative.

22. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to perform functionality comprising correcting the direction of the predetermined axis set in the portable display device, by a predetermined angle, in a direction along which a display screen of the portable display device, which faces upward, stands up in a vertical direction,

determining whether or not the direction of the predetermined axis, which is corrected, is within the first range,

controlling the action of the first object in accordance with the direction of the predetermined axis which is corrected, and

controlling the action of the second object in accordance with the direction of the predetermined axis which is corrected.

23. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising:

causing, as the first object, at least a target object which is a target of an attack according to an operation by a user, to move in the virtual world in accordance with the predetermined axis, when the result of the determination is that the direction of the predetermined axis is within the first range, and

setting a display range of the virtual world to be displayed on the portable display device, based on the action of the second object.

24. The non-transitory computer-readable storage medium according to claim 23, wherein the information processing program further causes the computer to perform functionality comprising:

setting the display range based on the action of the second object, by arranging a virtual camera for generating an image of the virtual world such that the first object and the second object are included in the display range, with a forward direction of the second object from behind the second object in the virtual world being a line-of-sight direction of the virtual camera, and

displaying, on the portable display device, as the first image, an image showing the virtual world viewed from the virtual camera.

25. The non-transitory computer-readable storage medium according to claim 23, wherein the information processing program further causes the computer to perform functionality comprising:

when the result of the determination is that the direction of the predetermined axis is within the first range, causing, as the first object, a shooting object which shoots a predetermined object toward the target to rotate in the virtual world in accordance with the direction of the predetermined axis, and

when the result of the determination is that the direction of the predetermined axis is outside the first range, causing the second object together with the shooting object to rotate in the virtual world in accordance with the direction of the predetermined axis, and causing the target object to move in the virtual world in accordance with the direction of the shooting object.

26. The non-transitory computer-readable storage medium according to claim 1, wherein

the portable display device includes at least one of a gyro sensor and an acceleration sensor, and

an attitude of the portable display device is calculated based on data outputted from at least one of the gyro sensor and the acceleration sensor.

27. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to perform functionality comprising:

obtaining data based on a load applied to a load detection device; and

causing the first object and the second object to move in the direction of the second object in the virtual world, based on the obtained data.

28. The non-transitory computer-readable storage medium according to claim 27, wherein

the information processing program further causes the computer to perform functionality comprising calculating a center-of-gravity position of the load applied to the load detection device, based on the obtained data, and

causing the first object and the second object to move in the direction of the second object, based on the calculated center-of-gravity position.

29. The non-transitory computer-readable storage medium according to claim 28, wherein

a movement speed of the first object and the second object is set, based on a distance from a preset reference position to the calculated center-of-gravity position, and a forward movement or a backward movement of the first object and the second object are set, based on the distance from the reference position to the center-of-gravity position, and the first object and the second object are caused to move in the direction of the second object.

30. The non-transitory computer-readable storage medium according to claim 29, wherein the information processing program further causes the computer to perform functionality comprising:

setting, as the reference position, a straight line which divides a part of the load detection device to which the load is applied, into a front part and a rear part, setting a movement speed of the first object and the second object, based on a distance from the straight line to the calculated center-of-gravity position, setting the first object and the second object to a forward movement when the center-of-gravity position is located in the front part with respect to the straight line, and setting the first object and the second object to a backward movement when the center-of-gravity position is located in the rear part with respect to the straight line.

31. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising:

outputting, to the portable display device, image data representing the first image, and

the portable display device configured to;

obtain image data outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

32. The non-transitory computer-readable storage medium according to claim 31, wherein the information processing program further causes the computer to perform functionality comprising:

generating compressed image data by compressing the image data representing the first image,

outputting the generated compressed image data to the portable display device,

the portable display device configured to;

obtain the compressed image data outputted from the information processing apparatus,

decompress the compressed image data to obtain image data representing the first display image, and

display the first image represented by the image data which has been obtained and decompressed.

33. The non-transitory computer-readable storage medium according to claim 1, wherein a range of motion of the second object arranged in the virtual world is controlled based on the attitude data, in accordance with the direction of the predetermined axis, when the result of the determination is that the direction of the predetermined axis is outside the first range.

34. An information processing system comprising a portable display device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the portable display device is configured to output, to the information processing apparatus, at least attitude data based on an attitude of the portable display device,

the information processing apparatus including a processing system having at least one processor, the information processing apparatus is configured to:

calculate an attitude of the portable display device with respect to a predetermined direction in real space;

determine, based on the calculated attitude of the portable display device, whether or not a direction of a predetermined axis is within a first range which is set with respect to the predetermined direction in the real space, based on the attitude of the portable display device with respect to the predetermined direction in the real space;

control an action of a first object arranged in a virtual world, with reference to a predetermined direction which is set in the virtual world and corresponds to the predetermined direction in the real space, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is within the first range;

control at least an action of a second object arranged in the virtual world, with reference to the predetermined direction set in the virtual world, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is outside the first range; and

output, to the portable display device, a first image showing the virtual world including at least the first object and the second object, and

the portable display device further configured to;

obtain data of the first image outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

35. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

calculating an attitude of the portable display device with respect to a predetermined direction in real space;

determining, based on the calculated attitude of the portable display device, whether or not a direction of a predetermined axis is within a first range which is set with respect to the predetermined direction in the real space, based on the attitude of the portable display device with respect to the predetermined direction in the real space;

controlling an action of a first object arranged in a virtual world, with reference to a predetermined direction which is set in the virtual world and corresponds to the predetermined direction in the real space, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is within the first range;

controlling at least an action of a second object arranged in the virtual world, with reference to the predetermined direction set in the virtual world, based on the direction of the predetermined axis with respect to the predetermined direction in the real space, when the result of the determination is that the direction of the predetermined axis is outside the first range; and

displaying, on the portable display device, a first image showing the virtual world including at least the first object and the second object.