Computer-readable storage medium, information processing system, and information processing method for displaying an image on a display device using attitude data of a display device

• US 9,925,464 B2
• Filed: 11/02/2011
• Issued: 03/27/2018
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Hayashi, Yugo, Sumaki, Kazuya
• Primary Examiner(s): Nguyen, Vu
• Application Number:
  US13/287,320
• Publication Number:
  US 20120229507A1
• Time in Patent Office: 2,337 Days
• Field of Search: 345619
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/323 between game devices with different hardware characteristics, e.g. hand-held game devices connectable to game consoles or arcade machines
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6045 : for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6669 : using a plurality of virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character change rooms
• CPC Class Codes: A63F 2300/6676 : by dedicated player input
• CPC Class Codes: A63F 2300/6684 : by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball

Abstract

An attitude of an object arranged in a virtual world is controlled based on attitude data outputted from a portable display device. Further, the object is caused to move in the virtual world, based on data based on a load applied to a load detection device. Then, a first image showing the virtual world including at least the object is displayed on the portable display device.

Claims

1. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is configured to display an image on a portable display device which outputs at least attitude data based on an attitude of a body of the portable display device, wherein the information processing program causes the computer to:

obtain load data based on a load applied to a load detection device;

control an attitude of an object arranged in a virtual world, by changing the attitude of the object to correspond to changes in the attitude of the body of the portable display device;

cause the object to move in the virtual world based on the load data; and

display, on the portable display device, a first image showing the virtual world viewed from a virtual camera having a position and an attitude set to follow the attitude of the object, the attitude of the object controlled based on the attitude data corresponding to the changes in the attitude of the body of the portable display device.

2. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to calculate an attitude of the portable display device based on the attitude data, and

control the attitude of the object based on the attitude of the portable display device.

3. The non-transitory computer-readable storage medium according to claim 2, wherein

an attitude of the portable display device is calculated with respect to a predetermined direction in a real space, and

the attitude of the object is controlled with respect to a predetermined direction which is set in the virtual world and corresponds to the predetermined direction in the real space, based on the attitude of the portable display device with respect to the predetermined direction in the real space.

4. The non-transitory computer-readable storage medium according to claim 3, wherein the information processing program further causes the computer to:

calculate, with a direction of gravity in the real space being the predetermined direction, an attitude of the portable display device with respect to the direction of gravity, and

control the attitude of the object with respect to a direction of gravity set in the virtual world, based on the attitude of the portable display device with respect to the direction of gravity in the real space.

5. The non-transitory computer-readable storage medium according to claim 4, wherein the information processing program further causes the computer to:

calculate at least an attitude in which the portable display device rotates around the direction of gravity in the real space, and

control the attitude of the object by causing the object to rotate around the direction of gravity set in the virtual world, based on the attitude in which the portable display device rotates around the direction of gravity in the real space.

6. The non-transitory computer-readable storage medium according to claim 4, wherein the information processing program further causes the computer to:

calculate at least an attitude in which the portable display device swings up and down around a horizontal direction perpendicular to the direction of gravity in the real space, and

control the attitude of the object in which the object swings up and down around a horizontal direction which is set in the virtual world and corresponds to the horizontal direction in the real space, based on the attitude in which the portable display device swings up and down around the horizontal direction in the real space.

7. The non-transitory computer-readable storage medium according to claim 2, wherein the information processing program further causes the computer to:

calculate at least an attitude in which the portable display device rotates around two axes orthogonal to a depth direction of a display screen of the portable display device, the depth direction being perpendicular to the display screen on which the first image is displayed, and

control the attitude of the object by causing the object to rotate around two axes orthogonal to a forward direction of the object, based on the attitude in which the portable display device rotates around the two axes orthogonal to the depth direction of the display screen.

8. The non-transitory computer-readable storage medium according to claim 7, wherein the information processing program further causes the computer to:

calculate at least an attitude in which the portable display device rotates around an axis along a horizontal direction of a display screen of the portable display device and an axis along a vertical direction of the display screen, the axes being orthogonal to the depth direction, and

control the attitude of the object by causing the object to rotate around an axis of a horizontal direction of the object, which is orthogonal to the forward direction of the object, in accordance with the attitude in which the portable display device rotates around the axis of the horizontal direction of the display screen, and cause the object to rotate around an axis of the object, which is orthogonal to the forward direction and the horizontal direction of the object, based on the attitude in which the portable display device rotates around the axis of the vertical direction of the display screen.

9. The non-transitory computer-readable storage medium according to claim 7, wherein the information processing program further causes the computer to:

calculate at least an orientation of the depth direction with respect to the direction of gravity in the real space, and

control the attitude of the object by setting a forward direction of the object such that the positional relation between the direction of gravity set in the virtual world and the forward direction of the object is substantially equal to the positional relation between the direction of gravity in the real space and the depth direction.

10. The non-transitory computer-readable storage medium according to claim 2, wherein the information processing program further causes the computer to:

set a reference attitude of the portable display device in the real space;

calculate an angular difference, around a predetermined axis in the real space, between the attitude of the portable display device at a current time and the reference attitude, based on data outputted from the portable display device;

set a reference attitude of the object in the virtual world; and

control the attitude of the object by causing the direction of the object to rotate from the reference attitude of the object around a predetermined axis in the virtual world, which corresponds to the predetermined axis in the real space, based on the calculated angular difference.

11. The non-transitory computer-readable storage medium according to claim 10, wherein the information processing program further causes the computer to:

rotationally displace the direction of the set reference attitude of the object, around the predetermined axis in the virtual world, by an amount of displacement according to the calculated angular difference, and

control the attitude of the object by causing the direction of the object to rotate from the displaced reference direction, around the predetermined axis in the virtual world, based on the calculated angular difference.

12. The non-transitory computer-readable storage medium according to claim 2, wherein the information processing program further causes the computer to:

arrange a first virtual camera for generating an image of the virtual world such that the object in the virtual world, which is viewed from behind, is included in the first image, and

display, on the portable display device, as the first image, an image showing the virtual world viewed from the first virtual camera.

13. The non-transitory computer-readable storage medium according to claim 12, wherein

the position and the attitude of the first virtual camera are controlled based on the attitude of the object and the position of the object in the virtual world.

14. The non-transitory computer-readable storage medium according to claim 12, wherein

the position and the attitude of the first virtual camera are controlled such that the display position of the object in the first image is fixed.

15. The non-transitory computer-readable storage medium according to claim 2, wherein

the object is caused to move in a forward direction of the controlled object.

16. The non-transitory computer-readable storage medium according to claim 2, wherein the information processing program further causes the computer to:

display, separately from the first image, a second image showing the virtual world on another display device connected to the information processing apparatus.

17. The non-transitory computer-readable storage medium according to claim 16, wherein the information processing program further causes the computer to display, on the other display device, as the second image, an image of the virtual world including the object, in which the virtual world is viewed from a viewpoint different from the viewpoint of the virtual world of the first image.

18. The non-transitory computer-readable storage medium according to claim 17, wherein the information processing program further causes the computer to:

set the viewpoint of the virtual world of the second image at a position apart from the object by a distance longer than the distance between the object and the viewpoint of the virtual world of the first image, and

display, on the other display device, as the second image, a range of the virtual world which is larger than the range of the virtual world shown in the first image.

19. The non-transitory computer-readable storage medium according to claim 17, wherein the information processing program further causes the computer to:

set a viewpoint for generating the second image, at a position in the virtual world, from which the object is viewed in a manner of bird'"'"'s eye viewing, and

display, on the other display device, as the second image, an image in which the object arranged in the virtual world is viewed in a manner of bird'"'"'s eye viewing.

20. The non-transitory computer-readable storage medium according to claim 16, wherein the information processing program further causes the computer to:

set a first virtual camera for generating the first image and a second virtual camera for generating the second image, and cause a line-of-sight direction of the second virtual camera in the virtual world to change in accordance with a change in a line-of-sight direction of the first virtual camera in the virtual world.

21. The non-transitory computer-readable storage medium according to claim 20, wherein the information processing program further causes the computer to:

arrange the first virtual camera such that a forward direction of the object from behind the object in the virtual world is an imaging direction of the first virtual camera, and arrange the second virtual camera such that a position in the forward direction of the object in the virtual world is a point of view of the second virtual camera, and the object is displayed at the same position in the second image, and control the positions and/or the attitudes of the first virtual camera and the second virtual camera in accordance with an action of the object.

22. The non-transitory computer-readable storage medium according to claim 2, wherein

the attitude of the object is controlled so as to be substantially equal to the attitude of the portable display device.

23. The non-transitory computer-readable storage medium according to claim 2, wherein the information processing program further causes the computer to:

calculate a center-of-gravity position of a load applied to the load detection device based on the obtained load data, and

cause the object to move in the movement direction, based on the calculated center-of-gravity position.

24. The non-transitory computer-readable storage medium according to claim 23, wherein

the object is caused to move based on reciprocating motion of the calculated center-of-gravity position.

25. The non-transitory computer-readable storage medium according to claim 24, wherein the information processing program further causes the computer to:

cause the object to move in the movement direction at a movement speed according to a first action performed by the object when the center-of-gravity position, in its reciprocating motion, shifts from one side of a predetermined boundary to the other side, and cause the object to move in the movement direction at a movement speed according to a second action performed by the object when the center-of-gravity position shifts from the other side of the predetermined boundary to the one side.

26. The non-transitory computer-readable storage medium according to claim 23, wherein the information processing program further causes the computer to:

set a movement speed of the object, based on a distance from a preset reference position to the calculated center-of-gravity position, set a movement direction of the object based on the direction from the reference position toward the center-of-gravity position, and cause the object to move in the movement direction at the movement speed.

27. The non-transitory computer-readable storage medium according to claim 2, wherein

the object is caused to move in the movement direction, based on a change in the load represented by the obtained load data.

28. The non-transitory computer-readable storage medium according to claim 27, wherein

the object is caused to move at a movement speed according to a first action performed by the object, at a time point when the load applied to the load detection device increases to be equal to or higher than a predetermined value.

29. The non-transitory computer-readable storage medium according to claim 27, wherein

the object is caused to move when the load applied to the load detection device, which has once increased to be equal to or higher than a predetermined value, decreases to be equal to or lower than a predetermined value.

30. The non-transitory computer-readable storage medium according to claim 29, wherein

the object is caused to move at a movement speed based on a length of time from when the load increases to be equal to or higher than the predetermined value to when the load decreases to be equal to or lower than the predetermined value.

31. The non-transitory computer-readable storage medium according to claim 2, wherein

the portable display device includes at least one of a gyro sensor and an acceleration sensor, and

an attitude of the portable display device is calculated based on data outputted from at least one of the gyro sensor and the acceleration sensor.

32. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to:

output image data representing the first image to the portable display device;

the portable display device;

obtains image data outputted from the information processing apparatus; and

displays the first image represented by the obtained image data.

33. The non-transitory computer-readable storage medium according to claim 32, wherein the information processing program further causes the computer to:

generate compressed image data by compressing the image data representing the first image,

output the generated compressed image data to the portable display device,

obtain the compressed image data outputted from the information processing apparatus,

decompress the compressed image data to obtain image data representing the first display image, and

display the first image represented by the obtained and decompressed image data.

34. The non-transitory computer-readable storage medium according to claim 1, wherein

the attitude of the object is controlled to correspond to a change in direction and a change in attitude of the body of the portable display device, and

the object is displayed on the portable display device so that the movement of the object and attitude of the object correspond to the obtained load data and the change in attitude and change in direction of the portable display device.

35. The non-transitory computer-readable storage medium according to claim 1, wherein the attitude of the object is controlled based on at least a horizontal angular difference of the body of the portable display device.

36. The non-transitory computer-readable storage medium according to claim 35, wherein the horizontal angular difference is an angular difference obtained by projecting an angular difference between a real space direction and a current direction in real space on a horizontal plane.

37. The non-transitory computer-readable storage medium according to claim 35, wherein the horizontal angular difference indicates an angle at which a direction of a surface of the portable display device is changed from an initial attitude around a vertical direction in real space.

38. The non-transitory computer-readable storage medium according to claim 1, wherein the portable display device displays the first image showing the virtual world viewed from the virtual camera having the position behind the object and the attitude set to follow the attitude of the object at the position behind the object.

39. The non-transitory computer-readable storage medium according to claim 38, wherein the virtual camera follows the object and maintains the position behind the object as the object moves in the virtual world.

40. The non-transitory computer-readable storage medium according to claim 1, wherein a direction of a line of sight of the virtual camera and a moving direction of the object are the same.

41. An information processing system including a portable display device, a load detection device, and an information processing apparatus configured to display an image on the portable display device, wherein

the load detection device is configured to output, to the information processing apparatus, data based on a load applied to the load detection device,

the portable display device is configured to output, to the information processing apparatus, at least attitude data according to an attitude of a body of the portable display device,

the information processing apparatus is configured to:

obtain load data outputted from the load detection device;

set a movement direction of an object arranged in a virtual world, by changing the movement direction of the object to correspond to changes in the attitude of the body of the portable display device;

cause the object to move in the movement direction in the virtual world based on the obtained load data; and

output, to the portable display device, a first image showing the virtual world viewed from a virtual camera having a position and an attitude set to follow the movement direction of the object, the movement direction of the object set based on the attitude data corresponding to the changes in the attitude of the body of the portable display device, and

the portable display device is further configured to;

obtain data of the first image outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

42. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus configured to display an image on a portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining load data based on a load applied to a load detection device;

controlling an attitude of an object arranged in a virtual world, by changing the attitude of the object to correspond to changes in the attitude of the body of the portable display device;

causing the object to move in the movement direction in the virtual world based on the obtained load data; and

displaying, on the portable display device, as a first image, an image showing the virtual world viewed from a virtual camera having a position and an attitude set to follow the attitude of the object, the attitude of the object controlled based on the attitude data corresponding to the changes in the attitude of the body of the portable display device.

43. An information processing system including a portable display device and a load detection device, wherein

the load detection device is configured to output, to the portable display device, data based on a load applied to the load detection device, and

the portable display device is configured to:

generate attitude data in accordance with an attitude of a body of the portable display device;

obtain load data outputted from the load detection device;

control the attitude of an object arranged in the virtual world, by changing the attitude of the object to correspond to changes in the attitude of the body of the portable display device;

cause the object to move in the virtual world, based on the obtained load data;

generate a first image showing the virtual world viewed from a virtual camera having a position and an attitude set to follow the attitude of the object, the attitude of the object controlled based on the attitude data corresponding to the changes in the attitude of the body of the portable display device; and

display the generated first image.