Computer-readable storage medium, information processing system, and information processing method for controlling a virtual object using attitude data

• US 9,643,085 B2
• Filed: 10/20/2011
• Issued: 05/09/2017
• Est. Priority Date: 03/08/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Matsunaga, Hiroshi, Hayashi, Yugo, Sumaki, Kazuya, Minato, Keiichi
• Primary Examiner(s): Nguyen, Vu
• Application Number:
  US13/277,561
• Publication Number:
  US 20120229516A1
• Time in Patent Office: 2,028 Days
• Field of Search: 345659
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/214 for locating contacts on a surface, e.g. floor mats or touch pads
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/323 between game devices with different hardware characteristics, e.g. hand-held game devices connectable to game consoles or arcade machines
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/573 using trajectories of game objects, e.g. of a golf ball according to the point of impact
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/403 Connection between platform and handheld device
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball

Abstract

An object arranged in a virtual world is caused to move in accordance with data based on a load applied to a load detection device. Then, based on attitude data outputted from a portable display device and a position of the object in the virtual world, a first virtual camera for generating an image of the virtual world is controlled, and a first image representing the virtual world viewed from the first virtual camera is displayed on the portable display device.

Claims

1. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a separately housed portable display device which outputs at least attitude data based on an attitude of a body of the portable display device, wherein the information processing program, when executed, causes the computer to:

obtain data based on a load applied to a load detection device;

cause an object arranged in a virtual world to move based on the obtained data;

render, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

control the first virtual camera, which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the body of the portable display device, and a position of the object in the virtual world;

render, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmit, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

2. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to calculate an attitude of the portable display device, based on the attitude data, and

control the first virtual camera based on the calculated attitude of the portable display device and the position of the object.

3. The non-transitory computer-readable storage medium according to claim 2, wherein

at least one of a position and an attitude of the first virtual camera in the virtual world is controlled based on the attitude of the portable display device.

4. The non-transitory computer-readable storage medium according to claim 3, wherein

an attitude of the portable display device is calculated with respect to a predetermined direction in a real space, and

at least one of the position and the attitude of the first virtual camera is controlled, with reference to a direction which corresponds to the predetermined direction and is set in the virtual world, based on the attitude of the portable display device with respect to the predetermined direction in the real space.

5. The non-transitory computer-readable storage medium according to claim 4, wherein

the predetermined direction is a direction of gravity in the real space, and an attitude of the portable display device is calculated with respect to the direction of gravity, and

at least one of the position and the attitude of the first virtual camera is controlled, with reference to a direction of gravity set in the virtual world, based on the attitude of the portable display device with respect to the direction of gravity in the real space.

6. The non-transitory computer-readable storage medium according to claim 5, wherein

at least an attitude in which the portable display device rotates around the direction of gravity in the real space is calculated, and

the attitude of the first virtual camera is controlled by rotating the first virtual camera around the direction of gravity set in the virtual world, based on the attitude in which the portable display device rotates around the direction of gravity in the real space.

7. The non-transitory computer-readable storage medium according to claim 2, wherein

at least one of a position and an attitude of the first virtual camera is controlled by controlling a line-of-sight direction of the first virtual camera in the virtual world, the line-of-sight direction being based on the attitude of the portable display device.

8. The non-transitory computer-readable storage medium according to claim 2, wherein

an attitude of the first virtual camera is controlled so as to be the same as the attitude of the portable display device.

9. The non-transitory computer-readable storage medium according to claim 2, wherein

the first virtual camera is controlled such that the object is included in the first image.

10. The non-transitory computer-readable storage medium according to claim 2, wherein

the object is caused to move in a direction based on a line-of-sight direction of the first virtual camera.

11. The non-transitory computer-readable storage medium according to claim 2, wherein

the portable display device includes at least one of a gyro sensor and an acceleration sensor, and

an attitude of the portable display device is calculated based on data outputted from at least one of the gyro sensor and the acceleration sensor.

12. The non-transitory computer-readable storage medium according to claim 2, wherein

image data representing the first image is transmitted to the portable display device, and

the portable display device is configured to:

obtain image data outputted from the information processing apparatus; and

display the first image represented by the obtained image data.

13. The non-transitory computer-readable storage medium according to claim 12, wherein

the information processing program further causes the computer to generate compressed image data by compressing the image data representing the first image,

output the generated compressed image data to the portable display device, and

obtain the compressed image data outputted from the information processing apparatus, wherein

the portable display device is further configured to decompress the compressed image data to obtain image data representing the first display image, and

display the first image represented by the obtained and decompressed image data.

14. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to set, based on the position of the object in the virtual world, the second virtual camera for generating an image of the virtual world, in a position different from the position of the first virtual camera such that the object is included in the second image.

15. The non-transitory computer-readable storage medium according to claim 14, wherein

the second virtual camera is set in a position apart from the object by a distance longer than the distance between the object and the first virtual camera, and

a range of the virtual world, which range is broader than the range of the virtual world as the first image, is displayed on the stationary display device as the second image.

16. The non-transitory computer-readable storage medium according to claim 14, wherein

the second virtual camera is set in a position in the virtual world from which the second virtual camera views the object in a manner of bird'"'"'s eye viewing, and

an image in which the object arranged in the virtual world is viewed in a manner of bird'"'"'s eye viewing is displayed on the stationary display device as the second image.

17. The non-transitory computer-readable storage medium according to claim 1, wherein

a line-of-sight direction of the second virtual camera in the virtual world is changed in accordance with a change in a line-of-sight direction of the first virtual camera in the virtual world.

18. The non-transitory computer-readable storage medium according to claim 1, wherein the first and second image are rendered using a graphics processing unit of the information processing apparatus.

19. The non-transitory computer-readable storage medium according to claim 1, wherein the signal transmitted from the information processing apparatus to the separately housed portable display device is encoded using a codec.

20. A non-transitory computer-readable storage medium comprising an information processing program executed on a computer of an information processing apparatus which is capable of displaying an image on a separately housed portable display device which outputs at least attitude data according to an attitude of a body of the portable display device, wherein the information processing program, when executed, causes the computer to:

obtain data based on a load applied to a load detection device;

render, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data and the attitude data, corresponding to the attitude of the body of the portable display device;

render, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmit, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

21. An information processing system including a portable display device, a load detection device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the load detection device configured to output, to the information processing apparatus, data based on a load applied to the load detection device,

the portable display device configured to output, to the separately housed information processing apparatus, at least attitude data according to an attitude of the portable display device,

the information processing apparatus configured to:

obtain data outputted from the load detection device;

cause an object arranged in the virtual world to move, based on the obtained data;

render a first image from a first virtual camera using processing circuitry of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the portable display device, and a position of the object in the virtual world;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing circuitry of the information processing apparatus;

transmit, to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus, and

the portable display device further configured to;

obtain the data of the first image transmitted from the information processing apparatus; and

display the first image represented by the obtained image data.

22. An information processing system including a portable display device, a load detection device, and an information processing apparatus capable of displaying an image on the portable display device, wherein

the load detection device configured to output, to the information processing apparatus, data based on a load applied to the load detection device,

the portable display device configured to output, to the separately housed information processing apparatus, at least attitude data according to an attitude of the portable display device,

the information processing apparatus configured to:

obtain data outputted from the load detection device;

render a first image from a first virtual camera using processing circuitry of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data, and the attitude data, corresponding to the attitude of the portable display device;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing circuitry of the information processing apparatus;

transmit, to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

output, to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus, and

the portable display device further configured to;

obtain the data of the first image transmitted from the information processing apparatus; and

display the first image represented by the obtained image data.

23. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a separately housed portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining data based on a load applied to a load detection device;

causing an object arranged in a virtual world to move based on the obtained data;

rendering, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

controlling the first virtual camera which generates the first image of the virtual world, based on the attitude data, corresponding to the attitude of the body of the portable display device, and a position of the object in the virtual world;

rendering, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmitting, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

outputting, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

24. An information processing method executed by one processor included in an information processing system that includes at least one information processing apparatus capable of displaying an image on a separately housed portable display device that outputs at least attitude data according to an attitude of a body of the portable display device, or by a plurality of processors included in the information processing system in a cooperative manner, the information processing method comprising:

obtaining data based on a load applied to a load detection device;

rendering, using processing circuitry of the information processing apparatus, a first image from a first virtual camera;

controlling the first virtual camera which generates the first image of the virtual world, based on the obtained data and the attitude data, corresponding to the attitude of the body of the portable display device;

rendering, using the processing circuitry of the information processing apparatus, a second image from a second virtual camera having a different viewpoint than the first virtual camera;

transmitting, from the information processing apparatus to the separately housed portable display device, an encoded signal including the first image representing the virtual world viewed from the first virtual camera and rendered using the processing circuitry of the information processing apparatus; and

outputting, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera and rendered using the processing circuitry of the information processing apparatus.

25. An information processing apparatus, comprising:

a processing system having at least one processor, the processing system configured to;

obtain data based on a load applied to a load detection device;

cause an object arranged in a virtual world to move based on the obtained data;

render a first image of a virtual world from a first virtual camera using the processing system of the information processing apparatus;

control the first virtual camera, which generates the first image of the virtual world, based on attitude data, corresponding to an attitude of a body of a separately housed portable display device, and a position of the object in the virtual world;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing system of the information processing apparatus;

transmit, from the information processing apparatus to the separately housed portable display device, a signal including the first image representing the virtual world viewed from the first virtual camera; and

output, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera.

26. An information processing apparatus, comprising:

a processing system having at least one processor, the processing system configured to;

obtain data based on a load applied to a load detection device;

render a first image of a virtual world from a first virtual camera using the processing system of the information processing apparatus;

control the first virtual camera which generates the first image of the virtual world, based on the obtained data and attitude data, corresponding to an attitude of a body of a separately housed portable display device;

render a second image from a second virtual camera having a different viewpoint than the first virtual camera using the processing system of the information processing apparatus;

transmit, from the information processing apparatus to the separately housed portable display device, a signal including the first image representing the virtual world viewed from the first virtual camera; and

output, from the information processing apparatus to a separately housed stationary display device operatively coupled to the information processing apparatus, the second image representing the virtual world viewed from the second virtual camera.