INFORMATION PROCESSING PROGRAM, INFORMATION PROCESSING SYSTEM, INFORMATION PROCESSING APPARATUS, AND INFORMATION PROCESSING METHOD

• US 20120172127A1
• Filed: 06/09/2011
• Published: 07/05/2012
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: KITAHARA, Shinji
• Granted Patent: US 8,684,837 B2
• Time in Patent Office: Days
• US Class Current: 463/37
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

A moving object and a topography object having a portion with different heights are positioned in a virtual space, and the moving object is controlled so as to move on the topography object. A real world image is obtained from an imaging device, and a position and an orientation of a marker in the real world image are detected. On the basis of a result of the detection, a virtual camera is positioned in the virtual space, and a moving object and a topography object are rendered on the basis of the virtual camera. Here, the real world image obtained from the imaging device is used as a texture for the topography object. The rendered moving object and the topography object are synthesized on the real world image and a synthesized image is displayed on a display device.

Claims

1. A computer-readable storage medium having stored therein an information processing program for causing a computer of an information processing apparatus connected with an imaging device and a display device to function as:

real world image obtaining means for obtaining a real world image by capturing an image of a real world by using the imaging device;

moving object positioning means for positioning a moving object in a virtual space;

topography object positioning means for positioning a topography object having a portion with different heights in the virtual space;

movement control means for moving the moving object depending on a shape of the topography object;

texture mapping means for mapping at least a part of the real world image as a texture onto the topography object;

virtual space image generation means for generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture;

display control means for displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.

2. The computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to function as

deformation means for deforming the topography object positioned by the topography object positioning means.

3. The computer-readable storage medium according to claim 2, wherein

the deformation means deforms the topography object periodically over time.

4. The computer-readable storage medium according to claim 2, wherein

the deformation means deforms the topography object in a direction along a surface of the topography object.

5. The computer-readable storage medium according to claim 2, wherein

the deformation means deforms the topography object when a predetermined condition is satisfied.

6. The computer-readable storage medium according to claim 1, wherein

the topography object positioning means includes:

initial positioning means for positioning a planar topography object in the virtual space; and

initial deformation means for changing a height of at least a part of the planar topography object positioned by the initial positioning means.

7. The computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to function as:

detection means for detecting a predetermined object to be indentified from the real world image; and

virtual camera positioning means for positioning a virtual camera in the virtual space on the basis of a result of the detection by the detection means, and

the virtual space image generation means renders the moving object and the topography object on the basis of the virtual camera.

8. The computer-readable storage medium according to claim 1, wherein

the imaging device and the display device are integrally provided with the information processing apparatus, and

the movement control means includes:

moving direction determination means for determining a moving direction of the moving object on the basis of a position and an orientation of the information processing apparatus in the real world.

9. The computer-readable storage medium according to claim 1, wherein

the imaging device and the display device are integrally provided with the information processing apparatus, and

the movement control means includes:

moving speed determination means for determining a moving speed of the moving object on the basis of a position and an orientation of the information processing apparatus in the real world.

10. The computer-readable storage medium according to claim 1, wherein

the movement control means moves the moving object in accordance with an operation by the user, and

the information processing program further causes the computer to function as:

game process means for executing a predetermined game process on the basis of a position of the moving object which has been moved by the movement control means in the virtual space.

11. An information processing apparatus comprising:

real world image obtaining means for obtaining a real world image by capturing an image of a real world by using an imaging device;

moving object positioning means for positioning a moving object in a virtual space;

topography object positioning means for positioning a topography object having a portion with different heights in the virtual space;

movement control means for moving the moving object depending on a shape of the topography object;

texture mapping means for mapping at least a part of the real world image as a texture onto the topography object;

virtual space image generation means for generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture;

display control means for displaying the virtual space image on a display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.

12. An information processing system comprising:

an imaging device;

a display device;

real world image obtaining means for obtaining a real world image by capturing an image of a real world by using the imaging device;

moving object positioning means for positioning a moving object in a virtual space;

topography object positioning means for positioning a topography object having a portion with different heights in the virtual space;

movement control means for moving the moving object depending on a shape of the topography object;

texture mapping means for mapping at least a part of the real world image as a texture onto the topography object;

virtual space image generation means for generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture;

display control means for displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.

13. An information processing method which is processed by a control section of an information processing apparatus, the method comprising:

a real world image obtaining step of obtaining a real world image obtained by capturing an image of a real world by using an imaging device;

a moving object positioning step of positioning a moving object in a virtual space;

a topography object positioning step of positioning a topography object having a portion with different heights in the virtual space;

a movement control step of moving the moving object depending on a shape of the topography object;

a texture mapping step of mapping at least a part of the real world image as a texture onto the topography object;

a virtual space image generation step of generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture;

a display control step of displaying the virtual space image on a display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.

14. An information processing system comprising:

a marker which is positioned in a real world;

an imaging device;

a display device;

real world image obtaining means for obtaining a real world image by capturing an image of the real world by using the imaging device;

detection means for detecting the marker from the real world image;

virtual camera positioning means for positioning a virtual camera in a virtual space on the basis of a result of the detection by the detection means;

moving object positioning means for positioning a moving object in the virtual space;

topography object positioning means for positioning a topography object having a portion with different heights in the virtual space;

movement control means for moving the moving object depending on a shape of the topography object;

texture mapping means for mapping at least a part of the real world image as a texture onto the topography object;

virtual space image generation means for generating a virtual space image by rendering, on the basis of the virtual camera, the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture; and

display control means for displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.