Information processing program, information processing system, information processing apparatus, and information processing method

• US 8,684,837 B2
• Filed: 06/09/2011
• Issued: 04/01/2014
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Kitahara, Shinji
• Primary Examiner(s): Suhol, Dmitry
• Assistant Examiner(s): LARSEN, CARL VICTOR
• Application Number:
  US13/156,690
• Publication Number:
  US 20120172127A1
• Time in Patent Office: 1,027 Days
• Field of Search: 463/30, 463/31, 345/619, 345/632, 345/633
• US Class Current: 463/31
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/525 Changing parameters of virtual cameras
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

A moving object and a topography object having a portion with different heights are positioned in a virtual space, and the moving object is controlled so as to move on the topography object. A real world image is obtained from an imaging device, and a position and an orientation of a marker in the real world image are detected. On the basis of a result of the detection, a virtual camera is positioned in the virtual space, and a moving object and a topography object are rendered on the basis of the virtual camera. Here, the real world image obtained from the imaging device is used as a texture for the topography object. The rendered moving object and the topography object are synthesized on the real world image and a synthesized image is displayed on a display device.

Claims

1. A non-transitory computer-readable storage medium having stored therein an information processing program for causing a computer of an information processing apparatus connected with an imaging device and a display device to perform functionality comprising:

obtaining a real world image by capturing an image of a real world by using the imaging device;

positioning a moving object in a virtual space;

positioning a topography object having a portion with different heights in the virtual space;

moving the moving object depending on a shape of the topography object;

mapping at least a part of the real world image as a texture onto the topography object;

generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture; and

displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on the real world image.

2. The non-transitory computer-readable storage medium according to claim 1, wherein the information processing program further causes the computer to perform functionality comprising deforming the positioned topography object.

3. The non-transitory computer-readable storage medium according to claim 2, wherein the topography object is deformed periodically over time.

4. The non-transitory computer-readable storage medium according to claim 2, wherein the topography object is deformed in a direction along a surface of the topography object.

5. The non-transitory computer-readable storage medium according to claim 2, wherein the topography object is deformed when a condition is satisfied.

6. The non-transitory computer-readable storage medium according to claim 1, wherein

positioning the topography object includes:

positioning a planar topography object in the virtual space; and

changing a height of at least a part of the positioned planar topography object.

7. The non-transitory computer-readable storage medium according to claim 1, wherein

the information processing program further causes the computer to perform functionality comprising:

detecting an object to be identified from the real world image;

positioning a virtual camera in the virtual space on the basis of a result of the detection; and

rendering the moving object and the topography object on the basis of the virtual camera.

8. The non-transitory computer-readable storage medium according to claim 1, wherein

the imaging device and the display device are integrally provided with the information processing apparatus, and

moving the moving object includes:

determining a moving direction of the moving object on the basis of a position and an orientation of the information processing apparatus in the real world.

9. The non-transitory computer-readable storage medium according to claim 1, wherein

the imaging device and the display device are integrally provided with the information processing apparatus, and

moving the moving object includes:

determining a moving speed of the moving object on the basis of a position and an orientation of the information processing apparatus in the real world.

10. The non-transitory computer-readable storage medium according to claim 1, wherein

the moving object moves in accordance with an operation by the user, and

the information processing program further causes the computer to perform functionality comprising:

executing a predetermined game process on the basis of a position of the moving object which has been moved in the virtual space.

11. The non-transitory computer-readable storage medium according to claim 1, wherein the topography object is formed from a real-world terrain and is viewed as extending from the real-world terrain.

12. The non-transitory computer-readable storage medium according to claim 1, wherein the topography object is formed from a real-world terrain and is viewed as extending into the real-world terrain.

13. The non-transitory computer-readable storage medium according to claim 1, wherein the virtual space image is displayed on the display device so as to be viewed by the user such that the virtual space image is superimposed on one of the real world image and the real world seen through a screen of the display device.

14. An information processing apparatus comprising:

a real world image obtainer for obtaining a real world image by capturing an image of a real world by using an imaging device;

a moving object positioner for positioning a moving object in a virtual space;

a topography object positioner for positioning a topography object having a portion with different heights in the virtual space;

a movement controller for moving the moving object depending on a shape of the topography object;

a texture mapper for mapping at least a part of the real world image as a texture onto the topography object;

a virtual space image generator for generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture; and

a display controller for displaying the virtual space image on a display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on the real world image.

15. An information processing system comprising:

an imaging device;

a display device; and

at least one processor configured to perform functionality comprising;

obtaining a real world image by capturing an image of a real world by using the imaging device,

positioning a moving object in a virtual space,

positioning a topography object having a portion with different heights in the virtual space,

moving the moving object depending on a shape of the topography object,

mapping at least a part of the real world image as a texture onto the topography object,

generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture, and

displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on the real world image.

16. An information processing method which is implemented by a control section of an information processing apparatus, the method comprising:

an image of a real world by using an imaging device;

positioning a moving object in a virtual space;

positioning a topography object having a portion with different heights in obtaining a real world image obtained by capturing the virtual space;

moving the moving object depending on a shape of the topography object;

mapping at least a part of the real world image as a texture onto the topography object;

generating a virtual space image by rendering the moving object and the topography object onto which at least the part of the real world image has been mapped as the texture; and

displaying the virtual space image on a display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on the real world image.

17. An information processing system comprising:

a marker which is positioned in a real world;

an imaging device;

a display device; and

at least one processor configured to perform functionality comprising;

obtaining a real world image by capturing an image of the real world by using the imaging device,

detecting the marker from the real world image,

positioning a virtual camera in a virtual space on the basis of a result of the detection,

positioning a moving object in the virtual space,

positioning a topography object having a portion with different heights in the virtual space,

moving the moving object depending on a shape of the topography object,

mapping at least a part of the real world image as a texture onto the topography object,

generating a virtual space image by rendering, on the basis of the virtual camera, the moving object and the topography object onto which at least the part of the real world image has been mapped as the displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed on the real world image.

18. An information processing system, comprising:

a memory; and

at least one processor operatively associated with the memory and configured to perform functionality comprising;

obtaining a real world image by capturing an image of a real world using an imaging device,

generating a topography object having a portion with different heights in a virtual space, the topography object having a texture containing at least a part of the obtained real world image,

conditioning movement of a generated moving object in the virtual space based on a shape of the topography object,

generating a virtual space image by rendering the moving object and the topography object, and

displaying the virtual space image on the display device so as to be viewed by a user in a manner such that the virtual space image is superimposed over the real world image.

19. The system of claim 18, wherein the texture of the topography object as viewed in the displayed virtual space image varies depending upon the height of the topography object in the virtual space.