Computer-readable storage medium having program stored therein, apparatus, system, and method, for performing game processing

• US 9,095,774 B2
• Filed: 06/14/2011
• Issued: 08/04/2015
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Original Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Inventors: Noge, Tetsuya
• Primary Examiner(s): LIM, SENG HENG
• Application Number:
  US13/159,865
• Publication Number:
  US 20120077582A1
• Time in Patent Office: 1,512 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/577 using determination of contact between game characters or objects, e.g. to avoid collision between virtual racing cars
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/818 Fishing
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/663 for simulating liquid objects, e.g. water, gas, fog, snow, clouds
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8035 Virtual fishing

Abstract

A storage medium has stored therein a game program that in game processing, controls a correlation between a player object and a non-player object through a virtual surface that is set on the basis of a marker for generating a superimposed image by combining a real world image and a virtual world image. The game program causes a computer of an apparatus, which includes a display device for providing the superimposed image to a user and an imaging section for taking an image of the marker, to operate as predetermined means.

Claims

1. A non-transitory computer-readable storage medium having a game program stored therein, the game program causing a computer of a game apparatus, which is connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, to perform functionality comprising:

obtaining a taken image obtained by the imaging device taking an image of the real space;

detecting a specific object from the obtained taken image;

calculating an imaging device position of the imaging device with respect to the detected specific object;

setting a virtual camera in a virtual space on the basis of the imaging device position;

setting, on the basis of the imaging device position, a virtual surface in the virtual space and defining subspaces extending at least along a depth direction relative to the virtual surface in the virtual space;

providing, on the basis of a virtual camera position of the virtual camera, a first virtual object in a first subspace of the virtual space that is defined by the virtual surface;

generating a virtual object image by taking, by the virtual camera, the virtual space including the first virtual object;

displaying the virtual object image on the display device such that the virtual object image is superimposed on the real space on the screen and viewed by a user;

providing a second virtual object different from the first virtual object, in the virtual space in accordance with a progress of a game; and

controlling the progress of the game in accordance with changing a positional relation of either one of the first virtual object or the second virtual object or both of them with respect to the virtual surface.

2. The non-transitory computer-readable storage medium according to claim 1, wherein the game program further causes the computer to execute functionality comprising performing a determination of a condition concerning a correlation between the first virtual object and the second virtual object for subsequent game processing.

3. The non-transitory computer-readable storage medium according to claim 1, wherein when at least a part of the first virtual object that is present in the first subspace is moved through the virtual surface into a second subspace that is farther away from the virtual camera than the virtual surface, a determination of a condition concerning a correlation between the first virtual object and the second virtual object for subsequent game processing is performed.

4. The non-transitory computer-readable storage medium according to claim 3, wherein the condition concerning the correlation is that a positional relation of the first virtual object and the second virtual object satisfies a predetermined condition.

5. The non-transitory computer-readable storage medium according to claim 3, wherein the condition concerning the correlation is that a moving amount of the first virtual object that is defined on the basis of the obtained imaging device position reaches a predetermined value.

6. The non-transitory computer-readable storage medium according to claim 3, wherein the condition concerning the correlation is that even when a moving amount of the first virtual object that is defined on the basis of the obtained imaging device position does not reach a predetermined value, the specific object from the taken image is not detected.

11. The non-transitory computer-readable storage medium according to claim 1, wherein the real space exists in a real world environment and the virtual space exists in a virtual gaming environment.

12. The non-transitory computer-readable storage medium according to claim 1, wherein the virtual surface represents a three-dimensional virtual surface extending along a first axis, a second axis, and a third axis in the virtual space.

13. The non-transitory computer-readable storage medium according to claim 12, wherein

the first axis extends along a longitudinal direction, the second axis extends along a latitudinal direction, and the third axis extends along the depth direction in the virtual space, and

the subspaces are defined in the third axis extending along the depth direction in the virtual space.

7. A game apparatus having at least one processor and configured to be connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, the game apparatus configured to:

obtain a taken image obtained by the imaging device taking an image of the real space;

detect a specific object from the obtained taken image;

calculate an imaging device position of the imaging device with respect to the specific object on the basis of a detection result of the specific object;

set a virtual camera in a virtual space on the basis of the imaging device position;

set, on the basis of the imaging device position, a virtual surface in the virtual space and define subspaces extending at least along a depth direction relative to the virtual surface in the virtual space;

provide, on the basis of a position of the virtual camera, a first virtual object in a first subspace of the virtual space that is defined by the virtual surface;

generate a virtual object image by taking, by the virtual camera, the virtual space including the first virtual object;

display the virtual object image on the display device such that the virtual object image is superimposed on the real space on the screen and viewed by a user;

provide a second virtual object different from the first virtual object, in the virtual space in accordance with a progress of a game; and

control the progress of the game in accordance with changing a positional relation of either one of the first virtual object or the second virtual object or both of them with respect to the virtual surface.

8. A game system comprising:

an imaging device;

a display device configured to allow a real space to be viewed on a screen of the display device; and

a game apparatus operatively coupled to the imaging device and the display device and configured to;

obtain a taken image obtained by the imaging device taking an image of the real space,

detect a specific object from the obtained taken image,

calculate an imaging device position of the imaging device with respect to the specific object on the basis of a detection result of the specific object,

set a virtual camera in a virtual space on the basis of the imaging device position,

set, on the basis of the imaging device position, a virtual surface in the virtual space and define subspaces extending at least along a depth direction relative to the virtual surface in the virtual space,

provide, on the basis of a position of the virtual camera, a first virtual object in a first subspace of the virtual space that is defined by the virtual surface,

generate a virtual object image by taking, by the virtual camera, the virtual space including the first virtual object,

display the virtual object image on the display device such that the virtual object image is superimposed on the real space on the screen and viewed by a user,

provide a second virtual object different from the first virtual object, in the virtual space in accordance with a progress of a game, and

control the progress of the game in accordance with changing a positional relation of either one of the first virtual object or the second virtual object or both of them with respect to the virtual surface.

9. A method for performing game processing in a game system having one or more processors, the method comprising:

obtaining a taken image obtained by taking an image of a real space the imaging device by an imaging device connected to the game system;

detecting a specific object from the obtained taken image;

calculating, via the one or more processors, an imaging device position of the imaging device with respect to the detected specific object;

setting a virtual camera in a virtual space on the basis of the imaging device position;

setting, on the basis of the imaging device position, a virtual surface in the virtual space and defining subspaces extending at least along a depth direction relative to the virtual surface in the virtual space;

providing, on the basis of a position of the virtual camera, a first virtual object in a first subspace of the virtual space that is defined by the virtual surface;

generating a virtual object image by taking, by the virtual camera, the virtual space including the first virtual object;

displaying the virtual object image on a display device connected to the game system, such that the virtual object image is superimposed on the real space on a screen of the display device and viewed by a user;

providing a second virtual object different from the first virtual object, in the virtual space in accordance with a progress of a game; and

controlling the progress of the game in accordance with changing a positional relation of either one of the first virtual object or the second virtual object or both of them with respect to the virtual surface.

10. A game system connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, the game system comprising:

a marker having a feature of serving as a specific object; and

a processing system having at least one processor, the processing system configured to;

obtain a taken image obtained by the imaging device taking an image of the real space;

detect the specific object from the obtained taken image;

calculate an imaging device position of the imaging device with respect to the specific object on the basis of a detection result of the specific object;

set a virtual camera in a virtual space on the basis of the imaging device position;

set, on the basis of the imaging device position, a virtual surface in the virtual space and define subspaces extending at least along a depth direction relative to the virtual surface in the virtual space;

provide, on the basis of a position of the virtual camera, a first virtual object in a first subspace of the virtual space that is defined by the virtual surface;

generate a virtual object image by taking, by the virtual camera, the virtual space including the first virtual object;

display the virtual object image on the display device such that the virtual object image is superimposed on the real space on the screen and viewed by a user;

provide a second virtual object different from the first virtual object, in the virtual space in accordance with a progress of a game; and

control the progress of the game in accordance with changing a positional relation of either one of the first virtual object or the second virtual object or both of them with respect to the virtual surface.