COMPUTER-READABLE STORAGE MEDIUM HAVING DISPLAY CONTROL PROGRAM STORED THEREIN, DISPLAY CONTROL APPARATUS, DISPLAY CONTROL SYSTEM, AND DISPLAY CONTROL METHOD

• US 20120079426A1
• Filed: 04/15/2011
• Published: 03/29/2012
• Est. Priority Date: 09/24/2010
• Status: Abandoned Application

• Current Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Original Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Inventors: JIN, Toshikazu, Nishimura, Yuuki
• Application Number:
  US13/087,806
• Publication Number:
  US 20120079426A1
• Time in Patent Office: Days
• US Class Current: 715/810
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/837 Shooting of targets
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 30/27 involving lenticular arrays
• CPC Class Codes: G02B 30/30 involving parallax barriers
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

A game apparatus obtains a real world image (71L, 71R) taken with an imaging device, and detects a marker (60) from the real world image (71L, 71R). The game apparatus calculates a relative position of the imaging device and the marker (60) on the basis of the detection result of the marker (60), and sets a virtual camera in a virtual space on the basis of the calculation result. The game apparatus locates a selection object (01) that is associated with a menu item selectable by a user and is to be selected by the user, as a virtual object at a predetermined position in the virtual space that is based on the position of the marker (60). The game apparatus takes an image of the virtual space with the virtual camera, generates an object image of the selection object (01), and generates a superimposed image in which the object image is superimposed on the real world image (71L, 71R).

Claims

1. A computer-readable storage medium having a display control program stored therein, the display control program causing a computer of a display control apparatus, which is connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, to operate as:

taken image obtaining means for obtaining a taken image obtained by using the imaging device;

detection means for detecting a specific object from the taken image;

calculation means for calculating a relative position of the imaging device and the specific object on the basis of a detection result of the specific object by the detection means;

virtual camera setting means for setting a virtual camera in a virtual space on the basis of a calculation result by the calculation means;

object location means for locating a selection object that corresponds to a menu item selectable by a user and is to be selected by the user, at a predetermined position in the virtual space that is based on a position of the specific object;

object image generation means for taking an image of the virtual space with the virtual camera and generating an object image of the selection object; and

display control means for displaying the object image on the display device such that the object image is superimposed on the real space on the screen and viewed by the user.

2. The computer-readable storage medium according to claim 1, wherein the display control program further causes the computer to operate as:

selection fixing means for fixing selection of the selection object in accordance with an operation of the user; and

activation means for activating a predetermined process of the menu item corresponding to the fixed selection object when the selection of the selection object is fixed by the selection fixing means.

3. The computer-readable storage medium according to claim 2, wherein the predetermined process includes a process based on the detection result of the specific object by the detection means.

4. The computer-readable storage medium according to claim 2, wherein

the display control program further causes the computer to operate as reception means for receiving an instruction to redisplay the selection object from the user during a period when the predetermined process is performed, and

when the instruction to redisplay the selection object is received by the reception means, the object location means locates the selection object again.

5. The computer-readable storage medium according to claim 2, wherein the activation means activates an application as the predetermined process.

6. The computer-readable storage medium according to claim 1, wherein the display control program further causes the computer to operate as selection means for selecting the selection object in accordance with a movement of either one of the display control apparatus or the imaging device.

7. The computer-readable storage medium according to claim 6, wherein the selection means selects the selection object when the selection object is located on a sight line of the virtual camera that is set by the virtual camera setting means or on a predetermined straight line parallel to the sight line.

8. The computer-readable storage medium according to claim 1, wherein the display control program further causes the computer to operate as cursor display means for displaying a cursor image at a predetermined position in a display area in which the object image is displayed.

9. The computer-readable storage medium according to claim 2, wherein the display control program further causes the computer to operate as:

selection means for selecting the selection object in accordance with a specific movement of either one of the display control apparatus or the imaging device; and

processing means for progressing the predetermined process activated by the activation means, in accordance with the specific movement of either one of the display control apparatus or the imaging device.

10. The computer-readable storage medium according to claim 1, wherein

the display control program further causes the computer to operate as:

selection means for selecting the selection object in accordance with an inclination of either one of the display control apparatus or the imaging device;

determination means for determining whether or not a distance between the specific object and the imaging device is equal to or less than a predetermined distance; and

warning display means for displaying a warning on the display device when it is determined that the distance between the specific object and the imaging device is equal to or less than the predetermined distance, and

the predetermined distance is set to such a distance that, by tilting either one of the display control apparatus or the imaging device to such an extent as to be able to select the selection object, the specific object is not included in the taken image.

11. A display control apparatus connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, the display control apparatus comprising:

taken image obtaining means for obtaining a taken image obtained by using the imaging device;

detection means for detecting a specific object from the taken image;

calculation means for calculating a relative position of the imaging device and the specific object on the basis of a detection result of the specific object by the detection means;

virtual camera setting means for setting a virtual camera in a virtual space on the basis of a calculation result by the calculation means;

object location means for locating a selection object that corresponds to a menu item selectable by a user and is to be selected by the user, at a predetermined position in the virtual space that is based on a position of the specific object;

object image generation means for taking an image of the virtual space with the virtual camera and generating an object image of the selection object; and

display control means for displaying the object image on the display device such that the object image is superimposed on the real space on the screen and viewed by the user.

12. A display control system connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, the display control system comprising:

taken image obtaining means for obtaining a taken image obtained by using the imaging device;

detection means for detecting a specific object from the taken image;

calculation means for calculating a relative position of the imaging device and the specific object on the basis of a detection result of the specific object by the detection means;

virtual camera setting means for setting a virtual camera in a virtual space on the basis of a calculation result by the calculation means;

object location means for locating a selection object that corresponds to a menu item selectable by a user and is to be selected by the user, at a predetermined position in the virtual space that is based on a position of the specific object;

object image generation means for taking an image of the virtual space with the virtual camera and generating an object image of the selection object; and

display control means for displaying the object image on the display device such that the object image is superimposed on the real space on the screen and viewed by the user.

13. A display control method for taking an image of a real world by using an imaging device and displaying an image of a virtual object in a virtual space by using a display device that allows a real space to be viewed on a screen thereof, the display control method comprising:

a taken image obtaining step of obtaining a taken image obtained by using the imaging device;

a detection step of detecting a specific object from the taken image;

a calculation step of calculating a relative position of the imaging device and the specific object on the basis of a detection result of the specific object at the detection step;

a virtual camera setting step of setting a virtual camera in a virtual space on the basis of a calculation result by the calculation step;

an object location step of locating a selection object that corresponds to a menu item selectable by a user and is to be selected by the user, as the virtual object at a predetermined position in the virtual space that is based on a position of the specific object;

an object image generation step of taking an image of the virtual space with the virtual camera and generating an object image of the selection object; and

a display control step of displaying the object image on the display device such that the object image is superimposed on the real space on the screen and viewed by the user.

14. A display control system comprising a marker and a display control apparatus connected to an imaging device and a display device that allows a real space to be viewed on a screen thereof, the display control apparatus comprising:

taken image obtaining means for obtaining a taken image obtained by using the imaging device;

detection means for detecting the marker from the taken image;

calculation means for calculating a relative position of the imaging device and the marker on the basis of a detection result of the marker by the detection means;

virtual camera setting means for setting a virtual camera in a virtual space on the basis of a calculation result by the calculation means;

object location means for locating a selection object that corresponds to a menu item selectable by a user and is to be selected by the user, at a predetermined position in the virtual space that is based on a position of the marker;

object image generation means for taking an image of the virtual space with the virtual camera and generating an object image of the selection object; and

display control means for displaying the object image on the display device such that the object image is superimposed on the real space on the screen and viewed by the user.