IMAGE DISPLAY SYSTEM, IMAGE DISPLAY APPARATUS, AND IMAGE DISPLAY METHOD

• US 20130201293A1
• Filed: 01/22/2013
• Published: 08/08/2013
• Est. Priority Date: 06/02/2010
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: KONNO, Hideki, UMEZU, Ryuji, SUGINO, Kenichi, KAWAMOTO, Kouichi, KUBO, Kenta
• Granted Patent: US 9,282,319 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/356 having separate monoscopic and stereoscopic modes

Abstract

A hand-held image display apparatus includes a touch panel, a stereoscopic display apparatus, a first imaging section, and a second imaging section. The hand-held image display apparatus detects a marker from a real world image shot by at least one of the first imaging section and the second imaging section, and determines the relative positions of a virtual object and a pair of virtual cameras with respect to each other in a virtual space, based on the result of the detection. Then, the hand-held image display apparatus superimposes virtual space images drawn based on the first virtual camera and the second virtual camera, onto the real world images shot by the first imaging section and the second imaging section, and displays images that are stereoscopically visible by naked eyes, on the stereoscopic display apparatus.

Claims

1. An image display system for superimposing a virtual space image onto a real world image and displaying the resultant composite image on an image display apparatus that is of a hand-held type,

the image display apparatus including:

a housing;

a stereoscopic display section which is provided on a front surface of the housing and is capable of providing an image that is stereoscopically visible by naked eyes; and

a first imaging section and a second imaging section provided on a back surface of the housing, and

the image display system comprising;

a marker detection section which automatically detects a marker placed in a real world from a real world image shot by at least one of the first imaging section and the second imaging section;

a virtual camera control section which determines and updates positions and orientations of a pair of virtual cameras including a first virtual camera and a second virtual camera in a virtual space, based on the marker in the real world image detected by the marker detection section;

a first virtual space image generation section which generates a first virtual space image by drawing the virtual space, based on the first virtual camera;

a second virtual space image generation section which generates a second virtual space image by drawing the virtual space, based on the second virtual camera; and

a display control section which displays an image that is stereoscopically visible by naked eyes on the stereoscopic display section, based on a first composite image obtained by superimposing the first virtual apace image onto a first real world image shot by the first imaging section, and on a second composite image obtained by superimposing the second virtual space image onto a second real world image shot by the second imaging section.

2. The image display system according to claim 1, wherein the first imaging section and the second imaging section are arranged, on the back surface of the housing, along a left-right direction of the housing so as to be symmetrical with respect to the middle of the screen of the stereoscopic display section, the first imaging section and the second imaging section being integrated with the housing.

3. The image display system according to claim 1, wherein the virtual camera control section determines and updates the positions and orientations of the pair of virtual cameras, based on the marker in the real world image detected by the marker detection section, without using coordinates in the real world that indicate the position of the marker.

4. The image display system according to claim 1, wherein the image display apparatus further includes a switch for switching display between stereoscopic display and planar display, the switch being provided at end portions on the front surface of the housing, and

the display control section switches display between the stereoscopic display in which an image is displayed in a stereoscopically visible manner, and the planar display in which an image is displayed in a planar manner, in accordance with a state of the switch.

5. The image display system according to claim 1, wherein a third imaging section is provided on the front surface of the housing and on a line vertically dividing in half the screen of the stereoscopic display section.

6. The image display system according to claim 1, wherein

a pair of speakers are provided in the housing, and

a pair of speaker holes corresponding to the pair of speakers are provided on the front surface of the housing such that the screen of the stereoscopic display section is sandwiched between the pair of speaker holes in the left-right direction.

7. The image display system according to claim 1, wherein the image display apparatus is a portable game apparatus.

8. An image display apparatus that is of a hand-held type and displays thereon a composite image obtained by superimposing a virtual space image onto a real world image, the image display apparatus comprising:

a housing;

a stereoscopic display section which is provided on a front surface of the housing and is capable of providing an image that is stereoscopically visible by naked eyes;

a first imaging section and a second imaging section provided on a back surface of the housing;

a marker detection section which automatically detects a marker placed in a real world from a real world image shot by at least one of the first imaging section and the second imaging section;

a virtual camera control section which determines and updates positions and orientations of a pair of virtual cameras including a first virtual camera and a second virtual camera in a virtual space, based on the marker in the real world image detected by the marker detection section;

a first virtual space image generation section which generates a first virtual space image by drawing the virtual space, based on the first virtual camera;

a second virtual space image generation section which generates a second virtual space image by drawing the virtual space, based on the second virtual camera; and

a display control section which displays an image that is stereoscopically visible by naked eyes on the stereoscopic display section, based on a first composite image obtained by superimposing the first virtual space image onto a first real world image shot by the first imaging section, and on a second composite image obtained by superimposing the second virtual space image onto a second real world image shot by the second imaging section.

9. A non-transitory computer-readable non-transitory storage medium having stored therein an image display program which causes a computer included in an image display system for superimposing a virtual space image onto a real world image and displaying the resultant composite image on an image display apparatus that is of a hand-held type, to execute:

automatically detecting a marker placed in a real world from a real world image shot by at least one of a first imaging section and a second imaging section provided on a back surface of a housing of the image display apparatus;

determining and updating positions and orientations of a pair of virtual cameras including a first virtual camera and a second virtual camera in a virtual space, based on the detected marker in the real world image;

generating a first virtual space image by drawing the virtual space, based on the first virtual camera;

generating a second virtual space image by drawing the virtual space, based on the second virtual camera; and

displaying an image that is stereoscopically visible by naked eyes on a stereoscopic display section which is provided on a front surface of the housing of the image display apparatus and is capable of providing an image that is stereoscopically visible by naked eyes, based on a first composite image obtained by superimposing the first virtual space image onto a first real world image shot by the first imaging section, and on a second composite image obtained by superimposing the second virtual space image onto a second real world image shot by the second imaging section.

10. An image display method for superimposing a virtual space image onto a real world image and displaying the resultant composite image on an image display apparatus that is of a hand-held type, the image display method comprising:

automatically detecting a marker placed in a real world from a real world image shot by at least one of a first imaging section and a second imaging section provided on a back surface of a housing of the image display apparatus;

determining and updating positions and orientations of a pair of virtual cameras including a first virtual camera and a second virtual camera in a virtual space, based on the detected marker in the real world image;

generating a first virtual space image by drawing the virtual space, based on the first virtual camera;

generating a second virtual space image by drawing the virtual space, based on the second virtual camera; and

displaying an image that is stereoscopically visible by naked eyes on a stereoscopic display section which is provided on a front surface of the housing of the image display apparatus and is capable of providing an image that is stereoscopically visible by naked eyes, based on a first composite image obtained by superimposing the first virtual space image onto a first real world image shot by the first imaging section, and on a second composite image obtained by superimposing the second virtual space image onto a second real world image shot by the second imaging section.