STORAGE MEDIUM HAVING STORED THEREIN STEREOSCOPIC IMAGE DISPLAY PROGRAM, STEREOSCOPIC IMAGE DISPLAY DEVICE, STEREOSCOPIC IMAGE DISPLAY SYSTEM, AND STEREOSCOPIC IMAGE DISPLAY METHOD

• US 20110304711A1
• Filed: 06/07/2011
• Published: 12/15/2011
• Est. Priority Date: 06/14/2010
• Status: Active Grant

• Current Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Original Assignee: HAL Laboratory, Inc., Nintendo Company Limited
• Inventors: Nishimura, Yuki, Ito, Yuichiro
• Granted Patent: US 8,902,298 B2
• Time in Patent Office: Days
• US Class Current: 348/51
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/5252 using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game
• CPC Class Codes: A63F 13/5255 according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/655 by importing photos, e.g. of the player
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/30 characterized by output arrangements for receiving control signals generated by the game device
• CPC Class Codes: A63F 2300/301 using an additional display connected to the game console, e.g. on the controller
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/6676 by dedicated player input
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/695 Imported photos, e.g. of the player
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 : Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/302 : for viewing without the aid of special glasses, i.e. using autostereoscopic displays

Abstract

A game device 10 sets a reference distance for a distance from a real camera in an image-capturing direction, and displays a right captured image and a left captured image in a display area so that a reference point, which is a point that is away from the real camera by the reference distance, is displayed at a center in the display area. Thus, where an image produced by using an augmented reality technique is stereoscopically displayed, it is possible to display a virtual image in a manner easy to view.

Claims

1. A computer-readable storage medium having stored therein an image display program for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display member by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, wherein the image display program instructs a computer to function as:

a reference distance setting section for setting a reference distance for a distance from the real camera in an image-capturing direction; and

a stereoscopic display control section for displaying the right captured image in the display area so that a reference point in the right captured image is displayed at a center of the display area and displaying the left captured image in the display area so that a reference point in the left captured image is displayed at the center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the image-capturing direction.

2. The storage medium according to claim 1, wherein the stereoscopic display control section includes:

a display area determination member for determining a right display area which is an area of the right captured image to be displayed in the display area and which is centered about a point corresponding to the reference point in the right captured image, and determining a left display area which is an area of the left captured image to be displayed in the display area and which is centered about a point corresponding to the reference point in the left captured image; and

a display control member for displaying the right display area of the right captured image and the left display area of the left captured image in the display area based on the areas determined by the display area determination member.

3. The storage medium according to claim 1, wherein the reference distance setting section fixedly sets the reference distance.

4. The storage medium according to claim 1, wherein:

the image display program instructs the computer to further function as a distance calculation section for calculating a distance to a predetermined image-capture object; and

the reference distance setting section sets the distance calculated by the distance calculation section as the reference distance.

5. The storage medium according to claim 4, wherein the reference distance setting section sets the reference distance using the distance calculated by the distance calculation section as a reference.

6. The storage medium according to claim 5, wherein the reference distance setting section sets, as the reference distance, a value obtained by adding or subtracting a predetermined value to/from the distance calculated by the distance calculation section.

7. The storage medium according to claim 4, wherein using at least one of the right captured image and the left captured image, the distance calculation section recognizes the predetermined image-capture object in the captured image, thereby calculating a distance between the predetermined image-capture object and the real camera.

8. The storage medium according to claim 4, wherein using at least one of the right captured image and the left captured image, the distance calculation section recognizes a position and an orientation of the predetermined image-capture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined image-capture object and the real camera, and to calculate a distance between the predetermined image-capture object and the real camera based on the position/orientation information.

9. The storage medium according to claim 4, wherein using both of the right captured image and the left captured image, the distance calculation section recognizes a position and an orientation of the predetermined image-capture object in the captured image so as to calculate position/orientation information representing a relative position and orientation relationship between the predetermined image-capture object and the real camera, thereby calculating a distance between the predetermined image-capture object and the real camera based on the position/orientation information.

10. The storage medium according to claim 1, wherein the image display program instructs the computer to further function as:

a virtual camera setting section for setting a right-eye virtual camera and a left-eye virtual camera in a virtual space; and

a virtual image stereoscopic display control section for stereoscopically displaying a virtual image by displaying in the display area a right virtual image obtained by capturing an image of the virtual space with the right-eye virtual camera and a left virtual image obtained by capturing an image of the virtual space with the left-eye virtual camera,

wherein the virtual camera setting section sets a projection matrix of the right virtual camera so that a virtual reference point in the right virtual image is displayed at the center of the display area and sets a projection matrix of the left virtual camera so that a virtual reference point in the left virtual image is displayed at the center of the display area, wherein the virtual reference point is a point that is away from the virtual cameras by a distance corresponding to the reference distance in an image-capturing direction of the two virtual cameras on a straight line passing through a median point between the two virtual cameras and extending in the image-capturing direction.

11. The storage medium according to claim 1, wherein the stereoscopic display control section displays the right captured image and the left captured image in the display area so that the reference point is displayed at the center of the display area by using an angle of view of the real camera and a distance between the real cameras.

12. An image display device for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display member by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, comprising:

a reference distance setting section for setting a reference distance for a distance from the real camera in an image-capturing direction; and

a stereoscopic display control section for displaying the right captured image in the display area so that a reference point in the right captured image is displayed at a center of the display area and displaying the left captured image in the display area so that a reference point in the left captured image is displayed at the center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the image-capturing direction.

13. An image display system for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display member by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, comprising:

a reference distance setting section for setting a reference distance for a distance from the real camera in an image-capturing direction; and

a stereoscopic display control section for displaying the right captured image in the display area so that a reference point in the right captured image is displayed at a center of the display area and displaying the left captured image in the display area so that a reference point in the left captured image is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the image-capturing direction.

14. The image display system according to claim 13, further comprising:

a predetermined image-capture object; and

a distance calculation section for calculating a distance to the predetermined image-capture object,

wherein the reference distance setting section sets the distance calculated by the distance calculation section as the reference distance.

15. An image display method to be executed by a computer for stereoscopically displaying a captured image on a stereoscopically-viewable stereoscopic display member by displaying, in substantially a same display area, a right-eye image and a left-eye image by using a right captured image outputted from a right-eye real camera and a left captured image outputted from a left-eye real camera, comprising:

a reference distance setting step of setting a reference distance for a distance from the real camera in an image-capturing direction; and

a stereoscopic display control step of displaying the right captured image in the display area so that a reference point in the right captured image is displayed at a center of the display area and displaying the left captured image in the display area so that a reference point in the left captured image is displayed at a center of the display area, wherein the reference point is a point that is away from the real cameras by the reference distance in the image-capturing direction of the two real cameras on a straight line passing through a median point between the two real cameras and extending in the image-capturing direction.