Computer-readable storage medium, image display apparatus, image display system, and image display method
First Claim
1. A non-transitory computer-readable storage medium having stored therein an image display program which is configured to stereoscopically display a real world image onto which a 3-dimensional virtual object is superimposed on a screen of a stereoscopic display apparatus configured to provide a stereoscopic view to a user, the stereoscopic display apparatus including a computing system that includes at least one processor circuit coupled to a memory device, the memory device configured to store real world image data output by a left camera and a right camera, the image display program comprising instructions that are, when executed by a computing system, configured to:
- recognize a predetermined shooting target from the stored real world image data output by the left camera and the right camera;
calculate multiple first position orientation information, which are each based on separate relate world image data acquired over a period of time, indicating relative positions and relative orientations of the right camera and the predetermined shooting target with respect to each other;
calculate multiple second position orientation information, which are each based on separate relate world image data acquired over the period of time, indicating relative positions and relative orientations of the left camera and the predetermined shooting target with respect to each other;
determine how much the calculated multiple first position orientation information and the calculated multiple second position orientation information vary;
as a result of determination that the calculated multiple first position orientation information and the calculated multiple second position orientation information vary less than the predetermined amount, calculate pose data, which is indicative of a single position and a single orientation in a virtual space, as a function of multiple calculated first position orientation information and multiple calculated second position orientation information;
calculate a midpoint, as the single position, between the relative position of the right camera and the relative position of the left camera;
calculate a position and an orientation of a right virtual camera for generating a right virtual space image of the virtual space based on the calculated pose data;
calculate a position and an orientation of a left virtual camera for generating a left virtual space image in the virtual space based on the calculated pose data;
generate the right virtual space image as viewed from the right virtual camera in accordance with the calculated position and the calculated orientation of the right virtual camera;
generate the left virtual space image as viewed from the left virtual camera in accordance with the calculated position and the calculated orientation of the left virtual camera;
superimpose the right virtual space image onto the real world image data outputted from the right camera;
superimpose the left virtual space image onto the real world image data outputted from the left camera; and
output the superimposed images to the stereoscopic display apparatus for a stereoscopic view,wherein the positions of the right virtual camera and the left virtual camera are symmetrically set about the calculated midpoint.
1 Assignment
0 Petitions
Accused Products
Abstract
First, the interval between a left virtual camera and a right virtual camera is determined based on the relative position and the relative orientation of an outer imaging section (left) with respect to a marker, which are calculated from a result of marker recognition in the left real world image, and on the relative position and the relative orientation of an outer imaging section (right) with respect to the marker, which are calculated from a result of marker recognition in the right real world image. After the interval between the left virtual camera and the right virtual camera is determined, the positions and the orientations of the left virtual camera and the right virtual camera are determined based on the determined interval such that the relationship between the position and the orientation of the left virtual camera, and the position and the orientation of the right virtual camera is ideal.
134 Citations
17 Claims
-
1. A non-transitory computer-readable storage medium having stored therein an image display program which is configured to stereoscopically display a real world image onto which a 3-dimensional virtual object is superimposed on a screen of a stereoscopic display apparatus configured to provide a stereoscopic view to a user, the stereoscopic display apparatus including a computing system that includes at least one processor circuit coupled to a memory device, the memory device configured to store real world image data output by a left camera and a right camera, the image display program comprising instructions that are, when executed by a computing system, configured to:
-
recognize a predetermined shooting target from the stored real world image data output by the left camera and the right camera; calculate multiple first position orientation information, which are each based on separate relate world image data acquired over a period of time, indicating relative positions and relative orientations of the right camera and the predetermined shooting target with respect to each other; calculate multiple second position orientation information, which are each based on separate relate world image data acquired over the period of time, indicating relative positions and relative orientations of the left camera and the predetermined shooting target with respect to each other; determine how much the calculated multiple first position orientation information and the calculated multiple second position orientation information vary; as a result of determination that the calculated multiple first position orientation information and the calculated multiple second position orientation information vary less than the predetermined amount, calculate pose data, which is indicative of a single position and a single orientation in a virtual space, as a function of multiple calculated first position orientation information and multiple calculated second position orientation information; calculate a midpoint, as the single position, between the relative position of the right camera and the relative position of the left camera; calculate a position and an orientation of a right virtual camera for generating a right virtual space image of the virtual space based on the calculated pose data; calculate a position and an orientation of a left virtual camera for generating a left virtual space image in the virtual space based on the calculated pose data; generate the right virtual space image as viewed from the right virtual camera in accordance with the calculated position and the calculated orientation of the right virtual camera; generate the left virtual space image as viewed from the left virtual camera in accordance with the calculated position and the calculated orientation of the left virtual camera; superimpose the right virtual space image onto the real world image data outputted from the right camera; superimpose the left virtual space image onto the real world image data outputted from the left camera; and output the superimposed images to the stereoscopic display apparatus for a stereoscopic view, wherein the positions of the right virtual camera and the left virtual camera are symmetrically set about the calculated midpoint. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. An image display apparatus which stereoscopically displays a real world image onto which a 3-dimensional virtual object is superimposed, the image display apparatus comprising:
-
a memory device configured to store real world image data output by a left camera and a right camera; and a processing system that includes at least one processor circuit coupled to the memory device, the processing system configured to; recognize a target object from the stored respective real world image data output by the left camera and the right camera; calculate multiple first relative information, which are each based on separate relative real world image data acquired from the right camera over a period of time, indicating relative positions and relative orientations of the right camera and the target object with respect to each other; calculate multiple second relative information, which are each based on separate relative real world image data acquired from the left camera over a period of time, indicating relative positions and relative orientations of the left camera and the target object with respect to each other; determine how much the calculated multiple first relative information and the calculated multiple second relative information vary; as a result of determination that the calculated multiple first relative information and the calculated multiple second relative information vary less than the predetermined amount, calculate pose data, which indicates a single position and a single orientation in a virtual space, as a function of the multiple first relative information and the multiple second relative information; calculate a midpoint, as the single position, between the relative position of the right camera and the relative position of the left camera; set a position and an orientation of a right virtual camera in the virtual space as a function of the calculated pose data, which is indicative of the single position and the single orientation; set a position and an orientation of a left virtual camera in the virtual space as a function of the pose data, which is indicative of the single position and the single orientation; generate a right virtual space image of a virtual space as viewed from the right virtual camera at the set position and orientation thereof; generate a left virtual space image of the virtual space as viewed from the left virtual camera at the set position and orientation thereof; superimpose the right virtual space image with the real world image data output by the right camera; superimpose the left virtual space image with the real world image data output by the left camera; and output the superimposed images for a stereoscopic view to a stereoscopic display apparatus, wherein the positions of the right virtual camera and the left virtual camera are symmetrically set about the calculated midpoint. - View Dependent Claims (11, 12, 13)
-
-
14. An image display system, the image display system comprising:
-
a left camera configured to output left real world image data; a right camera configured to output right real world image data; a predetermined shooting target; a memory device configured to store the left and right real world image data; a processing system that includes at least one processor circuit coupled to the memory device, the processing system configured to; recognize the predetermined shooting target in each of the left and right real world image data stored in the memory device; calculate multiple right position orientation information, which are each based on separate relative world image data acquired from the right camera over a period of time, that indicates relative positions and relative orientations of the right camera and the predetermined shooting target with respect to each other; calculate multiple left position orientation information, which are each based on separate relative world image data acquired from the left camera over a period of time, that indicates relative positions and relative orientations of the left camera and the predetermined shooting target with respect to each other; determine how much the calculated multiple left position orientation information and the calculated multiple right position orientation information vary; as a result of determination that the calculated multiple left position orientation information and the calculated multiple right position orientation information vary less than the predetermined amount, calculate combined position orientation data, which is indicative of a single position and a single orientation in a three-dimensional virtual space, as a function of the left and right position orientation information; calculate a midpoint, as the single position, between the relative position of the right camera and the relative position of the left camera; set the position and the orientation of a right virtual camera in the three-dimensional virtual space based on the calculated combined position orientation data; set the position and the orientation of a left virtual camera in the three-dimensional virtual space based on the calculated combined position orientation data; generate a right virtual space image of a virtual space as viewed from the set right virtual camera; and generate a left virtual space image of a virtual space as viewed from the set left virtual camera; and a stereoscopic display apparatus configured to display a first superimposed image and a second superimposed image as a stereoscopic image, where the first superimposed image is based on the right virtual space image and right real world image data and the second superimposed image is based on the left virtual space image and the left real world image data, wherein the positions of the right virtual camera and the left virtual camera are symmetrically set about the calculated midpoint. - View Dependent Claims (15)
-
-
16. An image display method for stereoscopically displaying a real world image onto which a 3-dimensional virtual object is superimposed, on a screen of a stereoscopic display apparatus capable of providing a stereoscopic view, the stereoscopic display apparatus including a computing system with at least one processor circuit coupled to a memory device, the memory device configured to store left and right real world image data respectively output by a left camera and a right camera, the image display method comprising:
-
recognizing a predetermined shooting target in the stored right real world image data and the stored left real world image data; calculating multiple first poses, which are each based on separate right real world image data acquired over a period of time, which indicates a relative position and a relative orientation of the right camera and the predetermined shooting target with respect to each other; calculating multiple second poses, which are each based on separate left real world image data acquired over a period of time, which indicates a relative position and a relative orientation of the left camera and the predetermined shooting target with respect to each other; determine how much the calculated multiple first poses and the calculated second poses information vary; as a result of determination that the calculated multiple first poses and calculated multiple second poses vary less than the predetermined amount, calculating one pose as a function of the multiple first poses and the multiple second poses, the calculated one pose indicating a single position and a single orientation in a virtual space; calculating a midpoint, as the single position, between the relative position of the right camera and the relative position of the left camera; determining the pose of a right virtual camera as a function of the single position and the single orientation indicated by the calculated one pose; determining the pose of a left virtual camera as a function of the single position and the single orientation indicated by the calculated one pose; generating a right virtual space image indicating the predetermined virtual space as it is looked at from the right virtual camera that is set at the determined pose of the right virtual camera; generating a left virtual space image indicating the predetermined virtual space as it is looked at from the left virtual camera that is set at the determined pose of the left virtual camera; and superimposing the right virtual space image onto the right real world image, and superimposing the left virtual space image onto the left real world image, to output images for a stereoscopic view to the stereoscopic display apparatus, wherein positions of the right virtual camera and the left virtual camera are symmetrically set about the calculated midpoint. - View Dependent Claims (17)
-
Specification