Image synthesis and display apparatus and simulation system using same
First Claim
1. An image synthesis and display apparatus comprising:
- image pickup means for picking up an image of a range of an actual space corresponding to a field of view presented to a user by a display;
extraction means for extracting an image from the image picked up by said pickup means, the extracted image representing a specified object;
synthesis means for inserting the image extracted by said extraction means into a simulation image of an exclusively virtual space, and for providing the simulation image with the inserted image to the display for displaying to the user;
a position sensor which measures a position and light axis direction of said image pickup means in a real space; and
a simulation image forming means which forms a simulated image which represents an object in the virtual space, based on a position and light axis direction in the virtual space corresponding to the position and light axis direction measured by said position sensor, wherein said simulation image forming means comprisesmovement information forming means for forming virtual movement information related to movement of a virtual position of the user in the virtual space,coordinate transforming means for transforming the position and light axis direction measured by said position sensor to a position and light axis direction in the virtual space, based on said virtual movement information, andcomputer graphic image generating means for generating computer graphics having a projection reference point and normal direction to a picture plane, in accordance with the position and light axis direction in the virtual space.
0 Assignments
0 Petitions
Accused Products
Abstract
An image synthesis and display apparatus. The apparatus includes a display device, an image pickup device, an extraction unit and a synthesis unit. The display device displays a simulation image to a user, and provides a field of view of the simulation image to the user. The image pickup device detects an image of a range of an actual space corresponding to the field of view provided to the user by the display device. The extraction unit extracts an image of an object from the image detected by the image pickup device. The synthesis unit inserts the image extracted by the extraction unit into a simulation of a virtual space, to produce the simulation image displayed by the display device. The image pickup device is aligned with the display device, and the image pickup device and the display device move with the user while maintaining the alignment between the image pickup device and the display device.
-
Citations
13 Claims
-
1. An image synthesis and display apparatus comprising:
-
image pickup means for picking up an image of a range of an actual space corresponding to a field of view presented to a user by a display; extraction means for extracting an image from the image picked up by said pickup means, the extracted image representing a specified object; synthesis means for inserting the image extracted by said extraction means into a simulation image of an exclusively virtual space, and for providing the simulation image with the inserted image to the display for displaying to the user; a position sensor which measures a position and light axis direction of said image pickup means in a real space; and a simulation image forming means which forms a simulated image which represents an object in the virtual space, based on a position and light axis direction in the virtual space corresponding to the position and light axis direction measured by said position sensor, wherein said simulation image forming means comprises movement information forming means for forming virtual movement information related to movement of a virtual position of the user in the virtual space, coordinate transforming means for transforming the position and light axis direction measured by said position sensor to a position and light axis direction in the virtual space, based on said virtual movement information, and computer graphic image generating means for generating computer graphics having a projection reference point and normal direction to a picture plane, in accordance with the position and light axis direction in the virtual space. - View Dependent Claims (2, 3, 4)
-
-
5. An image synthesis and display apparatus comprising:
-
image pickup means for picking up an image of a range of an actual space corresponding to a field of view presented to a user by a display; extraction means for extracting an image from the image picked up by said pickup means, the extracted image representing a specified object; synthesis means for inserting the image extracted by said extraction means into a simulation image of an exclusively virtual space, and for providing the simulation image with the inserted image to the display for displaying to the user; a position sensor which measures a position and light axis direction of said image pickup means in a real space; and a simulation image forming means which forms a simulated image which represents an object in the virtual space, based on a position and light axis direction in the virtual space corresponding to the position and light axis direction measured by said position sensor, wherein said display comprises first and second display devices which independently display images to the left eye and the right eye, respectively, of the user, said image pickup means comprises first and second image pickup devices which pick up a range of a real space corresponding to the field of view presented to the user, said extraction means being configured so as to perform independent extraction processing with respect to the individual images obtained by said first and second image pickup devices, said simulation image forming means forms simulated images corresponding to the left eye and the right eye of the user, respectively, based on position and view orientation of the left eye and right eye, respectively, in the virtual space, in accordance with the position and light axis direction measured by said position sensor, and said synthesis means synthesizes simulation images corresponding to the left eye and right eye, respectively, with a result of extraction with respect to images obtained by said first and second image pickup devices, synthesis being performed independently for the left and right. - View Dependent Claims (6, 7, 8)
-
-
9. An apparatus comprising:
-
a display device which displays a simulation image to a user, the display device providing a field of view of the simulation image to the user; an image pickup device which detects an image of a range of an actual space corresponding to the field of view provided to the user by the display device, the detected image including an image of an object; an extraction unit which extracts the image of the object from the image detected by the image pickup device; a synthesis unit which inserts the image extracted by the extraction unit into a simulation of a virtual space, to produce the simulation image displayed by the display device; a position sensor which measures a position and a light axis direction in real space of the image pickup device; and a simulation image forming unit which forms a simulated image representing an object in the virtual space, based on a position and a light axis direction in the virtual space corresponding to the position and light axis direction measured by the position sensor, wherein the display device comprises first and second display devices which independently display images to the left eye and the right eye, respectively, of the user, the image pickup device comprises first and second image pickup devices which detect first and second images, respectively, the first and second images each being an image of a range of an actual space corresponding to the field of view presented to the user, the first and second images each including an image of the object, the extraction unit extracts the image of the object from the first and second images of the first and second image pickup devices, the simulation image forming unit forms simulated images corresponding to the left eye and the right eye of the user, respectively, based on position and view orientation of the left eye and right eye, respectively, in the virtual space, in accordance with the position and light axis direction measured by the position sensor, and the synthesis unit inserts the image of the object extracted by the extraction unit from the first image into a simulation image corresponding to the left eye of the user, and inserts the image of the object extracted by the extraction unit from the second image into a simulation image corresponding to the right eye of the user, to produce the simulation image displayed to the user by the display device. - View Dependent Claims (10, 11, 12, 13)
-
Specification