SYSTEM AND METHOD FOR GENERATING VIRTUAL IMAGES ACCORDING TO POSITION OF VIEWERS

• US 20080111832A1
• Filed: 10/23/2007
• Published: 05/15/2008
• Est. Priority Date: 10/23/2006
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kanevsky, Dimitri, Zlatsin, Alexander, Emam, Ossama
• Granted Patent: US 7,995,076 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image

Abstract

A method generates and superposes virtual images to an existing image in order to show special effects. Images are generated depending on the relative position of one or more subjects viewing the existing image. The images are generated in order to see an augmented reality. The subjects are people located near a transparent support such as a screen or a window. These people watch the augmented reality through the screen/window. The virtual objects on the screen are created in such a way as to give a realistic impression for different people watching the same screen/window at the same time, thereby generating virtual images in order to create an augmented reality for one or many viewers watching the same real objects at the same time.

Claims

1. A method for generating virtual images in order to create an augmented reality for one or a plurality of viewers watching a real background at the same time, said method comprising the steps of:

capturing images of a real background comprising one or a plurality of real objects;

recognizing one or more of said real objects in said captured images;

identifying possible virtual objects related to recognized real objects and not present in said captured images;

automatically interpreting the captured images;

generating virtual images of virtual objects among identified possible virtual objects based on interpreted captured images;

positioning each virtual object relatively to the real objects;

adapting the virtual images depending on the position of said virtual objects;

determining a plurality of angles of view to watch the background;

adapting said virtual images for each angle of view, in order to superimpose said virtual images on the real background; and

displaying said virtual images so that said virtual objects are seen differently depending on the angle of view relatively to the real background.

2. The method according to claim 1 wherein the step of capturing images, comprises the further step of determining and associating a geographical position with said captured images.

3. The method according to claim 1 wherein the step of generating virtual images of virtual objects among identified possible virtual objects based on interpreted captures images, comprises the step of retrieving pre-stored virtual images of virtual objects.

4. The method according to claim 1 wherein the step of determining a plurality of angles of view to watch the real background, comprises the further step of determining directions in which viewers are watching by detecting the direction of eyes or head of each viewer watching the real background.

5. The method according to claim 1 wherein the step of displaying said virtual images so that said virtual objects are seen differently depending on the angle of view relatively to the real background, comprises the step of displaying said virtual images so that said virtual objects are seen differently by each viewer depending on the direction in which said viewer is watching.

6. The method according to claim 5 wherein the step of displaying said virtual images so that said virtual objects are seen differently by each viewer depending on the direction in which said viewer is watching, comprises the further step of displaying said virtual images on a support so that the virtual objects are seen by each viewer superimposed on the real the background visible through said support whatever the angle of view.

7. The method according to claim 5 wherein the step of displaying said virtual images so that said virtual objects are seen differently by each viewer depending on the direction in which said viewer is watching, comprises the further step of displaying to each viewer said virtual images on a head mount display, wherein said virtual images are seen superimposed on the real background.

8. The method according to claim 5 wherein the step of displaying said virtual images so that said virtual objects are seen differently by each viewer depending on the direction in which said viewer is watching, comprises the further step of displaying on a support at the same time different virtual images of the same virtual objects, said support being such that:

different virtual images can be seen independently according to the angle of view of the viewer in order to produce a three dimensional view of the virtual objects, and

virtual images are seen superimposed on the real background visible through said support so that virtual objects fit with the real background.

9. The method according to claim 1 wherein the step of generating virtual images of virtual objects among identified possible virtual objects based on interpreted captured images, comprises the further steps of:

receiving audio data;

automatically transcribing said audio data into text;

automatically interpreting said text; and

generating virtual images of virtual objects among identified possible virtual objects based on interpreted captured images and interpreted text.

10. The method according to claim 1 wherein the step of displaying said virtual images so that said virtual images are seen differently by each viewer depending on the direction to which said viewer is watching further comprises:

generating a text related to the real objects and/or virtual objects;

generating a speech corresponding to the generated text by means of a text-to-speech synthesizer;

synchronizing the generated virtual images and the speech; and

displaying virtual images with the speech.

11. The method according to claim 1 wherein the step of automatically interpreting the captured images comprises the step of automatically interpreting the captured images based on recognized real objects and/or a geographical position associated with said captured images.

12. A system for generating virtual images in order to create an augmented reality for one or a plurality of viewers watching a real background at the same time, comprising:

image capture apparatus for capturing images of a real background comprising one or a plurality of real objects;

object recognition apparatus for recognizing one or more of said real objects in said captured images;

a computer processing component for identifying possible virtual objects related to recognized real objects and not present in said captured images, for automatically interpreting the captured images and for generating virtual images of virtual objects among identified possible virtual objects based on interpreted captured images, for positioning each virtual object relatively to the real objects, for adapting the virtual images depending on the position of said virtual objects, for determining a plurality of angles of view to watch the background, and for adapting said virtual images for each angle of view, in order to superimpose said virtual images on the real background; and

a display component for displaying said virtual images so that said virtual objects are seen differently depending on the angle of view relatively to the real background.

13. A computer readable medium storing a program comprising instructions for carrying out a method for generating virtual images in order to create an augmented reality for one or a plurality of viewers watching a real background at the same time, wherein said method comprising the steps of:

capturing images of a real background comprising one or a plurality of real objects;

recognizing one or more of said real objects in said captured images;

identifying possible virtual objects related to recognized real objects and not present in said captured images;

automatically interpreting the captured images;

generating virtual images of virtual objects among identified possible virtual objects based on interpreted captured images;

positioning each virtual object relatively to the real objects;

adapting the virtual images depending on the position of said virtual objects;

determining a plurality of angles of view to watch the background;

adapting said virtual images for each angle of view, in order to superimpose said virtual images on the real background; and

displaying said virtual images so that said virtual objects are seen differently depending on the angle of view relatively to the real background.