Real-Time 3-D Interactions Between Real And Virtual Environments

• US 20100253700A1
• Filed: 04/01/2010
• Published: 10/07/2010
• Est. Priority Date: 04/02/2009
• Status: Abandoned Application

• Current Assignee: Philippe Bergeron
• Original Assignee: Philippe Bergeron
• Inventors: Bergeron, Philippe
• Application Number:
  US12/752,822
• Publication Number:
  US 20100253700A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63J 5/02 Arrangements for making stage effects; Auxiliary stage appliances
• CPC Class Codes: A63J 5/021 Mixing live action with images projected on translucent screens
• CPC Class Codes: G02B 30/23 using wavelength separation, e.g. using anaglyph techniques
• CPC Class Codes: G02B 30/56 by projecting aerial or floating images
• CPC Class Codes: G03B 35/00 Stereoscopic photography
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Systems and methods providing for real and virtual object interactions are presented. Images of virtual objects can be projected onto the real environment, now augmented. Images of virtual objects can also be projected to an off-stage invisible area, where the virtual objects can be perceived as holograms through a semi-reflective surface. A viewer can observe the reflected images while also viewing the augmented environment behind the pane, resulting in one perceived uniform world, all sharing the same Cartesian coordinates. One or more computer-based image processing systems can control the projected images so they appear to interact with the real-world object from the perspective of the viewer.

Claims

1. A system for allowing a real-world object to interact with a virtual object, the system comprising:

a first projection system configured to project a digital image on a real-world setting; and

an image processing computer configured to control the first projection system and to mask a first element of the setting while projecting image data on a second unmasked element of the setting.

2. The system of claim 1, further comprising a data acquisition sensor array configured to acquired object data regarding position information of a real-world object within the real-world setting.

3. The system of claim 2, wherein the sensor array comprises two or more sensors.

4. The system of claim 2, wherein the image processing computer is further configured to use the object data to determine expected position of the real-world object as a function of time.

5. The system of claim 4, further comprising a second projection system configured to project a second image under control of the image processing computer, and where the second image is projected as a function of the object data.

6. The system of claim 1, further comprising a real-world semi-reflective surface placed as an intermediary between a real-world viewer and the real-world setting.

7. The system of claim 6, wherein the second projection system projects an image on an invisible area, where a reflection of the image is then carried to the real-world setting, through the semi-reflective surface, in a manner that is invisible to the real-world viewer.

8. A method of interacting with a virtual image, the method comprising:

acquiring object data from a plurality of sensors that track position information of a real-world object in a real-world setting;

providing an image processing computer configured to determine an expected position of the real-world object within the real-world setting as a function of the object data; and

using a first projection system to project a digital image at a viewing location based on the expected position in a manner where a real-world viewer perceives the digital image to be in proper relation to the real-world object.

9. The method of claim 8, further comprising incorporating a priori choreography information into the function to determine the expect location.

10. The method of claim 8, further comprising capturing data of a second real-world object outside of the visible real-world setting, and using the data to render the digital image.

11. The method of claim 10, wherein the second real-world object is a live performer outside the view of the viewer.

12. The method of claim 8, further comprising providing an indicator visible to the real-world object yet invisible to viewer that indicates where the digital image should be from the perspective of the audience.

14. The stage of claim 11, wherein the semi-reflective surface provides for the viewer to see the real-world object and the holographic-looking objects at the same time.

15. The stage of claim 11, wherein the second projection system comprises an anaglyphic stereoscopic filter.

13. A stage for live performances, comprising:

a first projector system having a first projector and a first projector controller;

a second projector system having a second projector and a second projector controller;

an intermediary semi-transparent viewing surface located between a real-world viewer and a real-world object;

wherein the first projector displays a digital image on a viewing surface visible to the real-world viewer; and

wherein the second projector displays a digital image on a viewing semi-reflective surface invisible to the real-world viewer, the digital image representing a holographic object.