Interactive video window

• US 8,035,614 B2
• Filed: 10/30/2007
• Issued: 10/11/2011
• Est. Priority Date: 05/28/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US11/929,778
• Publication Number:
  US 20080150890A1
• Time in Patent Office: 1,442 Days
• Field of Search: 345156-166, 345173-179, 178 1801- 1809, 178 1901- 1906
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An interactive video window display system. A projector projects a visual image. A screen displays the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen, and wherein the screen is adjacent to a window. An illuminator illuminates an object on a front side of the window. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable by the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction. The projector, the camera, the illuminator, and the computer system are located on the same side of the window.

Claims

1. An interactive video window display system, comprising:

means for projecting a visual image;

means for displaying the visual image, the means for displaying located behind a window having a front side, wherein the visual image is projected onto a back side of the means for displaying and is configured for viewing on a front side of the means for displaying, wherein an angle between the projected visual image and the means for displaying is oblique and the means for projecting is configured to adjust the visual image such that the visual image is substantially rectangular on the means for displaying;

means for detecting interaction of an object on the front side of the window with the visual image, wherein the means for displaying is at least partially transparent to light detectable by the means for detecting and the means for detecting may detect the object through the means for displaying; and

a computing device configured to cause the means for projecting to change the visual image in response to the interaction of the object with the visual image, wherein the means for projecting and means for detecting are located on a same side of the window.

3. The interactive video display system of claim 1, further comprising an illuminator positioned to illuminate an object, wherein the illuminator is an infrared illuminator for illuminating the object with infrared illumination, and the camera is an infrared camera for detecting infrared illumination.

4. The interactive video display system of claim 3, wherein a first side of the screen includes a screen material that is substantially transparent when viewed from a substantially perpendicular angle and substantially translucent when viewed at an oblique angle.

5. The interactive video display system of claim 3, wherein the screen comprises a material that substantially scatters light projected by the projector but not light emitted by the illuminator.

6. The interactive video display system of claim 1, wherein the projector produces polarized light, the screen includes a scattering polarizer that substantially scatters the light produced by the projector, and the camera is sensitive to light of a polarization not substantially scattered by the scattering polarizer.

7. The interactive video display system of claim 6, further comprising a linear polarizer located adjacent to the front side of the screen, the linear polarizer configured to absorb light of the polarization not scattered by the screen.

8. The interactive video display system of claim 6, wherein the projector includes a visible-light projector and the camera includes a visible-light camera sensitive to light polarized in a transparent direction of the scattering polarizer.

9. The interactive video display system of claim 8, wherein output of the projector includes a live color image of a user interacting with the screen.

10. The interactive video display system of claim 6, further comprising a linear polarizer configured to polarize light received at the camera, the light polarized at camera-sensitive wavelengths whereby the camera disregards light scattered by the scattering polarizer.

11. The interactive video display system of claim 1, further comprising a micro-prism adjacent to the screen.

12. The interactive video display system of claim 1, further comprising a mirror for redirecting a view of the screen by the camera wherein the camera is pointed indirectly at the screen.

13. The interactive video display system of claim 1, further comprising a Fresnel lens positioned adjacent to the screen, the lens configured to correct distortion of a view by the camera.

14. The interactive video display system of claim 1, wherein the screen includes a material that changes from substantially translucent to substantially transparent, and wherein the screen is substantially translucent when the projector is projecting an image and substantially transparent when the camera is detecting an object in front of the screen.

15. The interactive video display system of claim 1, wherein the screen includes a material that is substantially translucent at a particular angle and substantially transparent at other angles, and wherein the projector projects light onto the screen from the particular angle.

16. The interactive video display system of claim 3, wherein the illuminator is configured to strobe in time with exposures of the camera.

17. The interactive video display system of claim 1, wherein an image of the camera is calibrated to the visual image whereby an interaction caused by a detected object is matched to a physical position of the object proximate to the screen.

2. An interactive video window display system, comprising:

an off-axis projector configured to project a visual image;

a screen having a front side and a back side, the screen configured to depict the visual image, wherein the visual image is projected onto the back side of the screen for presentation to a user on the front side of the screen;

a camera positioned behind a lower portion of the screen and angled upward, or positioned behind an upper portion of the screen and angled downward, in order to detect only objects that are near the screen, wherein the screen is partially transparent to light detectable by the camera; and

a computing device configured to detect interactions between the visual image and the detected objects and to direct the projector to change the visual image in response to the interactions, wherein the projector and camera, are located on a same side of the screen.

18. The interactive video display system of claim 2, wherein the camera is further configured to determine information about a distance of an object from the screen.

19. The interactive video display system of claim 18, wherein the camera includes a stereo camera.

20. The interactive video display system of claim 2, wherein the screen is configured to provide touch screen functionality when an object is proximate the screen.

21. The interactive video display system of claim 20, further comprising a transparent touchscreen adjacent to the front side of the screen.

22. The interactive video display system of claim 20, further comprising an edge-lit transparent sheet adjacent to the front side of the screen, and wherein the camera is further configured to distinguish light created when the object comes in contact with the edge-lit transparent sheet.

23. A method comprising:

projecting a visual image on a back side of a screen, wherein the visual image is visible to a user on a front side of the screen;

emitting light to illuminate an object in front of the screen, wherein the screen is partially transparent to the emitted light;

capturing images of the object by a camera positioned behind a lower portion of the screen and angled upward or positioned behind an upper portion of the screen and angled downward;

detecting, by a computing device, an interaction of the illuminated object with the visual image based at least on the captured images; and

initiating change, by the computing device, of the visual image, in response to the interaction of the illuminated object with the visual image.

24. The method of claim 23, wherein the emitted light is infrared light.

25. The method of claim 23, further comprising substantially scattering polarized light of the projected visual image at the screen.

26. The method of claim 25, further comprising transmitting light of the polarization not scattered by the screen.

27. The method of claim 23, further comprising correcting distortion of a view of the detected interaction.

28. The method of claim 23, wherein the emitted light is strobed in time.

29. The method of claim 23, wherein detecting the interaction of the illuminated object with the visual image includes determining information about a distance of the object from the screen.