Interactive Video Window

• US 20080150890A1
• Filed: 10/30/2007
• Published: 06/26/2008
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Granted Patent: US 8,035,614 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An interactive video window display system. A projector projects a visual image. A screen displays the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen, and wherein the screen is adjacent to a window. An illuminator illuminates an object on a front side of the window. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable by the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction. The projector, the camera, the illuminator, and the computer system are located on the same side of the window.

Claims

1. An interactive video window display system, comprising:

a projector configured to project a visual image;

a screen behind a window having a front side, the screen configured to display the visual image, wherein the visual image is projected onto a back side of the screen for presentation to a user on a front side of the screen;

an illuminator configured to illuminate an object on the front side of the window;

a camera configured to detect an interaction of an illuminated object with the visual image, the interaction occurring within a volume of space in front of the window, wherein the screen is partially transparent to light detectable by the camera whereby the camera may detect the illuminated object through the screen; and

a computing device configured to direct the projector to change the visual image in response to the interaction of the illuminated object with the visual image, wherein the projector, camera, illuminator, and computing device are located on the same side of the window.

2. The interactive video window display system of claim 1, wherein the illuminator is an infrared illuminator for illuminating the object with infrared illumination, and the camera is an infrared camera for detecting infrared illumination.

3. The interactive video window display system of claim 1, wherein the projector is an off-axis projector.

4. The interactive video window display system of claim 3, wherein the projector is an off-axis projector and a first side of the screen includes a screen material that is substantially transparent when viewed from a substantially perpendicular angle and substantially translucent when viewed at an oblique angle.

9. The interactive video window display system of claim 4, wherein the screen material is a wavelength-selective scattering material that substantially scatters light projected by the projector but not light emitted by the illuminator.

11. The interactive video window display system of claim 1, wherein the projector produces polarized light, and the screen includes a scattering polarizer that substantially scatters the light produced by the projector and the camera is sensitive to light of a polarization not substantially scattered by the scattering polarizer.

13. The interactive video window display system of claim 11, further comprising a linear polarizer located adjacent to the front side of the screen, the linear polarizer configured to absorb light of the polarization not scattered by the screen.

14. The interactive video window display system of claim 11, wherein the projector includes a visible-light projector and the camera includes a visible-light camera sensitive to light polarized in the transparent direction of the scattering polarizer.

15. The interactive video window display system of claim 14, wherein output of the projector includes a live color image of a user interacting with the window.

16. The interactive video window display system of claim 1, wherein the camera is located below the bottom of the screen whereby the camera does not detect objects that are not near the window.

17. The interactive video window display system of claim 11, further comprising a linear polarizer configured to polarize light received at the camera, the light polarized at camera-sensitive wavelengths whereby the camera disregards light scattered by the scattering polarizer.

18. The interactive video window display system of claim 1, further comprising a micro-prism adjacent to the screen.

19. The interactive video window display system of claim 1, further comprising a mirror for redirecting a view of the screen by the camera wherein the camera is pointed indirectly at the screen.

22. The interactive video window display system of claim 1, further comprising a Fresnel lens positioned adjacent to the screen, the lens configured to correct distortion of a view by the camera.

23. The interactive video window display system of claim 1, wherein the screen included a material that changes from substantially translucent to substantially transparent, and wherein the screen is substantially translucent when the projector is projecting an image and substantially transparent when the camera is detecting an object in front of the screen.

24. The interactive video window display system of claim 1, wherein the screen includes a material that is substantially translucent at a particular angle and substantially transparent at other angles, and wherein the projector projects light onto the screen from the particular angle.

25. The interactive video window display system of claim 1, wherein the illuminator is configured to strobe in time with exposures of the camera.

26. The interactive video widow display system of claim 1, wherein an image of the camera is calibrated to the visual image whereby the interaction caused by the object is matched to a physical position of the object proximate to the screen.

27. The interactive video window display system of claim 1, wherein the camera is further configured to determine information about a distance of the object from the screen.

28. The interactive video window display system of claim 27, wherein the camera includes a stereo camera.

31. The interactive video window display system of claim 1, wherein the screen is configured to provide touch screen functionality when the object is proximate the window.

32. The interactive video window display system of claim 31, further comprising a transparent touchscreen adjacent to the front side of the screen.

33. The interactive video window display system of claim 31, further comprising an edge-lit transparent sheet adjacent to the front side of the screen, and wherein the camera is further configured to distinguish light created when the object comes in contact with the edge-lit transparent sheet.

5-8. -8. (canceled)

10. (canceled)

12. (canceled)

20-21. -21. (canceled)

29-30. -30. (canceled)

34-99. -99. (canceled)

100. A method for managing interaction with a visual image, the method comprising:

projecting a visual image on a back side of a screen, wherein the visual image is visible to a user on a front side of the screen;

emitting light to illuminate an object in front of a window located in front of the screen, wherein the screen is partially transparent to the emitted light;

detecting an interaction of the illuminated object with the visual image, the interaction occurring within a volume of space in front of the window, the detection occurring through the screen; and

changing the visual image in response to the interaction of the illuminated object with the visual image.

101. The method of claim 100, wherein the emitted light is infrared light.

102. The method of claim 100, further comprising substantially scattering polarized light of the projected image at the screen.

103. The method of claim 102, further comprising transmitting light of the polarization not scattered by the screen.

104. The method of claim 102, further comprising polarizing light corresponding to the detected interaction of the illuminated object with the visual image sum that light scattered by the screen may be disregarded.

105. The method of claim 100, further comprising correcting distortion of a view of the detected interaction.

106. The method of claim 100, wherein the emitted light is strobed in time.

107. The method of claim 100, wherein detecting the interaction of the illuminated object with the visual image includes determining information about a distance of the object from the screen.

108. An interactive video window display system, comprising:

means for projecting a visual image;

means for displaying the visual image, the means for displaying located behind a window having a front side, wherein the visual image is projected onto a back side of the means for displaying and is presented on a front side of the means for displaying;

means for illuminating an object on the front side of the window;

means for detecting interaction of an illuminated object with the visual image, the interaction occurring within a volume of space in front of the window, wherein the means for displaying is at least partially transparent to light detectable by the means for detecting and the means for detecting may detect the illuminated object through the means for displaying; and

means for directing the means for projecting to change the visual image in response to the interaction of the illuminated object with the visual image, wherein the means for projecting, means for detecting, means for illuminating, and means for directing are located on the same side of the window.