Self-contained interactive video display system

• US 8,035,612 B2
• Filed: 09/20/2004
• Issued: 10/11/2011
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US10/946,263
• Publication Number:
  US 20050162381A1
• Time in Patent Office: 2,577 Days
• Field of Search: 345156-168, 345173-179, 178 1801- 1807, 178 1901- 1906
• US Class Current: 345/156
• CPC Class Codes: A63F 2300/1068 being specially adapted to detect the point of contact of the player on a surface, e.g. floor mat, touch pad
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

A self-contained interactive video display system. A projector projects a visual image onto a screen for displaying the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen. An illuminator illuminates an object near the front side of the screen. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable to the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction.

Claims

1. A display system comprising:

projecting means for projecting visual images;

depicting means comprising a scattering polarizer for depicting visual images projected by the projecting means, wherein the projecting means is configured to emit polarized light in a polarization substantially scattered by the scattering polarizer;

sensor means sensitive to light of a polarization not substantially scattered by the scattering polarizer positioned on a first side of the depicting means for acquiring images representative of at least a portion of one or more objects positioned on a second side of the depicting means, wherein the first and second sides of the depicting means are substantially opposite sides of the depicting means; and

computing means for detecting an interaction of at least one of the one or more objects with at least a portion of at least one of the visual images projected by the projecting means.

2. The display system of claim 1, wherein the computing means is configured to update the visual images projected by the projecting means in response to the detected interaction.

3. The display system of claim 1, wherein the depicting means comprises a time-based material configured to change from substantially translucent to substantially transparent in response to a state of the projecting means.

4. The display system of claim 1, wherein the depicting means comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein the projecting means projects light onto the depicting means from at least the particular angle.

5. The display system of claim 1, wherein the depicting means comprises a screen that is at least partially transparent.

13. The interactive video display system as recited in claim 1 wherein the projector, the camera, the illuminator, and the computer system are comprised within an enclosure, and wherein one side of the enclosure comprises the screen.

15. The interactive video display system as recited in claim 1, wherein the illuminator projects illumination through the screen onto the object.

16. The interactive video display system as recited in claim 15, wherein the illuminator is positioned so as to reduce any potential for glare effects on the camera.

30. The interactive video display system as recited in claim 1 wherein the interactive video display system provides touchscreen functionality when the object is proximate the screen.

31. The interactive video display system as recited in claim 30 further comprising a transparent touchscreen adjacent the front side of the screen.

32. The interactive video display system as recited in claim 30 further comprising an edge-lit transparent sheet adjacent the front side of the screen, and wherein the camera is operable to distinguish light created when the object comes in contact with the edge-lit transparent sheet.

6. An interactive video display system comprising:

a projector configured to project a visual image;

a screen positioned so that the visual image is projected onto a back side of the screen, wherein the screen is at least partially transparent and comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein the projector projects light onto the screen from the particular angle;

a camera configured to acquire images representative of at least a portion of an object near a front side of the screen that is opposite the back side of the screen; and

a computer system configured to receive the images from the camera and, based at least on the images, to detect an interaction of the object with at least a portion of the visual image and to direct the projector to change the visual image in response to the interaction.

7. A method of projecting visual images onto a screen, the method comprising:

projecting a visual image onto a first side of a screen, wherein the screen is at least partially transparent and configured for viewing from a second side of the screen opposite the first side of the screen;

acquiring images from an imaging device positioned on the first side of the screen, wherein the images are representative of at least a portion of an object positioned near the screen;

illuminating the object with strobed illumination that is synced with the imaging device; and

detecting an interaction of the object with at least a portion of the visual image based at least partly on the acquired images.

8. The method of claim 7, further comprising:

updating the visual image in response to the interaction.

9. An interactive video display system comprising:

a projector configured to project a visual image;

a screen positioned so that the visual image is projected onto a back side of the screen, wherein the screen is at least partially transparent;

a camera configured to acquire images representative of at least a portion of an object near a front side of the screen that is opposite the back side of the screen;

an illuminator configured to illuminate the object, wherein the illumination is strobed in time with exposures of the camera; and

a computer system configured to receive the images from the camera and, based at least on the images, to detect an interaction of the object with at least a portion of the visual image and to direct the projector to change the visual image in response to the interaction.

10. The interactive video display system as recited in claim 9, further comprising an illuminator configured to illuminate the object with infrared illumination, and wherein the camera is an infrared camera for detecting infrared illumination.

11. The interactive video display system as recited in claim 9 wherein linear polarizing films of perpendicular orientations are placed adjacent to the front side and the back side of the screen.

12. The interactive video display system as recited in claim 9 wherein the projector emits polarized light, and a linear polarizing film of perpendicular orientation is placed adjacent to the front side of the screen.

14. The interactive video display system as recited in claim 9 further comprising a mirror for projecting the visual image onto the screen, wherein the projector is pointed indirectly at the screen and the mirror redirects the visual image to the screen.

17. The interactive video display system as recited in claim 9 wherein the illuminator is located next to the screen such that the illuminator does not project illumination through the screen.

18. The interactive video display system as recited in claim 9 comprising a plurality of illuminators, wherein the illuminators are located next to the screen and behind the screen.

19. The interactive video display system as recited in claim 9 wherein the projector is an off-axis projector.

20. The interactive video display system as recited in claim 9 further comprising a Fresnel lens positioned adjacent to the screen to correct distortion of a view of the camera.

21. The interactive video display system as recited in claim 9, further comprising a series of mirror strips positioned at a distance from the screen to correct distortion of a view of the camera.

22. The interactive video display system as recited in claim 9 wherein the screen comprises a scattering polarizer, wherein the projector emits polarized light in a polarization substantially scattered by the scattering polarizer, and the camera is sensitive to light of a polarization not substantially scattered by the scattering polarizer.

23. The interactive video display system as recited in claim 9 wherein the screen comprises a material that substantially scatters light of wavelengths emitted by the projector, but does not substantially scatter wavelengths detectable to the camera.

24. The interactive video display system as recited in claim 9 wherein the screen comprises a time-based material that can change from substantially translucent to substantially transparent, wherein the screen is substantially translucent when the projector is projecting the visual image, and is substantially transparent when the camera is detecting objects near the front side of the screen.

25. The interactive video display system as recited in claim 9 wherein an image of the camera is calibrated to the visual image such that the interaction caused by the object is matched to a physical position of the object near to the screen.

26. The interactive video display system as recited in claim 9 wherein the interactive video display system is operable to determine information about a distance of the object from the screen.

27. The interactive video display system as recited in claim 26 wherein the camera is a stereo camera.

28. The interactive video display system as recited in claim 26 wherein the camera is a time-of-flight camera.

29. The interactive video display system as recited in claim 28 wherein the time-of-flight camera is positioned such that the time-of-flight camera does not reflect back onto itself.

33. The interactive video display system as recited in claim 9 wherein the object is a body part of a human user.

34. The interactive video display system as recited in claim 9 wherein the interactive video display system may be tiled with a second interactive video display system.

35. The interactive video display system as recited in claim 9 further comprising a second camera, a first Fresnel lens for the camera, and a second Fresnel lens for the second camera, allowing spaces viewed by the camera and the second camera to be stitched together.