Processing an image utilizing a spatially varying pattern

• US 7,710,391 B2
• Filed: 09/20/2004
• Issued: 05/04/2010
• Est. Priority Date: 05/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew, Zide, Joshua, Shaughnessy, Helen, Gleckman, Philip
• Primary Examiner(s): Shankar; Vijay
• Application Number:
  US10/946,414
• Publication Number:
  US 20050110964A1
• Time in Patent Office: 2,052 Days
• Field of Search: 345156-160, 345418-434, 345471-475
• US Class Current: 345/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

An interactive video window display system. A projector projects a visual image. A screen displays the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen, and wherein the screen is adjacent to a window. An illuminator illuminates an object on a front side of the window. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable by the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction. The projector, the camera, the illuminator, and the computer system are located on the same side of the window.

Claims

1. An interactive display system, comprising:

an illuminator configured to emit light in a spatially varying pattern at a wavelength detectable by a camera but substantially invisible to the human eye, the light illuminating an object in an interactive area with the spatially varying pattern, wherein the spatially varying pattern creates an appearance of a texture on the object in the interactive area;

a camera configured to generate an image comprising the appearance of the texture on the object;

a computing device configured to

calculate a three-dimensional position of various points on the object based at least in part on the appearance of the texture on the object in the image, and

generate a visual effect based on the three-dimensional position of the various points of the object; and

a video display for displaying the generated visual effect.

2. The interactive display system of claim 1, wherein the spatially varying pattern is a random dot pattern.

3. The interactive display system of claim 1, wherein the camera is a stereo camera.

4. The interactive display system of claim 3, wherein the image of the object is generated utilizing a stereo matching algorithm to detect the presence and distance of the object.

5. The interactive display system of claim 4, wherein the stereo matching algorithm is the Marr-Poggio algorithm.

6. The interactive display system of claim 1, wherein the camera is an infrared camera.

7. The interactive display system of claim 1, wherein the video display is vertically oriented.

8. The interactive display system of claim 7, wherein the interactive area is located in front of the video display.

9. The interactive display system of claim 1, wherein the video display is horizontally oriented.

10. The interactive display system of claim 9, wherein the interactive area is located above the video display.

11. The interactive display system of claim 1, wherein the illuminator includes a light source, lens system, and pattern slide to focus the spatially varying pattern in the interactive area.

12. The interactive display system of claim 11, wherein the light source is an infrared light source.

13. The interactive display system of claim 12, wherein the light source comprises a cluster of infrared light emitting diodes.

14. The interactive display system of claim 13, wherein the light emitting diodes may be strobed.

20. The interactive display system of claim 2, wherein the visual effect is displayed in conjunction with a representation of the object.

15. A method for interacting with a display, comprising:

emitting a spatially varying light pattern from an illuminator, the spatially varying light pattern illuminating an object in an interactive area;

generating an image comprising at least a portion of the object illuminated by the spatially varying light pattern;

calculating a three-dimensional position of various points on the object based at least in part on the spatially varying light pattern illuminating the object; and

generating a visual effect based on the three-dimensional position of various points on the object.

16. The method of claim 15, wherein the spatially varying light pattern is a random dot pattern.

17. The method of claim 15, wherein the image is generated using a stereo matching algorithm.

18. The method of claim 17, wherein the stereo matching algorithm is the Marr-Poggio algorithm.

21. The method of claim 15, wherein the visual effect is displayed in conjunction with a representation of the object.

19. An interactive display system, comprising:

means for emitting light in a spatial pattern to illuminate an object in an interactive area, wherein the spatial pattern creates an appearance of a texture on the object;

means for generating an image of the object in the interactive area, wherein the image comprises the appearance of the texture on the object; and

means for calculating a three-dimensional position of various points on the object based at least in part on the appearance of the texture on the object and generating a visual effect based on the three-dimensional position of various points of the object.