Compact interactive tabletop with projection-vision

• US 8,611,667 B2
• Filed: 05/18/2011
• Issued: 12/17/2013
• Est. Priority Date: 02/28/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew D., Bathiche, Steven N.
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US13/110,637
• Publication Number:
  US 20110254832A1
• Time in Patent Office: 944 Days
• Field of Search: 382/100, 382/151, 382/190, 382/199, 382/286, 382/294, 382/295, 382/296, 382/293, 382/276, 396/429, 345/175, 345/418
• US Class Current: 382/190
• CPC Class Codes: G10H 1/0025 Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece automatically producing a series of tones G10H1/26
• CPC Class Codes: G10H 1/36 Accompaniment arrangements
• CPC Class Codes: G10H 1/383 Chord detection and/or recognition, e.g. for correction, or automatic bass generation
• CPC Class Codes: G10H 2210/066 for pitch analysis as part of wider processing for musical purposes, e.g. transcription, musical performance evaluation; Pitch recognition, e.g. in polyphonic sounds; Estimation or use of missing fundamental
• CPC Class Codes: G10H 2210/111 Automatic composing, i.e. using predefined musical rules
• CPC Class Codes: G10H 2250/015 Markov chains, e.g. hidden Markov models [HMM], for musical processing, e.g. musical analysis or musical composition
• CPC Class Codes: G11B 31/02 with automatic musical instruments

Abstract

The subject application relates to a system(s) and/or methodology that facilitate vision-based projection of any image (still or moving) onto any surface. In particular, a front-projected computer vision-based interactive surface system is provided which uses a new commercially available projection technology to obtain a compact, self-contained form factor. The subject configuration addresses installation, calibration, and portability issues that are primary concerns in most vision-based table systems. The subject application also relates to determining whether an object is touching or hovering over an interactive surface based on an analysis of a shadow image.

Claims

1. A method implemented at least partially by a processor, the method comprising:

detecting, by the processor, a presence of at least one physical page within an interactive surface generated by a projection-vision system;

identifying, by the processor, the physical page as a projection surface;

subsequently projecting an image as a virtual representation onto the page; and

tracking, by the processor, the movement of the page within the interactive surface with enough precision to keep the virtual representation of the image on the page, as the page is moved.

2. The method of claim 1, wherein the detecting at least one page comprises detecting two pairs of parallel strong edge lines that are perpendicular to each other.

3. The method of claim 1, wherein the detecting at least one page further comprises detecting overlapping pages.

4. The method of claim 3, wherein:

the detecting at least one page further comprises detecting a second page; and

the projecting further comprises projecting a second image as a second virtual representation onto the second page.

5. The method of claim 1, wherein the projecting further comprising projecting at least one movie onto the page in real-time.

6. A projection-vision system comprising:

a memory;

a processing unit coupled to the memory;

a camera-projection component executed by the processing unit and configured to capture a plurality of objects placed within an interactive surface as an input image using a projector and at least one camera attached to an illuminant; and

an image processor component executed by the processing unit and configured to process the input image to yield a display image of one of the plurality of objects and to subsequently project the display image onto a projection surface.

7. The system of claim 6, the system being self-contained and portable.

8. The system of claim 6, the illuminant being an infrared source.

9. The system of claim 6, the illuminant being positioned off axis from the at least one camera.

10. The system of claim 6, wherein the display image is a printed page captured in the input image.

11. The system of claim 10, wherein the image processor component detects two pairs of parallel strong edge lines that are perpendicular to each other to isolate the printed page from the remainder of the input image.

12. The system of claim 6, wherein the camera-projection component focuses on the at least one camera by detecting four corners of a projected display on the projection surface.

13. The system of claim 6, wherein the system is integrated into at least one of a laptop, a tablet computer, a personal digital assistant, or a cellular telephone.

14. One or more computer-readable storage media comprising instructions that when executed by one or more processors, cause the one or more processors to:

illuminate an interactive surface;

detect a presence of at least one visually coded object from among a plurality of objects within the interactive surface, the plurality of objects being real objects;

capture the at least one visually coded object as an image; and

subsequently projecting the visually coded object as a virtual representation onto the interactive surface as a virtual object.

15. The one or more computer-readable storage media of claim 14, further comprising:

detecting at least one page; and

wherein the projecting the visually coded object as a virtual representation further comprises projecting the visually coded object onto the page.

16. The one or more computer-readable storage media of claim 14, comprising instructions that when executed by the one or more processors, cause the one or more processors to detect shadows overlapping the visually coded object.

17. The one or more computer-readable storage media of claim 14, comprising instructions that when executed by the one or more processors, cause the one or more processors to project the image onto at least one of a real or a virtual page on the projection surface in real time.

18. The one or more computer-readable storage media of claim 14, comprising instructions that when executed by the one or more processors, cause the one or more processors to project at least one of a movie or a different image onto the interactive surface in addition to the image of the visual coded object.

19. The one or more computer-readable storage media of claim 14, wherein the subsequently displaying the image as a virtual representation comprises projecting the image onto a projection surface.

20. The one or more computer-readable storage media of claim 14, wherein the subsequently displaying the image as a virtual representation comprises displaying the image on a display in addition to the interactive surface.