Compact Interactive Tabletop with Projection-Vision

• US 20150160784A1
• Filed: 12/11/2013
• Published: 06/11/2015
• Est. Priority Date: 02/28/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew D., Bathiche, Steven N.
• Granted Patent: US 10,026,177 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The subject application relates to a system(s) and/or methodology that facilitate vision-based projection of any image (still or moving) onto any surface. In particular, a front-projected computer vision-based interactive surface system is provided which uses a new commercially available projection technology to obtain a compact, self-contained form factor. The subject configuration addresses installation, calibration, and portability issues that are primary concerns in most vision-based table systems. The subject application also relates to determining whether an object is touching or hovering over an interactive surface based on an analysis of a shadow image.

Claims

1. A system comprising:

a memory;

a processing unit coupled to the memory;

an object sensing component executed by the processing unit and configured to sense an object over an interactive surface, the interactive surface being illuminated with an infrared source;

a shadow analysis component executed by the processing unit and configured to;

track a shadow of the object appearing on the interactive surface based on the interactive surface reflecting infrared when the object is moving; and

enable touch detection by examining the shadow of the object to determine if the object is hovering over or touching the interactive surface based on an analysis of the shadow of the object.

2. The system of claim 1, wherein the object sensing component is further configured to trigger content to appear on the interactive surface based on the object being determined as hovering or touching the interactive surface.

3. The system of claim 1, wherein the object sensing component is further configured to trigger content to appear on the interactive surface to enable the object to interact with the content on the interactive surface via touching the interactive surface.

4. The system of claim 1, wherein the object sensing component is further configured to trigger a menu including menu options to appear on the interactive surface based on the object being determined as hovering, the menu options selectable by the object via touching the interactive surface.

5. The system of claim 1, the system configured to be integrated into a laptop, a tablet computer, a personal digital assistant, or a cellular telephone.

6. A method implemented at least partially by a processor, the method comprising:

illuminating an interactive surface;

detecting an object over the interactive surface being illuminated with the infrared source;

observing a shadow of the object appearing on the interactive surface based on the object being detected;

reflecting infrared by the interactive surface based on the shadow of the object; and

determining that the object the object is hovering or touching the interactive surface based on an analysis of the shadow of the object.

7. The method of claim 6, wherein the determining the object is touching the interactive surface based at least in part on a binarized image of the object and the shadow of the object.

8. The method of claim 6, further comprising determining that the finger is not touching the interactive surface based at least in part on the finger and the shadow of the finger yielding two images.

9. The method of claim 6, further comprising calculating a height of the object over the interactive surface when the object and the shadow of the object are matched to yield one image, wherein the height serves as a signal for cursor control.

10. The method of claim 6, wherein when the finger is detected triggering a button panel or a window to appear.

11. The method of claim 10, further comprising enabling a touch to press the button panel or the window in the interactive surface.

12. One or more computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

illuminating at least a portion of a physical surface;

observing a shadow of a object appearing on the portion of the physical surface being illuminated; and

determining the object is hovering over or touching the portion of the physical surface based on an analysis of the shadow of the object.

13. The one or more computer-readable media of claim 12, wherein the illumination is infrared illumination.

14. The one or more computer-readable media of claim 12, the acts further comprising detecting an object over the portion of the physical surface being illuminated.

15. The one or more computer-readable media of claim 14, the acts further comprising calculating a height of the finger over the physical surface when the object and the shadow of the object are matched to yield one image, wherein the height serves as a signal for cursor control.

16. The one or more computer-readable media of claim 14, the acts further comprising determining that the object is not touching the portion of the physical surface based at least in part on the object and the shadow of the object yielding two images when the finger and the shadow of the object are binarized.

17. The one or more computer-readable media of claim 14, the acts further comprising determining that the object is touching the portion of the physical surface based at least in part on binarizing the object and the shadow of the object to yield one image.

18. The one or more computer-readable media of claim 12, the acts further comprising:

determining that the object is hovering over the portion of the physical surface; and

triggering a button panel or a window to appear.

19. The one or more computer-readable media of claim 18, the acts further comprising enabling a touch to press the button panel or the window.

20. The one or more computer-readable media of claim 12, the acts further comprising triggering content to appear on the portion of the physical surface in response to determining that the object is hovering over or touching the portion of the physical surface.