DEPTH CAMERA AS A TOUCH SENSOR

• US 20130057515A1
• Filed: 09/07/2011
• Published: 03/07/2013
• Est. Priority Date: 09/07/2011
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew David
• Application Number:
  US13/227,466
• Publication Number:
  US 20130057515A1
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Architecture that employs depth sensing cameras to detect touch on a surface, such as a tabletop. The act of touching is processed using thresholds which are automatically computed from depth image data, and these thresholds are used to generate a touch image. More specifically, the thresholds (near and far, relative to the camera) are used to segment a typical finger that touches a surface. A snapshot image is captured of the scene and a surfaced histogram is computed from the snapshot over a small range of deviations at each pixel location. The near threshold (nearest to the camera) is computed based on the anthropometry of fingers and hands, and associated posture during touch. After computing the surface histogram, the far threshold values (furthest from the camera) can be stored as an image of thresholds, used in a single pass to classify all pixels in the input depth image.

Claims

1. A system, comprising:

a sensing component that senses depth image data of a surface relative to which user actions of a user are performed;

a touch component that determines an act of touching the surface based on the depth image data; and

a processor that executes computer-executable instructions associated with at least one of the sensing component or the touch component.

2. The system of claim 1, wherein the touch component computes a model of the surface that includes depth deviation data at each pixel location as the depth image data.

3. The system of claim 1, wherein the touch component classifies pixels of the depth image data according to threshold values.

4. The system of claim 1, wherein the touch component computes physical characteristics of the user as sensed by the sensing component to interpret the user actions.

5. The system of claim 1, wherein the touch component establishes a maximum threshold value based on a histogram of depth values and finds a first depth value that exceeds a threshold value as the maximum threshold value.

6. The system of claim 1, wherein the sensing component captures a snapshot of the depth image data of the surface during an unobstructed view of the surface and the touch component models the surface based on the depth image data.

7. The system of claim 1, wherein the touch component identifies discrete touch points using filtering and associated groups of pixels that correspond to the touch points.

8. The system of claim 7, wherein the touch component tracks the touch points over time to implement familiar multi-touch interactions.

9. A method, comprising acts of:

receiving a surface over which user actions of a user are performed;

computing depth image data of an image of the surface;

determining an act of touching the surface based on the depth image data; and

utilizing a processor to execute instructions stored in memory to perform at least one of the acts of computing or determining.

10. The method of claim 9, further comprising computing a surface histogram over a subset of deviations of the depth image data at each pixel location of the image.

11. The method of claim 9, further comprising classifying pixels of the depth image data according to threshold values.

12. The method of claim 9, further comprising determining the act of touching by a finger of the user.

13. The method of claim 9, further comprising determining physical characteristics of the user to interpret the user actions.

14. The method of claim 9, further comprising establishing a maximum threshold value based on a histogram of raw shift values and finding a first depth value that exceeds a threshold value as the maximum threshold value.

15. The method of claim 9, further comprising modeling the surface by capturing a snapshot of the depth image data of the surface during an unobstructed view of the surface.

16. A method, comprising acts of:

receiving a surface over which user actions of a user are performed;

modeling the surface by capturing a snapshot of the depth image data of the surface during an unobstructed view of the surface;

computing depth image data of an image of the surface;

computing a surface histogram over a subset of deviations of the depth image data at each pixel location of the image;

determining an act of touching the surface based on the depth image data; and

utilizing a processor to execute instructions stored in memory to perform at least one of the acts of modeling, computing, or determining.

17. The method of claim 16, further comprising classifying pixels of the depth image data according to threshold values.

18. The method of claim 16, further comprising determining the act of touching by a finger of the user.

19. The method of claim 16, further comprising determining physical characteristics of the user to interpret the user actions.

20. The method of claim 16, further comprising establishing a maximum threshold value based on a histogram of raw shift values and finding a first depth value that exceeds a threshold value as the maximum threshold value.