TOUCH CLASSIFICATION

• US 20160012348A1
• Filed: 07/11/2014
• Published: 01/14/2016
• Est. Priority Date: 07/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Johnson, Dan, Sala, Pablo
• Granted Patent: US 9,558,455 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 5/027 Frames

Abstract

A method for touch classification includes obtaining frame data representative of a plurality of frames captured by a touch-sensitive device, analyzing the frame data to define a respective blob in each frame of the plurality of frames, the blobs being indicative of a touch event, computing a plurality of feature sets for the touch event, each feature set specifying properties of the respective blob in each frame of the plurality of frames, and determining a type of the touch event via machine learning classification configured to provide multiple non-bimodal classification scores based on the plurality of feature sets for the plurality of frames, each non-bimodal classification score being indicative of an ambiguity level in the machine learning classification.

Claims

1. A computer-implemented method comprising:

obtaining frame data representative of a plurality of frames captured by a touch-sensitive device;

analyzing the frame data to define a respective blob in each frame of the plurality of frames, the blobs being indicative of a touch event;

computing a plurality of feature sets for the touch event, each feature set specifying properties of the respective blob in each frame of the plurality of frames; and

determining a type of the touch event via machine learning classification configured to provide multiple non-bimodal classification scores based on the plurality of feature sets for the plurality of frames, each non-bimodal classification score being indicative of an ambiguity level in the machine learning classification.

2. The computer-implemented method of claim 1, wherein the machine learning classification is configured to generate the non-bimodal classification scores such that each non-bimodal classification score is representative of a probability that the touch event is of a respective type.

3. The computer-implemented method of claim 2, wherein each one of the non-bimodal classification scores is generated by a machine learning classifier configured to accept the plurality of feature sets as inputs.

4. The computer-implemented method of claim 3, wherein the machine learning classifier comprises a random decision forest classifier.

5. The computer-implemented method of claim 1, further comprising:

defining a track of the blobs across the plurality of frames for the touch event; and

computing a track feature set for the track,

wherein determining the type comprises applying the track feature set to a machine learning classifier.

6. The computer-implemented method of claim 1, wherein computing the plurality of feature sets comprises aggregating data indicative of the plurality of feature sets before application of the plurality of feature sets to a machine learning classifier in determining the type of the touch event.

7. The computer-implemented method of claim 1, wherein each feature set comprises data indicative of an appearance of an image patch disposed at the respective blob in each frame.

8. The computer-implemented method of claim 1, wherein each feature set comprises data indicative of an intensity gradient in the frame data for the respective blob in each frame.

9. The computer-implemented method of claim 1, wherein each feature set comprises data indicative of an isoperimetric quotient or other metric of a roundness of the respective blob in each frame.

10. The computer-implemented method of claim 1, wherein the machine learning classification comprises a lookup table-based classification.

11. The computer-implemented method of claim 1, wherein determining the type comprises applying the feature set for a respective frame of the plurality of frames to multiple look-up tables, each look-up table providing a respective individual non-bimodal classification score of the multiple non-bimodal classification scores.

12. The computer-implemented method of claim 11, wherein determining the type comprises combining each of the individual non-bimodal classification scores for the respective frame to generate a blob classification rating score for the respective frame.

13. The computer-implemented method of claim 12, wherein:

the multiple look-up tables comprise a first look-up table configured to provide a first rating that the touch event is an intended touch and further comprise a second look-up table to determine a second rating that the touch event is an unintended touch; and

determining the type comprises subtracting the second rating from the first rating to determine the blob classification rating score for the respective frame.

14. The computer-implemented method of claim 12, wherein determining the type comprises aggregating the blob classification rating scores across the plurality of frames to determine a cumulative, multi-frame classification score for the touch event.

15. The computer-implemented method of claim 14, wherein determining the type comprises:

determining whether the cumulative, multi-frame classification score passes one of multiple classification thresholds; and

if not, then iterating the feature set applying, the classification score combining, and the rating score aggregating acts in connection with a further feature set of the plurality of feature sets.

16. The computer-implemented method of claim 14, wherein determining the type further comprises, once the cumulative, multi-frame classification score exceeds passes a palm classification threshold for the touch event, classifying a further blob in a subsequent frame of the plurality of frames that overlaps the touch event as a palm touch event.

17. The computer-implemented method of claim 12, wherein combining each of the individual non-bimodal classification scores comprises adjusting the blob classification rating score by subtracting a value from the blob classification rating score if the respective blob overlaps an anti-blob.

18. The computer-implemented method of claim 12, wherein combining each of the individual non-bimodal classification scores comprises, when the blob has an area greater than a threshold area, and when the blob is within a threshold distance of a further blob having bimodal classification scores indicative of a palm, adjusting the blob classification rating score by subtracting a quotient calculated by dividing a blob area of the blob by the threshold area.

19. The computer-implemented method of claim 12, wherein combining each of the individual non-bimodal classification scores comprises:

determining if a number of edge pixels in the respective blob exceeds a threshold; and

if the threshold is exceeded, adjusting the blob classification rating score by subtracting a difference between the number of edge pixels and the threshold from the blob classification rating score.

20. A touch-sensitive device comprising:

a touch-sensitive surface;

a memory in which blob definition instructions, feature computation instructions, and machine learning classification instructions are stored; and

a processor coupled to the memory, configured to obtain frame data representative of a plurality of frames captured via the touch-sensitive surface and configured to execute the blob definition instructions to analyze the frame data to define a respective blob in each frame of the plurality of frames, the blobs being indicative of a touch event;

wherein the processor is further configured to execute the feature computation instructions to compute a plurality of feature sets for the touch event, each feature set specifying properties of the respective blob in each frame of the plurality of frames; and

wherein the processor is further configured to execute the machine learning classification instructions to determine a type of the touch event via machine learning classification configured to provide multiple non-bimodal classification scores based on the plurality of feature sets for the plurality of frames, each non-bimodal classification score being indicative of an ambiguity level in the machine learning classification.

21. The touch-sensitive device of claim 20, wherein each non-bimodal classification score is representative of a probability that the touch event is of a respective type.

22. The touch-sensitive device of claim 20, wherein:

each non-bimodal classification score is a blob classification score rating for a respective frame of the plurality of frames; and

the processor is further configured to execute the machine learning classification instructions to sum the blob classification score ratings over the plurality of frames.

23. The touch-sensitive device of claim 22, wherein the processor is further configured to execute the machine learning classification instructions to combine lookup table ratings from multiple lookup tables to compute each blob classification score rating.

24. The touch-sensitive device of claim 20, wherein the processor is further configured to execute the blob definition instructions to split a connected component into multiple blobs for separate analysis.

25. The touch-sensitive device of claim 20, wherein the processor is further configured to execute the blob definition instructions to define a track for each blob of the touch event across the plurality of frames.

26. The touch-sensitive device of claim 20, wherein the processor is further configured to execute the blob definition instructions to merge multiple connected components for analysis as a single blob.

27. A touch-sensitive device comprising:

a touch-sensitive surface;

a memory in which a plurality of instruction sets are stored; and

a processor coupled to the memory and configured to execute the plurality of instruction sets,

wherein the plurality of instructions sets comprise;

first instructions to cause the processor to obtain frame data representative of a plurality of sensor images captured by the touch-sensitive device;

second instructions to cause the processor to analyze the frame data to define a respective connected component in each sensor image of the plurality of sensor images, the connected components being indicative of a touch event;

third instructions to cause the processor to compute a plurality of feature sets for the touch event, each feature set specifying properties of the respective connected component in each sensor image of the plurality of sensor images;

fourth instructions to cause the processor to determine a type of the touch event via machine learning classification configured to provide multiple non-bimodal classification scores based on the plurality of feature sets for the plurality of frames, each non-bimodal classification score being indicative of an ambiguity level in the machine learning classification; and

fifth instructions to cause the processor to provide an output to the computing system, the output being indicative of the type of the touch event;

wherein the fourth instructions comprise aggregation instructions to cause the processor to aggregate information representative of the touch event over the plurality of sensor images.

28. The touch-sensitive device of claim 27, wherein:

the fourth instructions are configured to cause the processor to apply the plurality of feature sets to a machine learning classifier; and

the aggregation instructions are configured to cause the processor to aggregate the plurality of feature sets for the plurality of sensor images before applying the plurality of feature sets.

29. The touch-sensitive device of claim 27, wherein the aggregation instructions are configured to cause the processor to aggregate the multiple non-bimodal classification scores.