Input Classification for Multi-Touch Systems

• US 20140104192A1
• Filed: 10/17/2012
• Published: 04/17/2014
• Est. Priority Date: 10/17/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Davidson, Philip L., Han, Jefferson Y.
• Granted Patent: US 9,632,605 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/0447 Position sensing using the local deformation of sensor cells
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for input classification for multi-touch systems. In one aspect, a method includes maintaining a history of prior state information related to a touch sensitive display. The method further includes detecting that a previous contact with the touch sensitive display was incorrectly classified. The method further includes updating a classification of the previous contact based on the detection that the previous contact was incorrectly classified. The method further includes rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified based on the history of prior state information and the updated classification of the previous contact.

Claims

1. A computer implemented method, comprising:

maintaining a history of prior state information related to a touch sensitive display;

detecting that a previous contact with the touch sensitive display was incorrectly classified;

updating a classification of the previous contact based on the detection that the previous contact was incorrectly classified; and

rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified based on the history of prior state information and the updated classification of the previous contact.

2. The method of claim 1, wherein maintaining the history of prior state information related to the touch sensitive display comprises maintaining state information for a series of past frames, the state information comprising, for each frame, locations of intended touches detected, locations of incidental touches detected, locations of stylus inputs detected, and locations of objects displayed on the touch sensitive display.

3. The method of claim 2:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises receiving, at a current frame, a cancellation notice indicating that a particular classification made in a prior frame included in the series of past frames was incorrect;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the particular classification made in the prior frame; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises;

returning to the prior frame in which the particular classification has been updated; and

replaying the series of past frames from the prior frame to the current frame in view of the update to the particular classification.

4. The method of claim 1:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises detecting that the previous contact was classified as an intended touch input and should have been classified as an incidental touch input;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the classification of the previous contact from an intended touch input to an incidental touch input; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an incidental touch input.

5. The method of claim 4, wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an incidental touch input comprises reversing at least one action performed responsive to the previous contact.

6. The method of claim 5, wherein reversing at least one action performed responsive to the previous contact comprises reversing at least one of a movement of an object performed responsive to the previous contact, zooming of an object performed responsive to the previous contact, writing on an object performed responsive to the previous contact, and resizing of an object performed responsive to the previous contact.

7. The method of claim 4, wherein detecting that the previous contact was classified as an intended touch input and should have been classified as an incidental touch input comprises:

identifying one or more current contacts that have been classified as incidental touch input, the one or more current contacts occurring after the previous contact;

analyzing a relationship between the one or more current contacts that have been classified as incidental touch input to the previous contact that was classified as an intended touch input;

based on the analysis, determining that the one or more current contacts are sufficiently related to the previous contact; and

determining that the previous contact should have been classified as an incidental touch input based on the determination that the one or more current contacts are sufficiently related to the previous contact.

8. The method of claim 1:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises detecting that the previous contact was classified as an incidental touch input and should have been classified as an intended touch input;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the classification of the previous contact from an incidental touch input to an intended touch input; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an intended touch input.

9. The method of claim 8, wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an intended touch input comprises performing at least one action that should have occurred responsive to the previous contact.

10. The method of claim 8, wherein detecting that the previous contact was classified as an incidental touch input and should have been classified as an intended touch input comprises:

identifying one or more current contacts that have been classified as intended touch input, the one or more current contacts occurring after the previous contact;

analyzing a relationship between the one or more current contacts that have been classified as intended touch input to the previous contact that was classified as an incidental touch input;

based on the analysis, determining that the one or more current contacts are sufficiently related to the previous contact; and

determining that the previous contact should have been classified as an intended touch input based on the determination that the one or more current contacts are sufficiently related to the previous contact.

11. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

maintaining a history of prior state information related to a touch sensitive display;

detecting that a previous contact with the touch sensitive display was incorrectly classified;

updating a classification of the previous contact based on the detection that the previous contact was incorrectly classified; and

rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified based on the history of prior state information and the updated classification of the previous contact.

12. The system of claim 11, wherein maintaining the history of prior state information related to the touch sensitive display comprises maintaining state information for a series of past frames, the state information comprising, for each frame, locations of intended touches detected, locations of incidental touches detected, locations of stylus inputs detected, and locations of objects displayed on the touch sensitive display.

13. The system of claim 12:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises receiving, at a current frame, a cancellation notice indicating that a particular classification made in a prior frame included in the series of past frames was incorrect;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the particular classification made in the prior frame; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises;

returning to the prior frame in which the particular classification has been updated; and

replaying the series of past frames from the prior frame to the current frame in view of the update to the particular classification.

14. The system of claim 11:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises detecting that the previous contact was classified as an intended touch input and should have been classified as an incidental touch input;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the classification of the previous contact from an intended touch input to an incidental touch input; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an incidental touch input.

15. The system of claim 14, wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an incidental touch input comprises reversing at least one action performed responsive to the previous contact.

16. The system of claim 15, wherein reversing at least one action performed responsive to the previous contact comprises reversing at least one of a movement of an object performed responsive to the previous contact, zooming of an object performed responsive to the previous contact, writing on an object performed responsive to the previous contact, and resizing of an object performed responsive to the previous contact.

17. The system of claim 14, wherein detecting that the previous contact was classified as an intended touch input and should have been classified as an incidental touch input comprises:

identifying one or more current contacts that have been classified as incidental touch input, the one or more current contacts occurring after the previous contact;

analyzing a relationship between the one or more current contacts that have been classified as incidental touch input to the previous contact that was classified as an intended touch input;

based on the analysis, determining that the one or more current contacts are sufficiently related to the previous contact; and

determining that the previous contact should have been classified as an incidental touch input based on the determination that the one or more current contacts are sufficiently related to the previous contact.

18. The system of claim 11:

wherein detecting that the previous contact with the touch sensitive display was incorrectly classified comprises detecting that the previous contact was classified as an incidental touch input and should have been classified as an intended touch input;

wherein updating the classification of the previous contact based on the detection that the previous contact was incorrectly classified comprises updating the classification of the previous contact from an incidental touch input to an intended touch input; and

wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been correctly classified comprises rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an intended touch input.

19. The system of claim 18, wherein rewinding a state of the touch sensitive display to reflect a state that would have resulted had the previous contact been classified as an intended touch input comprises performing at least one action that should have occurred responsive to the previous contact.

20. The system of claim 18, wherein detecting that the previous contact was classified as an incidental touch input and should have been classified as an intended touch input comprises:

identifying one or more current contacts that have been classified as intended touch input, the one or more current contacts occurring after the previous contact;

analyzing a relationship between the one or more current contacts that have been classified as intended touch input to the previous contact that was classified as an incidental touch input;

based on the analysis, determining that the one or more current contacts are sufficiently related to the previous contact; and

determining that the previous contact should have been classified as an intended touch input based on the determination that the one or more current contacts are sufficiently related to the previous contact.