CONTACT GROUPING AND GESTURE RECOGNITION FOR SURFACE COMPUTING

• US 20100283739A1
• Filed: 05/05/2009
• Published: 11/11/2010
• Est. Priority Date: 05/05/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Guanggen, Zhang, Kuo, Sun, Xueqing, Huang, KeCheng
• Granted Patent: US 8,427,440 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Tools and techniques for contact grouping and gesture recognition for surface computing are provided. These tools receive, from a finger grouping subsystem, data representing velocity parameters associated with contact points, with the contact points representing physical actions taken by users in interacting with software applications presented on a surface computing system. These tools also identify gestures corresponding to the physical actions by analyzing the velocity parameters, and provide indications of the identified gestures to the software applications.

Claims

1. Apparatus comprising at least one physical computer-readable storage medium having stored thereon computer-executable instructions that, when loaded into at least one hardware processor and executed, transform the hardware processor to perform the following:

receive at least one indication of a physical action directed by at least one user toward a surface computing system, wherein the physical action as represented by a plurality of contact points, wherein the contact points represent physical actions taken by the user in interacting with a software application presented on the surface computing system;

associate respective velocity parameters with the contact points;

select at least first and second contact points to maximize a differential between velocity parameters associated with the selected contact inputs;

associate a first one of the selected contact points with a first data structure;

associate a second one of the selected contact points with a second data structure;

assign a velocity parameter associated with the first contact point to the first data structure;

associate a first remainder of the contact points with the first data structure; and

associate a second remainder of the contact points with the second data structure.

2. The apparatus of claim 1, wherein the instructions to receive at least one indication of a physical action include instructions to receive at least one indication that the user has performed a physical action with a plurality of fingers, and wherein the contact points represent respective physical actions performed by the fingers.

3. The apparatus of claim 1, wherein the instructions to receive at least one indication of a physical action include instructions to receive an indication that the user has performed a physical action with a plurality of hands and a plurality of fingers, wherein the contact points represent respective physical actions taken by different fingers on different hands.

4. The apparatus of claim 1, wherein the instructions to associate respective velocity parameters include instructions to associate parameters that indicate a speed and direction associated with the physical action, with respect to the surface computing system.

5. The apparatus of claim 1, further comprising instructions to calculate differentials between velocity parameters associated with pairs of the contact points, and wherein the instructions to select at least first and second contact points include instructions to identify a pair of contact points having a greatest differential in velocity.

6. The apparatus of claim 1, wherein the instructions to associate a first remainder of the contact points include instructions to select one of the contact points other than the first and second contact points, further comprising instructions to calculate velocity differentials between the selected other one of the contact points and the first and second data structures, and further comprising instructions to associate the selected other one of the contact points with the first or second data structure, so as to minimize the velocity differential therebetween

7. The apparatus of claim 1, further comprising instructions to recalculate the velocity parameters associated with the first and second data structures in response to associate in the first and second remainders of the contact points with the first and second data structures.

8. The apparatus of claim 7, wherein the instructions to recalculate the velocity parameters of the first and second data structures include instructions to calculate a velocity parameter of the first data structure by averaging the velocity parameters of the contact points assigned to the first data structure, and include instructions to calculate a velocity parameter of the second data structure by averaging the velocity parameters of the contact points assigned to second first data structure.

9. The apparatus of claim 8, further comprising instructions to reassign the contact points to the first and second data structures in response to recalculating the velocity parameters associated with the first and second data structures.

10. The apparatus of claim 9, wherein the instructions to reassign the contact points include instructions to calculate respective velocity differentials between the contact points and the first and second data structures, and further comprising instructions to reassign the contact points to the first or second data structures to minimize the velocity differential therebetween.

11. The apparatus of claim 10, further comprising instructions to discard a contact point reassigned to the first data structure, wherein the velocity parameter of the discarded contact point deviates a maximum value as compared to other contact points reassigned to the first data structure, and further comprising instructions to discard a further contact point reassigned to the second data structure, wherein the velocity parameter of the further discarded contact point deviates a maximum value as compared to other contact points reassigned to the second data structure.

12. Apparatus comprising at least one physical computer-readable storage medium having stored thereon computer-executable instructions that, when loaded into at least one hardware processor and executed, transform the hardware processor to perform the following:

receive, from a finger grouping subsystem, data representing a velocity parameter associated with at least one contact point, wherein the contact point represents a physical action taken by at least one user in interacting with a software application presented on a surface computing system;

identify a gesture corresponding to the physical action by analyzing the velocity parameter; and

provide an indication of the identified gesture to the software application.

13. The apparatus of claim 12, wherein the instructions to identify a gesture include instructions to identify a pan gesture.

14. The apparatus of claim 12, further comprising instructions to receive data representing a further velocity associated with at least a further contact point associated with the physical action, and wherein the instructions to identify a gesture include instructions to analyze the velocity in the further velocity relative to one another.

15. The apparatus of claim 14, wherein the instructions to identify a gesture include instructions to identify a pan, zoom, or rotate gesture based on analyzing the velocity and the further velocity.

16. The apparatus of claim 12, further comprising instructions to apply thresholding to the identified gesture.

17. The apparatus of claim 16, further comprising instructions to recalculate the identified gesture after applying the thresholding.

18. A surface computing system comprising:

at least one instance of processing hardware;

at least one bus system coupled to communicate with the processing hardware;

at least one computer-readable storage medium coupled to communicate with the processing hardware via the bus system, wherein the storage medium is encoded with computer-executable instructions that, when loaded into the processing hardware, transform the processing hardware to

receive at least one indication of a physical action directed by at least one user toward the surface computing system, wherein the physical action is represented by a plurality of contact points that correspond to a plurality of fingers associated with the user;

associate respective velocity and position parameters with the contact points;

select two of the contact points having a maximum velocity differential therebetween;

associate a first one of the selected contact points with a first virtual hand structure;

associate a second one of the selected contact points with a second virtual hand structure;

assign the velocity and position parameters of the first selected contact point to the first virtual hand structure;

assign the velocity and position parameters of the second selected contact point to the second virtual hand structure; and

calculate a gesture represented by the physical action, based on relative movement between the first and second virtual hand structures as indicated by analysis of the first and second contact points.

19. The surface computing system of claim 18, wherein the instructions to calculate a gesture include instructions to infer that the physical action represents a pan command, zoom command, or rotate command issued by the user to the software application.

20. The surface computing system of claim 18, wherein the instructions to calculate a gesture include instructions to analyze movement represented in the first virtual hand structure relative to movement represented in the second virtual hand structure.