Multi-touch manipulation of application objects

• US 8,884,907 B2
• Filed: 06/17/2013
• Issued: 11/11/2014
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan, Torset, Todd A., Geidl, Erik M., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US13/919,873
• Publication Number:
  US 20130278529A1
• Time in Patent Office: 512 Days
• Field of Search: 345156-174, 345/428, 345/650, 345/660, 715/205, 715/234, 715/863, 715/864, 178/18.05, 382/188
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

The manipulation system described herein provides a common platform and application-programming interface (API) for applications to communicate with various multi-touch hardware devices, and facilitates the interpretation of multi-touch input as one or more manipulations. Manipulations map more directly to user intentions than do individual touch inputs and add support for basic transformation of objects using multiple touch contacts. An application can use manipulations to support rotating, resizing, and translating multiple objects at the same time. The manipulation system outputs two-dimensional (2D) affine transforms that contain rotation, scale, and translation information. Thus, using the manipulation system the application author can focus more on building touch-capable applications and let the manipulation system handle the underlying transformations and communication with the multi-touch hardware.

Claims

1. A method of interpreting low-level touch inputs into corresponding manipulation events, the method performed on a computing device having a touch-sensitive input surface and a display, the method comprising:

receiving low-level touch inputs that correspond to touches of the touch-sensitive input surface, the low-level touch inputs comprising locations of contacts with the touch-sensitive input surface, the low-level touch inputs received by applications executing on the computing device, the low-level touch inputs comprising two-dimensional input points corresponding to touches of the touch-sensitive input surface;

sending the received low-level touch inputs to a manipulation system by use of a manipulation API (application programming interface), the manipulation system interpreting the received low-level touch inputs into corresponding manipulation events, wherein the manipulation system is invocable by arbitrary applications on the computing device via the manipulation API, the low-level touch inputs sent by the applications invoking the API;

receiving, at the applications, the manipulation events, the manipulation events comprising geometric transforms produced by the manipulation system according to the low-level inputs; and

handling, by the applications, the received manipulation events.

2. A method according to claim 1, wherein the manipulation API instantiates software-based manipulation processors according to the low-level touch inputs.

3. A method according to claim 1, further comprising:

through the manipulation API, informing the manipulation system of an association between the low-level touch inputs and an object.

4. A method according to claim 3, wherein the interpreting the low-level touch inputs into manipulation events is based at least in part on the association between certain of the low-level touch inputs and the object.

5. A method according to claim 4, further comprising:

informing the manipulation system of a locational attribute of the object by the application passing the locational attribute to the manipulation system via the manipulation API.

6. A method according to claim 5, wherein the locational attribute comprises a center of the object or a reference point of the object.

7. A method according to claim 1, further comprising managing a set of contact identifiers by the manipulation system.

8. A method according to claim 7, wherein the managing a set of contact identifiers comprises associating the contact identifiers with low-level touch inputs.

9. A method according to claim 8, wherein the contact identifiers are used for interpreting the low-level touch inputs into manipulation events.

10. A device for handling touch input from multi-touch hardware, the device configures such that when it is operating it comprises:

a plurality of modules that each receive low-level inputs for respective applications on the computer device, the low-level inputs originating from a contact-sensitive input device, each module translating portions of the low-level inputs to manipulations that are then received by the respective applications, the manipulations comprising geometric transforms, the low-level touch inputs comprising two-dimensional inputs; and

wherein the low-level inputs, which correspond to user interactions, are received by the modules and the applications receive corresponding manipulations translated by the modules that the applications use to display graphic objects, the low-level inputs having been produced responsive to a user contacting the contact-sensitive input device to interact with the applications.

11. A device according to claim 10, further comprising an API through which the modules were instantiated by the applications.

12. A device according to claim 10, wherein the devices, via the API, provides to the applications respective handles for the respective modules and the applications use the handles to provide the low-level inputs to the modules.

13. A device according to claim 10, wherein each module comprises a history of low-level inputs received from its corresponding application, and the history is used to interpret other low-level inputs into manipulations.

14. A device according to claim 13, wherein the device automatically identifies new and ending contacts and performs the translating accordingly.

15. A device according to claim 10, wherein the device determines and manages information indicating which low-level inputs are associated with which contacts with the multi-touch hardware.

16. A computer-readable storage device storing information that when processed causes a computing device to perform a process, the process comprising:

receiving inputs, each input having been inputted by a user and each input comprising a two-dimensional location and having a corresponding occurrence time, the inputs having been inputted by the user using first and second physical contacts contacting an input device of the computing device, the inputs comprising first inputs inputted by the first physical contact and second inputs inputted by the second physical contact;

handling the inputs at an application executing on the computing device by passing the inputs to a manipulation processing system executing on the computing device;

receiving the inputs by the manipulation processing system, the manipulation processing system determining from the inputs which of the inputs were inputted with the first physical contact and which of the inputs were inputted with the second physical contact, wherein the manipulation processing system is available to be invoked by arbitrary applications executing on the computing device that have inputs to be processed.

17. A computer-readable storage device according to claim 16, the process further comprising managing contact identifiers corresponding to the physical contacts.

18. A computer-readable storage device according to claim 17, the process further comprising tracking movements associated with the contact identifiers and using the movements to interpret the determine geometric manipulations.

19. A computer-readable storage device according to claim 18, the process further comprising associating, by the application, the inputs with an object created and managed by the application.

20. A computer-readable storage device according to claim 19, the process further comprising:

receiving, by the application, manipulations, the manipulations comprising translations, rotations, scalings, and/or combinations thereof, according to the inputs and the contact identifiers; and

displaying the object according to the manipulations.