Multi-touch object inertia simulation

• US 9,189,096 B2
• Filed: 06/17/2013
• Issued: 11/17/2015
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan, Torset, Todd A., Sykes, Kenneth W., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US13/919,957
• Publication Number:
  US 20130285961A1
• Time in Patent Office: 883 Days
• Field of Search: 345156-174, 345/428, 345/650, 345/660, 715/205, 715/234, 715/863, 715/864, 178/18.05, 382/188
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The inertia system provides a common platform and application-programming interface (API) for applications to extend the input received from various multi-touch hardware devices to simulate real-world behavior of application objects. To move naturally, application objects should exhibit physical characteristics such as elasticity and deceleration. When a user lifts all contacts from an object, the inertia system provides additional manipulation events to the application so that the application can handle the events as if the user was still moving the object with touch. The inertia system generates the events based on a simulation of the behavior of the objects. If the user moves an object into another object, the inertia system simulates the boundary characteristics of the objects. Thus, the inertia system provides more realistic movement for application objects manipulated using multi-touch hardware and the API provides a consistent feel to manipulations across applications.

Claims

1. A computing device for handling inputs from multi-input hardware, the multi-input hardware allowing a user to input the inputs by manipulating one or more physical input objects sensed by the multi-input hardware, the system comprising:

a module available for manipulation processing of inputs for arbitrary applications running on the computer device, wherein the inputs are passed to the module, the inputs having been inputted by the multi-input hardware sensing changing locations of a physical input object being moved by the user, wherein the module translates the inputs to geometric manipulations that are received by an application associated with the inputs, the application displaying a graphic object on the display by applying the manipulations to the graphic object to change a location and/or an appearance of the graphic object;

when the multi-input hardware stops providing the inputs corresponding to the physical input object, the module stops generating the manipulations and generates simulated geometric manipulations based on the inputs.

2. A computing device according to claim 1, wherein the module simulates decreasing inertia according to a state of the graphic object before the inputs stop being provided.

3. A computing device according to claim 2, wherein the state comprises a velocity and/or direction of the application object.

4. A computing device according to claim 1, wherein the inputs comprise respective positions and respective times of the physical input object being sensed by the multi-input hardware.

5. A computing device according to claim 1, wherein the manipulations and the simulated manipulations have a same format and a same communication channel to the application.

6. A computing device according to claim 1, wherein the physical input object comprises a stylus or a human digit.

7. A computing device according to claim 1, wherein the application receives the inputs and passes the objects to the module.

8. A computing device according to claim 7, wherein the application provides to the module information associating at least some of the inputs with the graphical object.

9. A computing device according to claim 8, wherein the module uses the module uses the information to compute the manipulations and to compute the simulated manipulations.

10. A device for handling touch input from multi-touch hardware, the device comprising:

a hardware interface that when the computer system is operating will communicate with the multi-touch hardware to receive touch contact information and movements of contacts;

one or more manipulation processors configured to manage interpretation of movement of each contact associated with a particular application object;

an input transformation component configured to interpret a meaning of received movements of various contacts to produce manipulations of application objects, wherein the application applies the manipulations to the application object to move the application object;

a simulation component configured to simulate continued movement of the application object after a user stops touching the object; and

an application interface configured to communicate with the application to receive contact movement information and provide manipulation transforms to the application.

11. A device according to claim 10 wherein the simulation component is further configured to receive initialization parameters from an application through the application interface.

12. A device according to claim 11 wherein the initialization parameters are provided by a manipulation processor that was managing the application object before the user stopped touching the object.

13. A device according to claim 10 wherein the simulation component is further configured to receive boundary constraints of the application object.

14. A device according to claim 10 wherein the simulation component is further configured to apply laws of physics to determine the continued movement of the object based on a past velocity and direction of movement of the object.

15. A device according to claim 10 wherein the input transformation component and simulation component generate events in a similar format so that the application can handle events from both components similarly.

16. A method performed by a computing device, the method comprising:

executing a manipulation system available to process inputs for arbitrary applications on the computing device, each input comprising a location and a corresponding input time captured by an input device capable of concurrently sensing locations of two or more physical objects, the manipulation system interpreting the inputs to generate manipulations that are received and handled by the applications, each manipulation comprising a geometric transformation and/or translation, wherein one of the applications applies manipulations received thereby to a graphic object managed by the application to graphically transform and/or translate the graphic object; and

determining that input from a physical object has ended, and in response the manipulation system, according to inputs of the physical object previously received by the manipulation system before the determining, automatically generating simulated manipulations that are received by the one of the applications, the simulated manipulations comprising respective geometric transformations and/or translations that are applied by the application to the graphic object to simulate continuation of the transformation and/or translation of the graphic object.

17. A method according to claim 16, wherein the simulated manipulations are computed according to one or more algorithms that simulate physical motion of physical objects.

18. A method according to claim 16, further comprising displaying a graphic object by the one of the applications on a display of the computing device, the graphic object being displayed with geometric transformations and/or translations corresponding to manipulations received by the one of the applications.

19. A method according to claim 18, further comprising, after the ending of the input from the physical object, further displaying the graphic object, by the application, with geometric transformation and/or translation according to the simulated manipulations.

20. A method according to claim 19, wherein the graphic object is displayed such that when the physical object is providing inputs the graphic object transforms and/or translates in correspondence with movements of the physical object and when the physical object stops providing inputs the graphic object transforms and/or translates in accordance with movement of the physical object before the physical object stopped providing inputs.

21. A method performed by a computing device comprising storage storing an operating system, a processor, a display, and a touch input surface, the method comprising:

executing an application displaying a user interface element, wherein a user of the computing device moves a finger touching the touch input surface to interactively move the user interface element on the display, the moving implemented by the operating system passing touch inputs to the application and the application using the touch inputs to obtain computed manipulations that the application applies to the user interface element to cause the displayed movement thereof;

responsive to determining that the user has stopped touching the touch input surface with the finger, notifying an inertia engine to simulate inertial movement according to at least some of the touch inputs;

after the finger has stopped touching the touch input surface, receiving, by the application, simulated movements, and applying, by the application, the simulated movements to the user interface element to simulate inertial movement of the user interface object.

22. A method according to claim 21, wherein the simulated inertial movement, when applied to the user interface element, causes the user interface element to be displayed with movement that appears to be a continuation of movement of the user interface element just prior to when the user stopped touching the touch input surface.