Multi-touch object inertia simulation

• US 8,477,103 B2
• Filed: 10/26/2008
• Issued: 07/02/2013
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan D., Torset, Todd A., Sykes, Kenneth W., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US12/258,439
• Publication Number:
  US 20100103118A1
• Time in Patent Office: 1,710 Days
• Field of Search: 345156-184, 345/419, 345/428, 345/650, 345/660, 715/204, 715/205, 715/234, 715/753, 715/863, 715/864, 178/18.05, 178/18.01, 348/14.12, 348/E7.078, 382/106, 382/188, 382/193
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The inertia system provides a common platform and application-programming interface (API) for applications to extend the input received from various multi-touch hardware devices to simulate real-world behavior of application objects. To move naturally, application objects should exhibit physical characteristics such as elasticity and deceleration. When a user lifts all contacts from an object, the inertia system provides additional manipulation events to the application so that the application can handle the events as if the user was still moving the object with touch. The inertia system generates the events based on a simulation of the behavior of the objects. If the user moves an object into another object, the inertia system simulates the boundary characteristics of the objects. Thus, the inertia system provides more realistic movement for application objects manipulated using multi-touch hardware and the API provides a consistent feel to manipulations across applications.

Claims

1. A method performed on a computing device comprising a touch sensitive input surface and a display, the method for providing realistic movement on the display of objects manipulated using multi-touch input via the touch sensitive surface, the method comprising:

determining that a user has released an application object of an application executing on the computing device by removing one or more contacts from a multi-touch input device, the application object displayed on the display;

after the determining, invoking an inertia API (application programming interface) executing on the computing device to process a simulation of movement of the application object for a current inertia processing period, wherein the inertia API provides an application-independent platform invocable by arbitrary applications on the computing device, the inertia API simulating inertial object movements independent of a type of the application object;

receiving, by the application, an inertia event generated by the inertia API after the determining, wherein the inertia event describes a manipulation of the object based on simulated inertia; and

handling the received inertia event based on an effect of the manipulation in a context of the application by modifying the application object and displaying the application object on the display accordingly.

2. The method of claim 1 further comprising, after determining that the user released the object, initializing the inertia system by passing a reference to a manipulation processor that was handling the movement of the object before the user released the object.

3. The method of claim 1 further comprising, after determining that the user released the object, initializing the inertia system by passing one or more parameters describing a state of the object when the user released the object.

4. The method of claim 1 wherein receiving the inertia event comprises receiving information describing a 2D affine transform of the application object.

5. The method of claim 1 wherein the manipulation described by the inertia event includes at least one of a rotation manipulation, a translation manipulation, and a scaling manipulation.

6. The method of claim 1 wherein the received inertia event is a rotation and wherein handling the received inertia event comprises rotating the application object on a display.

7. The method of claim 1 further comprising setting a timer that determines a length of the inertia processing period, and wherein the application invokes the inertia API at each firing of the timer.

8. The method of claim 7 further comprising receiving an indication from the inertia API that the simulation is complete, and expiring the timer when the indication is received.

9. The method of claim 1 wherein receiving the inertia event comprises receiving a notification through a COM event interface.

10. A computer-readable storage medium encoded with instructions for controlling a computing device comprised of a display and a touch-sensitive input surface to simulate movement, on the display, of an application object previously moved by multi-touch input via the touch-sensitive input surface, the application object is part of an application executing on the computing device, by a method comprising:

receiving one or more initial simulation parameters that provide a state of the application object corresponding to a user release of the application object by ceasing contact with the touch-sensitive input surface;

initializing a simulation engine that performs calculations to simulate inertia of the application object based on the initial simulation parameters;

receiving an indication of a current simulation period;

simulating, by the simulation engine, inertial movement of the application object based on the initial parameters, any previous inertial simulating by the simulation engine, and the time passed since any previous simulation period; and

firing an inertia event to send transform information describing a current movement of the application object to the application, the application object displayed on the display according to the transform information.

11. The computer-readable medium of claim 10 wherein the simulation engine simulates realistic deceleration behavior for the application object after it was set in motion by the user touching the object.

12. The computer-readable medium of claim 10 wherein the simulation engine simulates realistic elasticity behavior for the application object when the application object overlaps with another element on a display.

13. The computer-readable medium of claim 10 wherein receiving the indication that the current simulation period is due comprises receiving a call from the application to a simulation processing function.

14. The computer-readable medium of claim 10 further comprising determining whether the simulation is complete and if the simulation is complete, informing the application that the simulation is complete.

15. A computer device for handling touch input from multi-touch hardware, the system comprising:

a module that receives low-level inputs, for arbitrary applications, from a multi-touch input device, translates the low-level inputs to manipulations received by the arbitrary applications, wherein at least some of the manipulations are based on respective pluralities of the low-level inputs, the manipulations comprising two-dimensional affine transformations;

an application comprising a graphic object displayed on the display, the graphic object interacted with by a user contacting the multi-touch input device such that corresponding low-level inputs directed to the graphic object are received by the module and the application receives corresponding manipulations generated by the module that the application uses to display the graphic object, wherein when the user stops contacting the multi-touch input device the module continues to generate manipulations for the graphic object by simulating decreasing inertia of the object.

16. A computing device according to claim 15, wherein module simulates the decreasing inertia according to a state of the object before the user stops contacting the multi-touch input device.

17. A computing device according to claim 16, wherein the state comprises a velocity and direction of the application object.

18. A computing device according to claim 15, wherein the low-level inputs comprise positions at which the user is contacting the multi-touch input device.

19. A computing device according to claim 15, wherein the module causes the application to receive a notice that the graphic object has stopped.

20. A computing device according to claim 15, wherein the manipulations before and after the user stops contacting the multi-touch input device have a same format and a same communication channel to the application.