Multi-touch object inertia simulation

• US 10,503,395 B2
• Filed: 01/04/2018
• Issued: 12/10/2019
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan, Torset, Todd A., Sykes, Kenneth W., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US15/862,127
• Publication Number:
  US 20180129410A1
• Time in Patent Office: 705 Days
• Field of Search: 345156-174, 345428, 345650, 345660, 715205, 715234, 715863, 715864, 178 1805, 382188
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The inertia system provides a common platform and application-programming interface (API) for applications to extend the input received from various multi-touch hardware devices to simulate real-world behavior of application objects. To move naturally, application objects should exhibit physical characteristics such as elasticity and deceleration. When a user lifts all contacts from an object, the inertia system provides additional manipulation events to the application so that the application can handle the events as if the user was still moving the object with touch. The inertia system generates the events based on a simulation of the behavior of the objects. If the user moves an object into another object, the inertia system simulates the boundary characteristics of the objects. Thus, the inertia system provides more realistic movement for application objects manipulated using multi-touch hardware and the API provides a consistent feel to manipulations across applications.

Claims

1. A method performed by a computing device, the computing device comprising processing hardware, storage hardware, and an input device, the input device configured to detect multiple concurrent input points, the method comprising:

executing an operating system, the operating system controlling execution of an application by the processing hardware, the application comprising a graphic object, a manipulation processor, and an inertia simulator;

receiving, from the operating system, inputs, the inputs comprising two-dimensional location information corresponding to respective input points detected by the input device and corresponding to input inputted by physical contact with the input device;

in accordance with logic of the application that links the manipulation processor with the inputs, passing the inputs from the application to the manipulation processor until the inputs are determined to have terminated;

receiving, by the application, manipulation notifications generated by the manipulation processor, each manipulation notification comprising a respective geometric transform derived by the manipulation processor interpreting the two-dimensional location information of the inputs;

based on a signal sent from the manipulation processor, the signal sent based on the manipulation processor determining from the inputs that the stroke has terminated, initializing the inertia simulator with initialization information that is based on motion information provided by the manipulation processor;

applying, by the application, the geometric transforms of the manipulation notifications to the graphic object displayed by the computing device, wherein the application of the geometric transforms results in display of corresponding graphical transformation of the graphic object on the display, wherein, before the inputs terminate, the graphical transformations correspond to the inputs; and

after the initializing, displaying the graphic object according to inertial movements computed by the inertia simulator based on the initialization information.

2. A method according to claim 1, wherein the computing device comprises an application programming interface (API) accessible to arbitrary applications executed by the operating system, and wherein the application uses the API to instantiate the manipulation processor.

3. A method according to claim 1, wherein the input device comprises a touch sensitive surface, wherein input points comprise respective identifiers and timestamps, each identifier identifying a corresponding input point, wherein the input points correspond to contacts of digits with the touch sensitive surface, wherein inputs inputted by a same input point respectively comprise a same identifier of the same input point, and wherein the manipulation processor uses the identifiers and the timestamps to derive the geometric transforms.

4. A method according to claim 1, wherein the inertia simulator generates other manipulation notifications, wherein both the manipulation notifications generated based on the inputs and the manipulation notifications generated by the inertia simulator are received by the application through a same software interface, and wherein both the manipulation notifications generated based on the inputs and the manipulation notifications generated by the inertia simulator comprise a same notification type.

5. A method according to claim 1, wherein the application comprises a plurality of graphical objects, including the graphical object, wherein the application comprises a plurality of manipulation processors, including the manipulation processor, and wherein logic of the application determines which of the inputs are handled by which of the manipulation processors.

6. A method according to claim 1, wherein the motion information corresponds to physical motion detected by the input device in correspondence with termination of the stroke.

7. A method according to claim 1, wherein the initialization information is obtained from the manipulation processor by the inertia simulator.

8. A method according to claim 1, wherein the initializing comprises setting an initial position property of the inertia simulator to an initial position value and/or setting an initial velocity property of the inertia simulator to an initial velocity value, wherein a first geometric transform derived by the inertia simulator is based on the initial position value and/or the initial velocity value.

9. A method according to claim 8, wherein the initial position value depends on a last geometric transform derived by the manipulation processor.

10. A method according to claim 1, further comprising repeatedly invoking, by the application, a method or property of the inertia simulator to obtain the geometric transforms.

11. A computing device comprising:

a contact-sensitive surface;

a display;

processing hardware;

storage hardware storing instructions executable by the processing hardware of the computing device, the instructions configured to cause the processing hardware to perform a process, the process comprising;

providing a first module and a second module for an application executing on the computing device, the first module and the second module invoked through one or more APIs that are accessible to applications executing on the computing device; and

computing, by the first and second modules, affine transform events and manipulating, by the application, a graphic object displayed by the display according to the affine transform events, wherein some of the affine transform events are computed by the first module receiving and tracking locations of an input point on the contact-sensitive surface and wherein other of the affine transform events are computed by the second module simulating inertial affine transforms for the graphic object, each affine transform event comprising a translation and/or rotation and/or scale transform, wherein the locations are provided by the contact-sensitive surface sensing the input point.

12. A computing device according to claim 11, wherein the application is configured to listen for an event or message indicating that the input point has ended and in response cause the second module to initialize.

13. A computing device according to claim 11, wherein the transform events computed by the second module are received through calls to a corresponding function thereof.

14. A computing device according to claim 13, wherein the calls are made by the application, and wherein the function comprises a processing function that performs the simulating of the inertial affine transforms.

15. A computing device according to claim 11, wherein the application comprises application logic configured to determine that the locations are associated with the graphic object and based thereon cause the first module to receive the points.

16. A computing device according to claim 11, wherein the affine transform events of the second module are received through one or more calls to a function of the second module.

17. A computing device comprising:

a contact-sensitive surface;

a display;

processing hardware;

storage hardware storing instructions executable by the processing hardware of the computing device, the instructions configured to cause the processing hardware to perform a process, the process comprising;

providing a first module and a second module for an application executing on the computing device, the first module and the second module invoked through one or more APIs that are accessible to applications executing on the computing device; and

computing, for the application, affine transform events and manipulating, by the application, a graphic object displayed by the display according to the affine transform events, wherein some of the affine transform events are computed by the first module receiving and tracking locations of an input point on the contact-sensitive surface and wherein other of the affine transform events are computed by the second module simulating inertial affine transforms for the graphic object, each affine transform event comprising a translation and/or rotation and/or scale transform, wherein the locations are provided by the contact-sensitive surface sensing the input point.

18. A computing device according to claim 17, the process further comprising providing a software interface, the application receiving at least the affine transform events of the first module through the software interface.

19. A computing device according to claim 17, wherein the locations are received in messages generated by an operating system and/or a common platform of the computing device, the operating system configured to generate the messages for a mouse input device and for the contact-sensitive input surface.

20. A computing device according to claim 17, the process further comprising linking the first and second modules which causes the second module to be seeded with an initial velocity received from the first module or computed from a velocity provided by the first module.