Multi-touch object inertia simulation

• US 9,898,190 B2
• Filed: 01/20/2017
• Issued: 02/20/2018
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan, Torset, Todd A., Sykes, Kenneth W., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US15/411,617
• Publication Number:
  US 20170168708A1
• Time in Patent Office: 396 Days
• Field of Search: 345156-184, 345428, 345650, 345660, 715205, 715234, 715863, 715864, 178 1805, 382188
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The inertia system provides a common platform and application-programming interface (API) for applications to extend the input received from various multi-touch hardware devices to simulate real-world behavior of application objects. To move naturally, application objects should exhibit physical characteristics such as elasticity and deceleration. When a user lifts all contacts from an object, the inertia system provides additional manipulation events to the application so that the application can handle the events as if the user was still moving the object with touch. The inertia system generates the events based on a simulation of the behavior of the objects. If the user moves an object into another object, the inertia system simulates the boundary characteristics of the objects. Thus, the inertia system provides more realistic movement for application objects manipulated using multi-touch hardware and the API provides a consistent feel to manipulations across applications.

Claims

1. A method performed by a computing device, the computing device comprising processing hardware, storage hardware, a display, and an input device, the input device configured to detect multiple concurrent input points, the method comprising:

executing an operating system, the operating system controlling execution of an application by the processing hardware, the application comprising a graphic object, a manipulation processor, and an inertia simulator;

receiving, from the operating system, inputs, the inputs comprising two-dimensional location information corresponding to respective input points detected by the input device and corresponding to a stroke inputted via physical contact with the input device;

in accordance with logic of the application that links the manipulation processor with the inputs, passing the inputs from the application to the manipulation processor while the stroke is inputted via physical contact with the input device;

receiving, by the application, manipulation notifications generated by the manipulation processor, each manipulation notification comprising a respective transform derived by the manipulation processor interpreting the two-dimensional location information of the inputs;

initializing, based at least on a signal sent from the manipulation processor subsequent to the stroke no longer being inputted via physical contact with the input device, the signal including information associated with a motion parameter, a simulation of inertia associated with continuing a movement of the graphic object subsequent to the stroke no longer being inputted via physical contact with the input device, the simulation of inertia based at least on the motion parameter;

applying, by the application, the transforms of the manipulation notifications to the graphic object displayed by the computing device, wherein each application of a transform results in display of a corresponding graphical transformation of the graphic object on the display, wherein, while the stroke is inputted via physical contact with the input device, the graphical transformations correspond to the inputs; and

continuing the movement of the graphic object according to the simulation of inertia.

2. A method according to claim 1, wherein the display comprises the input device, and wherein the input device comprises a touch sensing element that detects the input points.

3. A method according to claim 1, wherein the transforms comprise at least a translation component.

4. A method according to claim 1, wherein the signals comprise events.

5. A method according to claim 1, wherein the motion parameter is derived from or comprises a transform of a manipulation notification.

6. A method according to claim 1, wherein the graphic object moves on the display in correspondence with the stroke while the stroke is being inputted and moves on the display in correspondence with the simulation of inertia subsequent to the stroke no longer being inputted via physical contact with the input device.

7. An apparatus comprising:

a display;

an input device configured to detect multiple concurrent input points;

processing hardware; and

storage hardware storing an operating system and an application, the application comprised of a graphics object, a manipulation processor, and an inertia simulator;

the storage hardware further storing instructions configured to cause the processing hardware to perform a process comprising;

executing the operating system and the application, the operating system controlling execution of the application by the processing hardware;

receiving, from the operating system, inputs, the inputs comprising two-dimensional location information corresponding to respective input points detected by the input device and corresponding to a stroke inputted via physical contact with the input device;

in accordance with logic of the application that links the manipulation processor with the inputs, passing the inputs from the application to the manipulation processor while the stroke is inputted via physical contact with the input device;

receiving, by the application, manipulation notifications generated by the manipulation processor, each manipulation notification comprising a respective transform provided by the manipulation processor interpreting the two-dimensional location information of the inputs;

initializing, based at least on a signal initiated by the manipulation processor subsequent to the stroke no longer being inputted via physical contact with the input device, the signal including or identifying information associated with a motion parameter, a simulation of inertia associated with continuing a movement of the graphics object subsequent to the stroke no longer being inputted via physical contact with the input device, the simulation of inertia based at least on the motion parameter;

applying, by the application, the transforms of the manipulation notifications to the graphics object displayed by the computing device, wherein each application of a transform results in display of a corresponding graphical transformation of the graphics object on the display, wherein, while the stroke is inputted via physical contact with the input device, the graphical transformations correspond to the inputs; and

continuing the movement of the graphics object according to the simulation of inertia.

8. An apparatus according to claim 7, wherein the inertia simulator stops generating the manipulation notifications according to the inertia simulation.

9. An apparatus according to claim 7, wherein the operating system comprises a programming interface through which the application instantiates the manipulation processor and the inertia simulator.

10. An apparatus according to claim 7, wherein based on the linkage of the inputs with the manipulation processor the inputs are automatically passed to the manipulation processor.

11. An apparatus according to claim 7, wherein the graphics object continues to move on the display without interruption after the manipulation notifications terminate.

12. An apparatus according to claim 7, wherein the inertial simulation comprises a deceleration rate parameter that is set by the application.

13. An apparatus according to claim 7, wherein linking information links the inertial simulator and the graphics object.

14. An apparatus according to claim 7, wherein the motion parameter seeds the inertia simulation.

15. Computer storage hardware storing information configured to cause a computing device to perform a process, the computing device comprising processing hardware, storage hardware, a display, and an input device, the input device configured to detect multiple concurrent input points, the process comprising:

executing an operating system, the operating system controlling execution of an application by the processing hardware, the application comprising an object, a manipulation processor, and an inertia simulator, the object comprising a graphic;

receiving, from the operating system, inputs, the inputs comprising two-dimensional information corresponding to respective input points detected by the input device, the two-dimensional information also corresponding to a stroke inputted via physical contact with the input device;

in accordance with logic of the application that links the inputs with the manipulation processor, passing at least some of the inputs from the application to the manipulation processor while the stroke is inputted via the physical contact with the input device;

receiving, by the application, manipulation notifications generated by the manipulation processor, each manipulation notification comprising a respective transform derived by the manipulation processor interpreting at least some of the two-dimensional information of the inputs;

initializing, based at least on a signal sent from the manipulation processor, a simulation of inertia associated with continuing a simulated movement of the object subsequent to the stroke no longer being inputted via physical contact with the input device, the simulation performed by the inertia simulator, the signal comprising information associated with a motion parameter, the simulation of inertia based at least on the motion parameter, wherein the signal is sent from the manipulation processor subsequent to or upon the stroke no longer being inputted via the physical contact with the input device;

applying, by the application, the transforms of the manipulation notifications to the object, wherein each application of a transform results in display of a corresponding graphical transformation of the graphic on the display, wherein, while the stroke is inputted via physical contact with the input device, the graphical transformations correspond to the inputs; and

continuing the movement of the object according to simulation of inertia.

16. Computer storage hardware according to claim 15, wherein the two-dimensional information comprises location and/or movement information.

17. Computer storage hardware according to claim 15, wherein the graphic moves in correspondence with the graphical transformations.

18. Computer storage hardware according to claim 15, wherein the signal comprises an event.

19. Computer storage hardware according to claim 15, wherein the inertia simulator and the manipulation processor comprise respective instances of respective object classes.

20. Computer storage hardware according to claim 15, wherein the graphical transformations comprise panning the graphic.