Multi-touch manipulation of application objects

• US 9,477,333 B2
• Filed: 10/08/2014
• Issued: 10/25/2016
• Est. Priority Date: 10/26/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Townsend, Reed L., Tu, Xiao, Scott, Bryan, Torset, Todd A., Geidl, Erik M., Pradhan, Samir S., Teed, Jennifer A.
• Primary Examiner(s): Dharia, Prabodh M
• Application Number:
  US14/509,050
• Publication Number:
  US 20150022478A1
• Time in Patent Office: 748 Days
• Field of Search: 345156-178, 345/428, 345/650, 345/660, 715/205, 715/234, 715/863, 715/864, 178/18.05, 382/188
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

The manipulation system described herein provides a common platform and application-programming interface (API) for applications to communicate with various multi-touch hardware devices, and facilitates the interpretation of multi-touch input as one or more manipulations. Manipulations map more directly to user intentions than do individual touch inputs and add support for basic transformation of objects using multiple touch contacts. An application can use manipulations to support rotating, resizing, and translating multiple objects at the same time. The manipulation system outputs two-dimensional (2D) affine transforms that contain rotation, scale, and translation information. Thus, using the manipulation system the application author can focus more on building touch-capable applications and let the manipulation system handle the underlying transformations and communication with the multi-touch hardware.

Claims

1. A computer-implemented method associated with generating a manipulation event, the computer-implemented method comprising:

receiving, from an application executing on a computing device having a touch-sensitive input surface, via a manipulation application programming interface, a plurality of touch inputs, the plurality of touch inputs corresponding to a plurality of touches previously directed to the application;

interpreting, after being received via the manipulation application programming interface, the received plurality of touch inputs into at least one corresponding manipulation event, the at least one corresponding manipulation event representing at least one manipulation based on relative movements of the plurality of touches on the touch-sensitive input surface; and

providing, via the manipulation application programming interface, the at least one corresponding manipulation event to the application.

2. The computer-implemented method of claim 1, wherein the at least one manipulation comprises at least one manipulation of an object, the application being configured to manipulate the object based on the provided at least one corresponding manipulation event and to cause the manipulated object to be displayed by the computing device.

3. The computer-implemented method of claim 2, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

4. The computer-implemented method of claim 2, wherein the interpreting comprises:

tracking a movement of each touch of the plurality of touches; and

determining the at least one corresponding manipulation event based on the tracked movements.

5. The computer-implemented method of claim 4, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

6. The computer-implemented method of claim 2, wherein the application is further configured to determine whether the plurality of touches are located within a boundary associated with the object.

7. The computer-implemented method of claim 1, wherein the plurality of touch inputs comprise two-dimensional input points corresponding to the plurality of touches of the touch-sensitive input screen.

8. The computer-implemented method of claim 1, wherein the manipulation application programming interface is configured to be invoked by arbitrary applications executing on the computing device, the plurality of touch inputs being received from the application of the arbitrary applications that currently is invoking the manipulation application programming interface.

9. The computer-implemented method of claim 1, wherein the at least one corresponding manipulation event comprises at least one geometric transform produced by an element that received the plurality of touch inputs via the manipulation application programming interface.

10. The computer-implemented method of claim 1, wherein the plurality of touch inputs comprises locations of the touches on the touch-sensitive input surface.

11. The computer-implemented method of claim 10, wherein the plurality of touch inputs further comprise values corresponding to pressures applied by the plurality of touches on the touch-sensitive input surface.

12. The computer-implemented method of claim 1, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation event, a translation event, and a scaling event.

13. The computer-implemented method of claim 1, wherein the interpreting comprises determining a meaning of a relative movement between a first touch of the plurality of touches and a second touch of the plurality of touches, wherein each type of relative movement between the first touch and the second touch has a predetermined meaning associated therewith.

14. The computer-implemented method of claim 13, wherein the predetermined meaning is a meaning selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

15. The computer-implemented method of claim 1, wherein the interpreting comprises:

tracking a movement of each touch of the plurality of touches; and

determining the at least one corresponding manipulation event based on the tracked movements.

16. The computer-implemented method of claim 15, wherein the at least one corresponding manipulation event is an event selected from the group consisting of a rotation event, a translation event, and a scaling event.

17. A computing device comprising:

a touch-sensitive input display;

a processor and storage, that, when the computing device is operating, execute an application and implement a manipulation application programming interface, the processor receiving, from the application, via the manipulation application programming interface, a plurality of touch inputs, the plurality of touch inputs corresponding to a plurality of touches of the touch-sensitive input display directed to the application;

the processor, after receiving the plurality of touch inputs via the manipulation application programming interface, interpreting the plurality of touch inputs into at least one corresponding manipulation event, the at least one corresponding manipulation event representing at least one manipulation based on relative movements of the plurality of touches on the touch-sensitive input surface; and

the application receiving, via the manipulation application programming interface, the at least one corresponding manipulation event.

18. The computing device of claim 17, wherein the at least one manipulation comprises at least one manipulation of an object, the application being configured to manipulate the object based on the at least one manipulation event and to cause the object to be displayed by the touch-sensitive input display.

19. The computing device of claim 18, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

20. The computing device of claim 18, wherein the processor performs the interpreting by tracking a movement of each touch of the plurality of touches, and by determining the at least one corresponding manipulation event based on the tracked movements.

21. The computing device of claim 20, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

22. The computing device of claim 17, wherein the plurality of touch inputs comprise two-dimensional input points corresponding to the plurality of touches of the touch-sensitive input screen.

23. The computing device of claim 17, wherein the manipulation application programming interface is configured to be invoked by arbitrary applications executing on the computing device, the plurality of touch inputs being received from the application of the arbitrary applications that currently is invoking the manipulation application programming interface.

24. The computing device of claim 17, wherein the at least one manipulation event indicates at least one geometric transform, the at least one manipulation event produced by an element that received the plurality of touch inputs via the manipulation application programming interface.

25. The computing device of claim 17, wherein the plurality of touch inputs comprises locations of the touches on the touch-sensitive input surface.

26. The computing device of claim 25, wherein the plurality of touch inputs further comprises values corresponding to pressures applied by the plurality of touches on the touch-sensitive input surface.

27. The computing device of claim 17, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation event, a translation event, and a scaling event.

28. The computing device of claim 17, wherein the interpreting comprises determining a meaning of a relative movement between a first touch of the plurality of touches and a second touch of the plurality of touches, wherein each type of relative movement between the first touch and the second touch has a predetermined meaning associated therewith.

29. The computing device of claim 28, wherein the predetermined meaning is a meaning selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

30. The computing device of claim 17, wherein the processor performs the interpreting by tracking a movement of each touch of the plurality of touches, and by determining the at least one corresponding manipulation event based on the tracked movements.

31. The computing device of claim 30, wherein the at least one corresponding manipulation event is an event selected from the group consisting of a rotation event, a translation event, and a scaling event.

32. A computing device comprising:

a contact-sensitive input device;

processing hardware and storage storing information to enable the processing hardware, when the computing device is operating, to execute an operating system to execute arbitrary applications and to provide an application programming interface to the arbitrary applications;

an application executed by the processing hardware when the computing device is operating;

an object executed by the processing hardware when the computing device is operating, the object instantiated via the application programming interface by an application executing on the computing device, the object comprising interpretation logic, wherein the application passes touch inputs to the object, the touch inputs corresponding to touches of the contact-sensitive input device;

the object, when the computing device is operating, applies the interpretation logic to interpret the touch inputs to identify a manipulation operation indicated by the touch inputs; and

the application, when the computing device is operating, receiving an indication of the manipulation operation, the indication having been triggered by the object having identified the manipulation operation.

33. A computing device according to claim 32, wherein the manipulation operation comprises a geometric transform.

34. A computing device according to claim 33, wherein the geometric transform comprises a rotation transform, a translation transform, and/or a scaling transform.

35. A computing device according to claim 33, wherein the operating system provides the touch inputs.

36. A computing device according to claim 35, wherein the object generates an event to cause the application to receive the indication of the manipulation operation.

37. A computing device according to claim 32, wherein the object maintains a history of touch inputs received by the object.

38. A computer-implemented method associated with generating a manipulation event, the computer-implemented method comprising:

receiving, through an application executing on a computing device having a touch-sensitive input surface, via an application programming interface, touch events, the touch events corresponding to points of contact with the touch-sensitive input surface;

receiving the touch events from the application via the application programming interface, and in response interpreting the touch events into a corresponding transformation event, the transformation event representing at least one geometric transformation that corresponds to relative movements of the points of contact with the touch-sensitive input surface; and

receiving, through the application programming interface, by the application, the at least one transformation event.

39. A computer-implemented method according to claim 38, wherein the application receives the transformation event with a callback or an event handler.

40. A computer-implemented method according to claim 38, wherein the touch events are initiated by an operating system of the computing device.

41. One or more computer-readable storage hardware storing information that, when executed by a computing device, cause the computing device to perform a process associated with generating a manipulation event, the process comprising:

receiving, from an application executing on a computing device having a touch-sensitive input surface, via a manipulation application programming interface, a plurality of touch inputs, the plurality of touch inputs corresponding to a plurality of touches previously directed to the application;

interpreting, after being received via the manipulation application programming interface, the received plurality of touch inputs into at least one corresponding manipulation event, the at least one corresponding manipulation event representing at least one manipulation based on relative movements of the plurality of touches on the touch-sensitive input surface; and

providing, via the manipulation application programming interface, the at least one corresponding manipulation event to the application.

42. The computer-implemented method of claim 41, wherein the at least one manipulation comprises at least one manipulation of an object, the application being configured to manipulate the object based on the provided at least one corresponding manipulation event and to cause the manipulated object to be displayed by the computing device.

43. The computer-implemented method of claim 42, wherein the interpreting comprises:

tracking a movement of each touch of the plurality of touches; and

determining the at least one corresponding manipulation event based on the tracked movements.

44. The computer-implemented method of claim 43, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

45. The computer-implemented method of claim 42, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

46. The computer-implemented method of claim 41, wherein the plurality of touch inputs comprise two-dimensional input points corresponding to the plurality of touches of the touch-sensitive input screen.

47. The computer-implemented method of claim 41, wherein the manipulation application programming interface is configured to be invoked by arbitrary applications executing on the computing device, the plurality of touch inputs being received from the application of the arbitrary applications that currently is invoking the manipulation application programming interface.

48. The computer-implemented method of claim 41, wherein the at least one corresponding manipulation event comprises at least one geometric transform produced by an element that received the plurality of touch inputs via the manipulation application programming interface.

49. The computer-implemented method of claim 41, wherein the plurality of touch inputs comprises locations of the touches on the touch-sensitive input surface.

50. The computer-implemented method of claim 49, wherein the plurality of touch inputs further comprise values corresponding to pressures applied by the plurality of touches on the touch-sensitive input surface.

51. The computer-implemented method of claim 41, wherein the at least one manipulation event comprises at least one manipulation selected from the group consisting of a rotation event, a translation event, and a scaling event.

52. The computer-implemented method of claim 41, wherein the interpreting comprises determining a meaning of a relative movement between a first touch of the plurality of touches and a second touch of the plurality of touches, wherein each type of relative movement between the first touch and the second touch has a predetermined meaning associated therewith.

53. The computer-implemented method of claim 52, wherein the predetermined meaning is a meaning selected from the group consisting of a rotation of the object, a translation of the object, and a scaling of the object.

54. The computer-implemented method of claim 53, wherein the interpreting comprises:

tracking a movement of each touch of the plurality of touches; and

determining the at least one corresponding manipulation event based on the tracked movements.

55. The computer-implemented method of claim 54, wherein the at least one corresponding manipulation event is an event selected from the group consisting of a rotation event, a translation event, and a scaling event.

56. The computer-implemented method of claim 54, wherein the application is further configured to determine whether the plurality of touches are located within a boundary associated with the object.