METHODS OF INTERFACING WITH MULTI-POINT INPUT DEVICES AND MULTI-POINT INPUT SYSTEMS EMPLOYING INTERFACING TECHNIQUES

• US 20080180404A1
• Filed: 01/31/2008
• Published: 07/31/2008
• Est. Priority Date: 01/31/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Perceptive Pixel Incorporated (Microsoft Corporation)
• Inventors: Han, Jefferson Y., Davidson, Philip L.
• Granted Patent: US 8,269,729 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Methods and systems for interfacing with multi-point input devices employ various techniques for controlling displayed images, including 2D and 3D image translation, scale/zoom, rotation control and globe axis tilt control. Various techniques employ three or more simultaneous inputs, changes in characteristics of those inputs, and pressure sensing, among other things.

Claims

1. A method of interfacing with a multi-point input device, comprising the steps of:

displaying an image on a display device;

detecting positions of at least three elements simultaneously contacting the display device;

identifying a change in an attribute of the at least three elements; and

controlling a change in the image displayed on the display device in accordance with the identified change in the attribute of the at least three elements.

2. The method of claim 1, wherein identifying a change in an attribute comprises identifying movement of a centroid of the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling translation of the image displayed on the display device in accordance with movement of the centroid.

3. The method of claim 1, wherein identifying a change in an attribute comprises identifying changes in distances of the elements to a centroid of the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling a scale of the image as a function of the identified changes in distances of the elements to the centroid.

4. The method of claim 1, wherein identifying a change in an attribute comprises calculating an area defined by the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling a scale of the image as a function of the calculated area.

5. The method of claim 1, wherein identifying a change in an attribute comprises identifying a position of a centroid of the at least three elements, and identifying changes in angles of the elements relative to the centroid;

and controlling a change in the image displayed on the display device comprises controlling a rotation of the image as a function of the identified changes in angles of the elements relative to the centroid.

6. The method of claim 1, wherein identifying a change in an attribute comprises establishing a plurality of geometric position constraints, each of the geometric position constraints corresponding to an initial position of a respective one of the elements, identifying new positions of the elements, and solving, using numeric optimization, the orientation parameters of the displayed image to best match to the new positions of the elements;

and controlling a change in the image displayed on the display device comprises controlling the image in accordance with the solved orientation parameters.

7. The method of claim 1, wherein displaying an image on a display device comprises displaying an image corresponding to a three-dimensional graphical representation of an object.

8. The method of claim 7, wherein identifying a change in an attribute comprises identifying movement of a centroid of the at least three elements, and controlling a change in the image displayed on the display device comprises controlling translation of the three-dimensional graphical representation of the object in accordance with movement of the centroid.

9. The method of claim 8, wherein controlling translation of the three-dimensional graphical representation of the object comprises controlling the image so that a centroid of the object moves in accordance with movement of the centroid.

10. The method of claim 7, wherein identifying a change in an attribute comprises identifying changes in distances of the elements to a point disposed within an area defined by the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling a scale of the three-dimensional graphical representation of the object as a function of the identified changes in distances of the elements to the point.

11. The method of claim 7, wherein identifying a change in an attribute comprises identifying a position of a centroid of the at least three elements, and identifying changes in distances of the elements to the centroid;

and controlling a change in the image displayed on the display device comprises controlling a scale of the three-dimensional graphical representation of the object as a function of the identified changes in distances of the elements to the centroid.

12. The method of claim 7, wherein identifying a change in an attribute comprises calculating an area defined by the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling a scale of the three-dimensional graphical representation of the object as a function of the calculated area.

13. The method of claim 7, wherein identifying a change in an attribute comprises identifying a position of a centroid of the at least three elements, and identifying changes in distances of the elements to the centroid;

and controlling a change in the image displayed on the display device comprises controlling movement of the three-dimensional graphical representation of the object along an axis normal to a display surface of the display device as a function of the identified changes in distances of the elements to the centroid.

14. The method of claim 7, wherein identifying a change in an attribute comprises calculating a change in an area defined by the at least three elements;

and controlling a change in the image displayed on the display device comprises controlling movement of the three-dimensional graphical representation of the object along an axis normal to a display surface of the display device as a function of changes in the area.

15. The method of claim 7, wherein

identifying a change in an attribute comprises identifying a position and movement of a centroid of the at least three elements, and identifying changes in distances of the elements to the centroid;

and

controlling a change in the image displayed on the display device comprises controlling translation of the three-dimensional graphical representation of the object along X and Y axes in accordance with movement of the centroid, and controlling movement of the three-dimensional graphical representation of the object along a Z axis as a function of the identified changes in distances of the elements to the centroid, the X, Y and Z axes being perpendicular to one another.

16. The method of claim 15, wherein the Z axis is normal to a display surface of the display device.

17. The method of claim 7, wherein

identifying a change in an attribute comprises identifying changes in angles of the elements about a point disposed within an area defined by the at least three elements;

and

controlling a change in the image displayed on the display device comprises controlling a rotation of the object displayed on the display device about an axis normal to a display surface of the display device as a function of the identified changes in angles of the elements about the point.

18. The method of claim 17, wherein controlling a rotation of the object displayed on the display device comprises controlling the rotation of the object about an axis passing through a centroid of the object represented by the three-dimensional graphical representation displayed on the display device.

19. The method of claim 7, wherein

identifying a change in an attribute comprises identifying changes in angles of the elements about a centroid of the at least three elements;

and

controlling a change in the image displayed on the display device comprises controlling a rotation of the object displayed on the display device about an axis normal to a display surface of the display device as a function of the identified changes in angles of the elements about the centroid.

20. A method of interfacing with a multi-point input device, comprising the steps of:

displaying an image on a display device;

detecting positions of at least three elements simultaneously contacting a multi-point input device;

identifying a change in an attribute of the at least three elements; and

controlling a change in the image displayed on the display device in accordance with the identified change in the attribute of the at least three elements.

21. A multi-point input system, comprising

a display device for displaying an image on a display surface, the display device adapted to detect positions of at least three elements simultaneously contacting the display surface;

and

a controller for identifying a change in an attribute of the at least three elements, and for controlling a change in the image displayed on the display device in accordance with the identified change in the attribute of the at least three elements.

22. The multi-point input system of claim 21, wherein the controller identifies movement of a centroid of the at least three elements;

and controls a change in the image displayed on the display device by controlling translation of the image displayed on the display device in accordance with movement of the centroid.

23. The multi-point input system of claim 21, wherein the controller identifies changes in distances of the elements to a centroid of the at least three elements;

and controls a scale of the image displayed on the display device as a function of the identified changes in distances of the elements to the centroid.

24. The multi-point input system of claim 21, wherein the controller calculates an area defined by the at least three elements;

and controls a scale of the image displayed on the display device as a function of the calculated area.

25. The multi-point input system of claim 21, wherein the controller identifies a position of a centroid of the at least three elements, identifies changes in angles of the elements relative to the centroid, and controls a rotation of the image displayed on the display device as a function of the identified changes in angles of the elements relative to the centroid.

26. The multi-point input system of claim 21, wherein the controller establishes a plurality of geometric position constraints, each of the geometric position constraints corresponding to an initial position of a respective one of the elements, identifies new positions of the elements, solves, using numeric optimization, the orientation parameters of the displayed image to best match the new positions of the elements;

and controls the image displayed on the display device in accordance with the solved orientation parameters.

27. The multi-point input system of claim 21, wherein the display device displays an image corresponding to a three-dimensional graphical representation of an object.

28. The multi-point input system of claim 27, wherein the controller identifies movement of a centroid of the at least three elements, and controls translation of the three-dimensional graphical representation of the object on the display device in accordance with movement of the centroid.

29. The multi-point input system of claim 28, wherein the controller controls movement of the image on the display device so that a centroid of the object moves in accordance with movement of the centroid.

30. The multi-point input system of claim 27, wherein the controller identifies changes in distances of the elements to a point disposed within an area defined by the at least three elements;

and controls a scale of the three-dimensional graphical representation of the object displayed on the display device as a function of the identified changes in distances of the elements to the point.

31. The multi-point input system of claim 27, wherein the controller identifies a position of a centroid of the at least three elements, identifies changes in distances of the elements to the centroid;

and controls a scale of the three-dimensional graphical representation of the object displayed on the display device as a function of the identified changes in distances of the elements to the centroid.

32. The multi-point input system of claim 27, wherein the controller calculates an area defined by the at least three elements, and controls a scale of the three-dimensional graphical representation of the object displayed on the display device as a function of the calculated area.

33. The multi-point input system of claim 27, wherein the controller identifies a position of a centroid of the at least three elements, identifies changes in distances of the elements to the centroid;

and controls movement as displayed on the display device of the three-dimensional graphical representation of the object along an axis normal to the display surface of the display device as a function of the identified changes in distances of the elements to the centroid.

34. The multi-point input system of claim 27, wherein the controller calculates a change in an area defined by the at least three elements;

and controls movement as displayed on the display device of the three-dimensional graphical representation of the object along an axis normal to the display surface of the display device as a function of changes in the area.

35. The multi-point input system of claim 27, wherein the controller identifies a position and movement of a centroid of the at least three elements, identifies changes in distances of the elements to the centroid;

controls translation as displayed on the display device of the three-dimensional graphical representation of the object along X and Y axes in accordance with movement of the centroid, and controls movement of the three-dimensional graphical representation of the object along a Z axis as a function of the identified changes in distances of the elements to the centroid, the X, Y and Z axes being perpendicular to one another.

36. The multi-point input system of claim 35, wherein the Z axis is normal to the display surface of the display device.

37. The multi-point input system of claim 27, wherein the controller identifies changes in angles of the elements about a point disposed within an area defined by the at least three elements, and controls a rotation of the object displayed on the display device about an axis normal to the display surface of the display device as a function of the identified changes in angles of the elements about the point.

38. The multi-point input system of claim 37, wherein the controller controls the rotation of the object about an axis passing through a centroid of the object represented by the three-dimensional graphical representation displayed on the display device.

39. The multi-point input system of claim 27, wherein the controller identifies changes in angles of the elements about a centroid of the at least three elements, and controls a rotation of the object displayed on the display device about an axis normal to the display surface of the display device as a function of the identified changes in angles of the elements about the centroid.

40. A multi-point input system system, comprising

a display device for displaying an image;

a multi-point input device having a contact surface, the multi-point input device adapted to detect positions of at least three elements simultaneously contacting the contact surface; and

a controller for identifying a change in an attribute of the at least three elements based on the detected positions of the elements contacting the contact surface, and for controlling a change in the image displayed by the display device in accordance with the identified change in the attribute of the at least three elements.