Gestures and touches on force-sensitive input devices

• US 9,244,562 B1
• Filed: 07/29/2010
• Issued: 01/26/2016
• Est. Priority Date: 07/31/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Rosenberg, Ilya D., Beguin, Julien G., Moscovich, Tomer, Perlin, Kenneth H
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): Chowdhury, Afroza
• Application Number:
  US12/846,268
• Time in Patent Office: 2,007 Days
• Field of Search: 345/173, 345/174, 715/863, 715/864, 178 1805- 1806
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 2203/04112 Electrode mesh in capacitive digitiser: electrode for touch sensing is formed of a mesh of very fine, normally metallic, interconnected lines that are almost invisible to see. This provides a quite large but transparent electrode surface, without need for ITO or similar transparent conductive material
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04144 using an array of force sensing means position sensing using the local deformation of sensor cells G06F3/0447
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/045 using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A force-sensitive touch sensor detects location and force of touches applied to the sensor. Movement of an object touching the force-sensitive touch sensor correlates to movement of a pointer on a display device. Varying levels of force applied to the force-sensitive touch sensor are interpreted as different commands. Objects displayed on the display device can be manipulated by a combination of gestures across a surface of the force-sensitive touch sensor and changes in force applied to the force-sensitive touch sensor.

Claims

1. A device comprising:

one or more processors;

a memory coupled to the one or more processors;

an input/output interface coupled to the one or more processors configured to receive input from a force-sensitive touch sensor and send output to a display; and

a touch control module stored in the memory and configured to execute on the one or more processors to;

recognize touches at a first contact point and a second contact point on the force-sensitive touch sensor;

determine a magnitude of force applied at the first contact point;

determine a magnitude of force applied at the second contact point;

determine a difference between the magnitude of force applied at the first contact point and the magnitude of force applied at the second contact point;

identify an isometric operation, wherein the isometric operation is an operation in which the determined difference between the magnitude of force applied at the first contact point and the magnitude of force applied at the second contact point is greater than a threshold while a location of the at least one of the first contact point or the second contact point remains substantially the same, the threshold being based at least in part on at least one of the determined magnitude of force applied at the first contact point or the determined magnitude of force applied at the second contact point;

identify an isotonic operation, wherein the isotonic operation is an operation in which a location of at least one of the first contact point or the second contact point changes while the determined difference between the magnitude of force applied at the first contact point and the magnitude of the force applied at the second contact point is less than the threshold;

interpret the isometric operation as a first mode of input; and

interpret the isotonic operation as a second mode of input.

2. The device of claim 1, wherein the force-sensitive touch sensor and the display comprise a force-sensitive touch-screen display.

3. The device of claim 1, wherein the force-sensitive touch sensor comprises an interpolating force-sensitive resistance array.

4. The device of claim 1, wherein the touch control module is further configured to execute on the one or more processors to, upon detection of the isotonic operation, temporarily suspend interpretation of isometric operations.

5. The device of claim 1, wherein the touch control module is further configured to execute on the one or more processors to:

determine a surface area on the force-sensitive touch sensor for the first contact point and the second contact point;

determine, for each of the touches, a type of object contacting the first contact point and the second contact point on the force-sensitive touch sensor.

6. The device of claim 5, wherein the type of object contacting the first contact point or the second contact point on the force-sensitive touch sensor comprises at least one of a finger or a stylus.

7. The device of claim 1, wherein the touch control module is further configured to execute on the one or more processors to interpret the first mode of input as selection of an item and interpret the second mode of input as assigning a value to the item.

8. The device of claim 7, wherein the selection of the item comprises selecting the item from a list of items and a change in isometric operations scrolls through the list of items.

9. The device of claim 1, wherein the touch control module is further configured to execute on the one or more processors to interpret the first mode of input as selecting of an item in response to the magnitude of force exceeding a threshold magnitude and interpret the second mode of input as moving of the item on the display.

10. The device of claim 9, wherein the touch control module is further configured to execute on the one or more processors to modify a functionality of the item in response to moving the item to contact another item on the display.

11. The device of claim 10, wherein the modified functionality of the item is provided as a command to the one or more processors in response to the force ceasing.

12. The device of claim 1, the touch control module further configured to execute on the one or more processors to:

detect a touch applied to a first area of the force-sensitive touch sensor and to a second area of the force-sensitive touch sensor, wherein a contact area of an object generating the touch overlaps in part with the first area and overlaps in part with the second area;

compare a magnitude of force applied to the first area to a magnitude of force applied to the second area;

initiate a first command in response to determining the magnitude of force applied to the first area is greater; and

initiate a second command in response to determining the magnitude of force applied to the second area is greater.

13. The device of claim 1, the touch control module further configured to execute on the one or more processors to:

detect an object touching the force-sensitive touch sensor;

generate a pointer on the display, a location of the pointer on the display based at least in part on a position of the object on the force-sensitive touch sensor;

alter the location of the pointer on the display in response to the object moving on the force-sensitive touch sensor;

detect a magnitude of force generated by the object touching the force-sensitive touch sensor; and

activate a function represented by a soft button underneath the pointer in response to detecting that the magnitude of force exceeds a threshold force.

14. The device of claim 1, the touch control module further configured to execute on the one or more processors to:

generate a representation of a three-dimensional object on the display;

recognize a plurality of touches on the force-sensitive touch sensor;

detect a magnitude of force generated by each of the plurality of touches; and

alter the display of the three-dimensional object responsive to the plurality of touches and the magnitude of force associated with each of the plurality of touches.

15. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors coupled to an input/output interface configured to receive input from a force-sensitive touch sensor and send output to a display, cause the one or more processors to perform acts comprising:

receiving input indicating that isotonic operations should not be identified;

recognizing touches at a first contact point and at a second contact point on the force-sensitive touch sensor;

determining a magnitude of force applied at the first contact point;

determining a magnitude of force applied at the second contact point, wherein the magnitude of force applied at the first contact point is different than the magnitude of force applied at the second contact point;

determining that the magnitude of force applied at the first contact point varies from the magnitude of force applied at the second contact point by at least a predetermined amount;

determining that a location of at least one of the first contact point or the second contact point remains substantially the same;

identifying an isometric operation; and

interpreting the isometric operation as a first mode of input.

16. The one or more non-transitory computer-readable storage media of claim 15, the acts further comprising:

detecting a touch applied to a first area of the force-sensitive touch sensor and to a second area of the force-sensitive touch sensor, wherein a contact area of an object generating the touch overlaps in part with the first area and overlaps in part with the second area;

comparing a magnitude of force applied to the first area to a magnitude of force applied to the second area;

initiating a first command in response to determining the magnitude of force applied to the first area is greater; and

initiating a second command in response to determining the magnitude of force applied to the second area is greater.

17. The one or more non-transitory computer-readable storage media of claim 15, the acts further comprising:

in response to interpreting that a particular operation is an isometric operation and a detected touch corresponding to the particular operation is a detected touch applied to a first area of the force-sensitive touch sensor and to a second area of the force-sensitive touch sensor, wherein a contact area of an object generating the detected touch overlaps in part with the first area and overlaps in part with the second area;

comparing a magnitude of force applied to the first area to a magnitude of force applied to the second area;

initiating a first command in response to determining the magnitude of force applied to the first area is greater; and

initiating a second command in response to determining the magnitude of force applied to the second area is greater.

18. The one or more non-transitory computer-readable storage media of claim 15, wherein the force-sensitive touch sensor comprises a force-sensitive touch-screen display.

19. The one or more non-transitory computer-readable storage media of claim 15, wherein the force-sensitive touch sensor comprises an interpolating force-sensitive resistance array.

20. The one or more non-transitory computer-readable storage media of claim 15, wherein the force-sensitive touch sensor generates anti-aliased images of pressure.

21. The one or more non-transitory computer-readable storage media of claim 16, wherein the first area and the second area respond to isometric inputs and do not respond to isotonic inputs.

22. The one or more non-transitory computer-readable storage media of claim 16, wherein the second area is adjacent to the first area.

23. The one or more non-transitory computer-readable storage media of claim 16, wherein the second area is within about 5 mm of the first area.

24. The one or more non-transitory computer-readable storage media of claim 16 wherein the touch is applied by a finger and the magnitude of force applied to the first area relative to the magnitude of force applied to the second area varies based at least in part upon an angle at which the finger contacts the force-sensitive touch sensor.

25. The one or more non-transitory computer-readable storage media of claim 16, wherein the first command comprises increasing a value and the second command comprises decreasing the value.

26. The one or more non-transitory computer-readable storage media of claim 25, wherein a rate at which the value increases depends at least in part on the magnitude of force applied to the first area and a rate at which the value decreases depends at least in part on the magnitude of force applied to the second area.

27. The one or more non-transitory computer-readable storage media of claim 25, wherein the value comprises a pitch of a musical note.

28. The one or more non-transitory computer-readable storage media of claim 16, the acts further comprising:

detecting the touch applied to a third area of the touch sensor and to a fourth area of the touch sensor in addition to the first and second areas of the touch sensor;

comparing a magnitude of force applied to the first, second, third, and fourth areas;

initiating a third command in response to determining the magnitude of force applied to the third area is greatest; and

initiating a fourth command in response to determining the magnitude of force applied to the fourth area is greatest.

29. The one or more non-transitory computer-readable storage media of claim 28, wherein the first, second, third, and fourth areas are adjacent.

30. The one or more non-transitory computer-readable storage media of claim 28, wherein each of the first, second, third, and fourth areas are within 5 mm of the other of the first, second, third, and fourth areas.

31. The one or more non-transitory computer-readable storage media of claim 28, wherein a line bisecting the first and second areas is perpendicular to a line bisecting the third and fourth areas.

32. The one or more non-transitory computer-readable storage media of claim 28, wherein the first command comprises moving a vertical scroll bar up, the second command comprises moving the vertical scroll bar down, the third command comprises moving a horizontal scroll bar left, and the fourth command comprises moving the horizontal scroll bar right.

33. The one or more non-transitory computer-readable storage media of claim 15, the acts further comprising

generating a representation of a three-dimensional object on the display;

and

altering the display of the three-dimensional object responsive to the one or more touches and the magnitude of force associated with each of the one or more of touches.

34. The one or more non-transitory computer-readable storage media of claim 33, the acts further comprising detecting a swipe gesture initiated by the magnitude of force associated with the touches exceeding a threshold force and, responsive to the swipe gesture, moving the representation of the three-dimensional object along a plane parallel to the display in a same direction as the swipe gesture.

35. The one or more non-transitory computer-readable storage media of claim 34, wherein the magnitude of force for each of the plurality of touches exceeds the threshold force.

36. The one or more non-transitory computer-readable storage media of claim 34, wherein a total magnitude of force of the plurality of touches exceeds the threshold force.

37. The one or more non-transitory computer-readable storage media of claim 33, the acts further comprising detecting a pinch gesture initiated by the magnitude of force associated with the touches exceeding a threshold force, wherein the plurality of touches move closer together and, responsive to the pinch gesture, moving the representation of the three-dimensional object so that the three-dimensional object appears farther from the display.

38. The one or more non-transitory computer-readable storage media of claim 33, the acts further comprising detecting a spread gesture initiated by the magnitude of force associated with the touches exceeding a threshold force, wherein the plurality of touches move farther apart and, responsive to the spread gesture, moving the representation of the three-dimensional object so that the three-dimensional object appears closer to the display.

39. The one or more non-transitory computer-readable storage media of claim 33, the acts further comprising detecting a multi-touch rotation gesture initiated by the magnitude of force associated with the touches exceeding a threshold force, wherein a shape defined by the contact points of the plurality of touches rotates on the force-sensitive touch sensor and, responsive to the rotation, the representation of the three-dimensional object rotates around an axis that is a normal of a plane defined by the display.

40. The one or more non-transitory computer-readable storage media of claim 33, the acts further comprising:

detecting a differential in a magnitude of force by a first of the plurality of touches and a second of the plurality of touches; and

rotating the representation of the three-dimensional object along an axis parallel to the display and perpendicular to a line connecting a location of the first of the plurality of touches with the second of the plurality of touches, wherein the three-dimensional object is rotated towards the one of the plurality of touches having a greater magnitude of force.

41. The one or more non-transitory computer-readable storage media of claim 15, wherein the input is a first input, and the acts further comprising:

receiving a second input indicating that isometric operations should not be identified;

determining at least one of that a location of the first contact point has changed by more than a threshold or a location of the second contact point has changed by more than the threshold;

determining that magnitude of force applied at the first contact point and the magnitude of force applied at the second contact point does not vary by more than the predetermined amount;

identifying an isotonic operation; and

interpreting the isotonic operation as a second mode of input.

42. A computer-implemented method comprising:

performing the following under control of computing device comprising one or more processors coupled to an input/output interface configured to receive input from a force-sensitive touch sensor and send output to a display, the computing device configured with executable instructions;

recognizing touches at one or more contact points on the force-sensitive touch sensor;

determining a magnitude of force applied at each of the contact points, wherein the magnitude of force is determined to be a light touch if it is below a first force threshold, a medium touch if it is above the first force threshold but below a second force threshold, or a hard touch if it is above the second force threshold, at least one of the force thresholds being determined based at least in part on a determined magnitude of force applied to at least one contact point;

determining changes in the location or changes in the magnitude of force;

identify an isometric operation, wherein the isometric operation is an operation comprising any combination of a light touch, a medium touch, or a hard touch; and

identify an isotonic operation, wherein the isotonic operation is an operation in which a location of at least one of the contact points changes.

43. The computer-implemented method of claim 42, further comprising:

detecting an object touching a force-sensitive touch sensor;

generating a pointer on the display, a location of the pointer on the display based at least in part on a position of the object on the force-sensitive touch sensor;

altering the location of the pointer on the display in response to the object moving on the force-sensitive touch sensor;

detecting a magnitude of force generated by the object touching the force-sensitive touch sensor; and

activating a function represented by a soft button underneath the pointer in response to detecting that the magnitude of force exceeds at least the first force threshold.

44. The computer-implemented method of claim 42, further comprising:

detecting a touch of an object on a force-sensitive touch sensor;

generating a pointer on the display, a location of the pointer on the display based at least in part on a position of the detected touch of the object on the force-sensitive touch sensor;

in response to identifying that a first particular operation is an isotonic operation corresponding to the detected touch of the object moving on the force-sensitive touch sensor;

altering the location of the pointer on the display in response to the object moving on the force-sensitive touch sensor;

during or subsequent to the moving of the detected touch of the object on the force-sensitive touch sensor or in response to identifying that a second particular operation is an isometric operation corresponding to the detected touch of the object;

detecting a magnitude of force generated by the detected touch of the object touching the force-sensitive touch sensor; and

activating a function represented by a soft button underneath the pointer in response to detecting that the magnitude of force exceeds at least the first force threshold.

45. The computer-implemented method of claim 43, wherein generating a pointer comprises creating a first visual representation of the pointer in response to determining the object comprises a finger and creating a second visual representation of the pointer in response to determining the object comprises a stylus.

46. The computer-implemented method of claim 43, wherein detecting the object comprises sensing the object using an interpolating force-sensitive resistance array.

47. The computer-implemented method of claim 43, wherein generating the pointer on the display comprises generating a visual representation of the pointer that changes continuously as the magnitude of the force changes.

48. The computer-implemented method of claim 43, wherein generating the pointer on the display comprises generating a visual representation of the pointer that changes discontinuously as the magnitude of the force changes, the change in the visual representation of the pointer occurring in response to the magnitude of the force varying between at least one of the light touch and the medium touch or the medium touch and the hard touch.

49. The computer-implemented method of claim 48, wherein generating the pointer on the display comprises generating first visual representation in response to detecting that the magnitude of the force is below the first force threshold and a second visual representation in response to detecting that the magnitude of the force is above the first force threshold.

50. The computer-implemented method of claim 43, further comprising activating an additional function represented by the soft button in response to detecting that the magnitude of force exceeds the second force threshold.

51. The computer-implemented method of claim 43, further comprising:

detecting an additional object touching the force-sensitive touch sensor;

generating an additional pointer on the display based at least in part on a position of the additional object on the force-sensitive touch sensor;

altering a location of the additional pointer on the display in response to the additional object moving on the force-sensitive touch sensor; and

detecting a magnitude of force generated by the additional object touching the force-sensitive touch sensor.

52. The computer-implemented method of claim 51, wherein the object comprises a finger of a first user and the additional object comprises a finger of a second user.

53. The computer-implemented method of claim 51, wherein the object comprises a finger and the additional object comprises a stylus.

54. The computer-implemented method of claim 51, wherein a visual representation of the pointer and a visual representation of the additional pointer are different.

55. A device comprising:

one or more processors;

a memory coupled to the one or more processors;

an input/output interface coupled to the one or more processors configured to receive input from a force-sensitive touch sensor; and

a touch control module stored in the memory and configured to execute on the one or more processors to;

detect touches at a first contact point and a second contact point on the force-sensitive touch sensor;

determine a magnitude of force applied at the first contact point;

determine a magnitude of force applied at the second contact point;

determine a difference between the magnitude of force applied at the first contact point and the magnitude of force applied at the second contact point;

identify a first type of operation, the first type of operation is an operation where the difference between the magnitude of force applied at the first contact point and the magnitude of force applied at the second contact point exceeds a threshold, the threshold being based at least in part on at least one of the determined magnitude of force applied at the first contact point or the determined magnitude of force applied at the second contact point; and

identify a second type of operation, the second type of operation is an operation where the location of at least one of the first contact point or the second contact point changes on the force-sensitive touch sensor while the difference between the magnitude of the force applied at the first contact point and the magnitude of the force applied at the second contact point is below the threshold.