Devices, methods, and graphical user interfaces for manipulating user interface objects with visual and/or haptic feedback

• US 10,268,341 B2
• Filed: 09/30/2015
• Issued: 04/23/2019
• Est. Priority Date: 03/08/2015
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Kocienda, Kenneth L., Karunamuni, Chanaka G.
• Primary Examiner(s): Levy, Amy M
• Application Number:
  US14/870,754
• Publication Number:
  US 20160259527A1
• Time in Patent Office: 1,301 Days
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

In accordance with some embodiments, a device displays a user interface that includes user interface objects associated with respective object-specific operations that are triggered by changes in contact intensity. While displaying the user interface, the device detects a first input. In response to detecting the first input: if a focus selector is at a first location, the device performs a first operation associated with the first object; if a focus selector is at the second location, the device performs a second operation associated with the second object; and if a focus selector is at a location that is away from any objects that are associated with object-specific operations that are triggered by changes in contact intensity, perform a third operation that includes updating the user interface on the display to concurrently visually distinguish the first and second objects in the user interface.

Claims

1. A method, comprising:

at an electronic device with a display, a touch-sensitive surface, and one or more sensors for detecting intensities of contacts on the touch-sensitive surface;

displaying, on the display, a user interface that includes a plurality of intensity-sensitive user interface objects that are associated with respective object-specific operations that are triggered by intensity-based inputs that are identified based on changes in contact intensity that occur above a nominal intensity threshold that is met when touches are detected on the touch-sensitive surface, wherein the plurality of intensity-sensitive user interface objects include a first intensity-sensitive object displayed at a first location in the user interface and a second intensity-sensitive object displayed at a second location in the user interface;

while displaying the user interface that includes the plurality of intensity-sensitive user interface objects, detecting a first input that includes detecting a first contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a first respective location in the user interface and detecting an increase in a characteristic intensity of the first contact above a first intensity threshold that is greater than the nominal intensity threshold; and

in response to detecting the first input, performing a first respective operation associated with the first input, wherein the first respective operation is selected based on a determination as to whether the first respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, the second location in the user interface at which the second intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects that are associated with object-specific operations, wherein performing the first respective operation includes;

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the first location in the user interface at which the first intensity-sensitive object is displayed, performing a first operation associated with the first intensity-sensitive object that includes displaying, on the display, additional information associated with the first intensity-sensitive object;

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the second location in the user interface at which the second intensity-sensitive object is displayed, performing a second operation associated with the second intensity-sensitive object that includes displaying, on the display, additional information associated with the second intensity-sensitive object, wherein the second operation associated with the second intensity-sensitive object is distinct from the first operation associated with the first intensity-sensitive object; and

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, performing a third operation that updates the user interface on the display to concurrently visually distinguish the plurality of intensity-sensitive user interface objects, including the first and second intensity-sensitive objects in the user interface from regions of the user interface that are adjacent to the plurality of intensity-sensitive user interface objects, without displaying the additional information associated with the first intensity-sensitive object or the additional information associated with the second intensity-sensitive object, even if the characteristic intensity of the first contact increases above the first intensity threshold.

2. The method of claim 1, wherein the first operation associated with the first intensity-sensitive object includes emphasizing the first intensity-sensitive object relative to the second intensity-sensitive object.

3. The method of claim 2, wherein an amount of visual effect applied to emphasize the first intensity-sensitive object relative to the second intensity-sensitive object is dynamically varied in accordance with a current change in the characteristic intensity of the contact above the first intensity threshold.

4. The method of claim 1, wherein the second operation associated with the second intensity-sensitive object includes emphasizing the second intensity-sensitive object relative to the first intensity-sensitive object.

5. The method of claim 1, wherein the third operation includes emphasizing the first intensity-sensitive object and the second intensity-sensitive object.

6. The method of claim 5, wherein the emphasizing in the third operation includes emphasizing the first intensity-sensitive object in the same way that the first operation emphasizes the first intensity-sensitive object and emphasizing the second intensity-sensitive object in the same way that the second operation emphasizes the second intensity-sensitive object.

7. The method of claim 1, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation.

8. The method of claim 7, wherein the second intensity-sensitive object is associated with a second type of intensity-triggered operation that is distinct from the first type of intensity-triggered operation.

9. The method of claim 1, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation for revealing first content associated with the first intensity-sensitive object.

10. The method of claim 9, wherein the second intensity-sensitive object is associated with the first type of intensity-triggered operation for revealing second content associated with the second intensity-sensitive object.

11. The method of claim 1, wherein the first intensity-sensitive object is associated with a first type of action API associated with changes in contact intensity.

12. The method of claim 1, wherein performing the first operation associated with the first intensity-sensitive object includes:

presenting first information that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

presenting second information, that is distinct from the first information, that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

13. The method of claim 12, wherein the first information that corresponds to the first intensity-sensitive object is a preview associated with the first intensity-sensitive object, and the second information that corresponds to the first intensity-sensitive object is a second user interface associated with the first intensity-sensitive object.

14. The method of claim 1, wherein performing the second operation associated with the second intensity-sensitive object includes:

presenting first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

performing an action represented in the first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

15. The method of claim 14, wherein the first information that corresponds to the second intensity-sensitive object is a menu of actions associated with the second intensity-sensitive object, and the action represented in the first information that corresponds to the second intensity-sensitive object is a direct-selection action represented in the menu of actions associated with the second intensity-sensitive object.

16. The method of claim 1, including:

while displaying the user interface on the display, detecting a second input that includes detecting a second contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a second respective location in the user interface followed by lift-off of the second contact without detecting an increase in a characteristic intensity of the second contact above the first intensity threshold; and

,

in response to detecting the second input, performing a second respective operation associated with the second input, wherein the second respective operation is selected based on a determination as to whether the second respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, wherein performing the second respective operation includes;

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the first location in the user interface at which the first intensity-sensitive object is displayed, performing a second operation associated with the first intensity-sensitive object that is distinct from the first operation associated with the first object; and

,

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, performing a fourth operation that corresponds to a user interface element in the remainder of the user interface.

17. An electronic device, comprising:

a display;

a touch-sensitive surface;

one or more sensors to detect intensities of contacts with the touch-sensitive surface;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying, on the display, a user interface that includes a plurality of intensity-sensitive user interface objects that are associated with respective object-specific operations that are triggered by intensity-based inputs that are identified based on changes in contact intensity that occur above a nominal intensity threshold that is met when touches are detected on the touch-sensitive surface, wherein the plurality of intensity-sensitive user interface objects include a first intensity-sensitive object displayed at a first location in the user interface and a second intensity-sensitive object displayed at a second location in the user interface;

while displaying the user interface that includes the plurality of intensity-sensitive user interface objects, detecting a first input that includes detecting a first contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a first respective location in the user interface and detecting an increase in a characteristic intensity of the first contact above a first intensity threshold that is greater than the nominal intensity threshold; and

in response to detecting the first input, performing a first respective operation associated with the first input, wherein the first respective operation is selected based on a determination as to whether the first respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, the second location in the user interface at which the second intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects that are associated with object-specific operations, wherein performing the first respective operation includes;

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the first location in the user interface at which the first intensity-sensitive object is displayed, performing a first operation associated with the first intensity-sensitive object that includes displaying, on the display, additional information associated with the first intensity-sensitive object;

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the second location in the user interface at which the second intensity-sensitive object is displayed, performing a second operation associated with the second intensity-sensitive object that includes displaying, on the display, additional information associated with the second intensity-sensitive object, wherein the second operation associated with the second object is distinct from the first operation associated with the first intensity-sensitive object; and

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, performing a third operation that updates the user interface on the display to concurrently visually distinguish the plurality of intensity-sensitive user interface objects, including the first and second intensity-sensitive objects, in the user interface from regions of the user interface that are adjacent to the plurality of intensity-sensitive user interface objects, without displaying the additional information associated with the first intensity-sensitive object or the additional information associated with the second intensity-sensitive object, even if the characteristic intensity of the first contact increases above the first intensity threshold.

18. The electronic device of claim 17, wherein the first operation associated with the first intensity-sensitive object includes emphasizing the first intensity-sensitive object relative to the second intensity-sensitive object.

19. The electronic device of claim 18, wherein an amount of visual effect applied to emphasize the first intensity-sensitive object relative to the second intensity-sensitive object is dynamically varied in accordance with a current change in the characteristic intensity of the contact above the first intensity threshold.

20. The electronic device of claim 17, wherein the second operation associated with the second intensity-sensitive object includes emphasizing the second intensity-sensitive object relative to the first intensity-sensitive object.

21. The electronic device of claim 17, wherein the third operation includes emphasizing the first intensity-sensitive object and the second intensity-sensitive object.

22. The electronic device of claim 21, wherein the emphasizing in the third operation includes emphasizing the first intensity-sensitive object in the same way that the first operation emphasizes the first intensity-sensitive object and emphasizing the second intensity-sensitive object in the same way that the second operation emphasizes the second intensity-sensitive object.

23. The electronic device of claim 17, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation.

24. The electronic device of claim 23, wherein the second intensity-sensitive object is associated with a second type of intensity-triggered operation that is distinct from the first type of intensity-triggered operation.

25. The electronic device of claim 17, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation for revealing first content associated with the first intensity-sensitive object.

26. The electronic device of claim 25, wherein the second intensity-sensitive object is associated with the first type of intensity-triggered operation for revealing second content associated with the second intensity-sensitive object.

27. The electronic device of claim 17, wherein the first intensity-sensitive object is associated with a first type of action API associated with changes in contact intensity.

28. The electronic device of claim 17, wherein performing the first intensity-sensitive operation associated with the first object includes:

presenting first information that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

presenting second information, that is distinct from the first information, that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

29. The electronic device of claim 28, wherein the first information that corresponds to the first intensity-sensitive object is a preview associated with the first intensity-sensitive object, and the second information that corresponds to the first intensity-sensitive object is a second user interface associated with the first intensity-sensitive object.

30. The electronic device of claim 17, wherein performing the second operation associated with the second intensity-sensitive object includes:

presenting first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

performing an action represented in the first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

31. The electronic device of claim 30, wherein the first information that corresponds to the second intensity-sensitive object is a menu of actions associated with the second intensity-sensitive object, and the action represented in the first information that corresponds to the second object is a direct-selection action represented in the menu of actions associated with the second intensity-sensitive object.

32. The electronic device of claim 17, including instructions for:

while displaying the user interface on the display, detecting a second input that includes detecting a second contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a second respective location in the user interface followed by lift-off of the second contact without detecting an increase in a characteristic intensity of the second contact above the first intensity threshold; and

,

in response to detecting the second input, performing a second respective operation associated with the second input, wherein the second respective operation is selected based on a determination as to whether the second respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, wherein performing the second respective operation includes;

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the first location in the user interface at which the first intensity-sensitive object is displayed, performing a second operation associated with the first object that is distinct from the first operation associated with the first intensity-sensitive object; and

,

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, performing a fourth operation that corresponds to a user interface element in the remainder of the user interface.

33. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display, a touch-sensitive surface, and one or more sensors to detect intensities of contacts with the touch-sensitive surface cause the device to:

display, on the display, a user interface that includes a plurality of intensity-sensitive user interface objects that are associated with respective object-specific operations that are triggered by intensity-based inputs that are identified based on changes in contact intensity that occur above a nominal intensity threshold that is met when touches are detected on the touch-sensitive surface, wherein the plurality of intensity-sensitive user interface objects include a first intensity-sensitive object displayed at a first location in the user interface and a second intensity-sensitive object displayed at a second location in the user interface;

while displaying the user interface that includes the plurality of intensity-sensitive user interface objects, detect a first input that includes detecting a first contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a first respective location in the user interface and detecting an increase in a characteristic intensity of the first contact above a first intensity threshold that is greater than the nominal intensity threshold; and

in response to detecting the first input, performing a first respective operation associated with the first input, wherein the first respective operation is selected based on a determination as to whether the first respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, the second location in the user interface at which the second intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects that are associated with object-specific operations, wherein performing the first respective operation includes;

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the first location in the user interface at which the first intensity-sensitive object is displayed, perform a first operation associated with the first intensity-sensitive object that includes displaying, on the display, additional information associated with the first intensity-sensitive object;

in accordance with a determination that a the first respective location in the user interface that corresponds to the first contact is the second location in the user interface at which the second intensity-sensitive object is displayed, perform a second operation associated with the second intensity-sensitive object that includes displaying, on the display, additional information associated with the second intensity-sensitive object, wherein the second operation associated with the second intensity-sensitive object is distinct from the first operation associated with the first intensity-sensitive object; and

in accordance with a determination that the first respective location in the user interface that corresponds to the first contact is the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, perform a third operation that updates the user interface on the display to concurrently visually distinguish the plurality of intensity-sensitive user interface objects, including the first and second intensity-sensitive objects, in the user interface from regions of the user interface that are adjacent to the plurality of intensity-sensitive user interface objects, without displaying the additional information associated with the first intensity-sensitive object or the additional information associated with the second intensity-sensitive object, even if the characteristic intensity of the first contact increases above the first intensity threshold.

34. The non-transitory computer readable storage medium of claim 33, wherein the first operation associated with the first intensity-sensitive object includes emphasizing the first intensity-sensitive object relative to the second intensity-sensitive object.

35. The non-transitory computer readable storage medium of claim 34, wherein an amount of visual effect applied to emphasize the first intensity-sensitive object relative to the second intensity-sensitive object is dynamically varied in accordance with a current change in the characteristic intensity of the contact above the first intensity threshold.

36. The non-transitory computer readable storage medium of claim 33, wherein the second operation associated with the second intensity-sensitive object includes emphasizing the second intensity-sensitive object relative to the first intensity-sensitive object.

37. The non-transitory computer readable storage medium of claim 33, wherein the third operation includes emphasizing the first intensity-sensitive object and the second intensity-sensitive object.

38. The non-transitory computer readable storage medium of claim 37, wherein the emphasizing in the third operation includes emphasizing the first intensity-sensitive object in the same way that the first operation emphasizes the first intensity-sensitive object and emphasizing the second intensity-sensitive object in the same way that the second operation emphasizes the second intensity-sensitive object.

39. The non-transitory computer readable storage medium of claim 33, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation.

40. The non-transitory computer readable storage medium of claim 39, wherein the second intensity-sensitive object is associated with a second type of intensity-triggered operation that is distinct from the first type of intensity-triggered operation.

41. The non-transitory computer readable storage medium of claim 33, wherein the first intensity-sensitive object is associated with a first type of intensity-triggered operation for revealing first content associated with the first intensity-sensitive object.

42. The non-transitory computer readable storage medium of claim 41, wherein the second intensity-sensitive object is associated with the first type of intensity-triggered operation for revealing second content associated with the second intensity-sensitive object.

43. The non-transitory computer readable storage medium of claim 33, wherein the first intensity-sensitive object is associated with a first type of action API associated with changes in contact intensity.

44. The non-transitory computer readable storage medium of claim 33, wherein performing the first operation associated with the first intensity-sensitive object includes:

presenting first information that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

presenting second information, that is distinct from the first information, that corresponds to the first intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

45. The non-transitory computer readable storage medium of claim 44, wherein the first information that corresponds to the first intensity-sensitive object is a preview associated with the first intensity-sensitive object, and the second information that corresponds to the first intensity-sensitive object is a second user interface associated with the first intensity-sensitive object.

46. The non-transitory computer readable storage medium of claim 33, wherein performing the second operation associated with the second intensity-sensitive object includes:

presenting first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above the first intensity threshold; and

performing an action represented in the first information that corresponds to the second intensity-sensitive object when the characteristic intensity of the contact increases above a second intensity threshold that is greater than the first intensity threshold.

47. The non-transitory computer readable storage medium of claim 46, wherein the first information that corresponds to the second intensity-sensitive object is a menu of actions associated with the second intensity-sensitive object, and the action represented in the first information that corresponds to the second intensity-sensitive object is a direct-selection action represented in the menu of actions associated with the second intensity-sensitive object.

48. The non-transitory computer readable storage medium of claim 33, including instructions which cause the device to:

while displaying the user interface on the display, detect a second input that includes detecting a second contact on the touch-sensitive surface at a location on the touch-sensitive surface that corresponds to a second respective location in the user interface followed by lift-off of the second contact without detecting an increase in a characteristic intensity of the second contact above the first intensity threshold; and

,

in response to detecting the second input, perform a second respective operation associated with the second input, wherein the second respective operation is selected based on a determination as to whether the second respective location in the user interface corresponds to the first location in the user interface at which the first intensity-sensitive object is displayed, or a location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, wherein performing the second respective operation includes;

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the first location in the user interface at which the first intensity-sensitive object is displayed, performing a second operation associated with the first intensity-sensitive object that is distinct from the first operation associated with the first intensity-sensitive object; and

,

in accordance with a determination that the second respective location in the user interface that corresponds to the second contact is at the location in the user interface that is away from any intensity-sensitive user interface objects, in the plurality of intensity-sensitive user interface objects, that are associated with object-specific operations that are triggered by changes in contact intensity, performing a fourth operation that corresponds to a user interface element in the remainder of the user interface.