Devices, methods, and graphical user interfaces for generating tactile outputs

• US 9,753,541 B1
• Filed: 09/20/2016
• Issued: 09/05/2017
• Est. Priority Date: 09/06/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Robert, Julien Y., Fox, Tyler M., Moussette, Camille, Miladinov, Viktor, Mouilleseaux, Jean-Pierre M., Cordier, Madeleine S.
• Primary Examiner(s): Abdulselam, Abbas
• Application Number:
  US15/271,073
• Time in Patent Office: 350 Days
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/177 of tables; using ruled lines

Abstract

An electronic device displays a user interface for a first software application. The user interface includes multiple elements to which user interaction models from multiple user interaction models provided by an application-independent module have been assigned (including a first element to which a first user interaction model has been assigned). The multiple elements have content provided by an application-specific module for the first software application. The first user interaction model defines how the user interface responds to inputs directed to the first element. The device detects an input directed to the first element; and updates the user interface based on characteristics of the input. If the input meets tactile output criteria specified by the first user interaction model, the device generates a first tactile output corresponding to the input, and if not, the device forgoes generation of the first tactile output.

Claims

1. A method, comprising:

at an electronic device with a display, one or more input devices, and one or more tactile output generators;

displaying, on the display, a user interface for a first software application, wherein;

the user interface includes a plurality of elements to which user interaction models from a plurality of user interaction models provided by an application-independent module have been assigned, the plurality of elements including a first element to which a first user interaction model has been assigned, the plurality of elements having content provided by an application-specific module for the first software application; and

the first user interaction model defines how the user interface responds to inputs directed to the first element of the user interface;

while displaying the user interface, detecting, via the one or more input devices, an input directed to the first element of the user interface; and

in response to detecting the input;

updating the user interface on the display based on characteristics of the input; and

in accordance with a determination, by the application-independent module, that the input meets tactile output criteria specified by the first user interaction model, generating a first tactile output corresponding to the input; and

in accordance with a determination, by the application-independent module, that the input does not meet the tactile output criteria specified by the first user interaction model, forgoing generation of the first tactile output.

2. The method of claim 1, including updating the user interface on the display based on characteristics of the input and the first user interaction model.

3. The method of claim 1, including:

assigning the first user interaction model to the first element using a predefined application programming interface (API).

4. The method of claim 1, wherein the plurality of elements includes a second element to which a second user interaction model from the plurality of user interaction models provided by the application-independent module has been assigned, and the method includes:

while displaying the user interface, detecting, via the one or more input devices, a second input directed to the second element of the user interface; and

,

in response to detecting the second input directed to the second element of the user interface;

updating the user interface on the display based on characteristics of the detected second input; and

in accordance with a determination, by the application-independent module, that the second input meets tactile output criteria specified by the second user interaction model, generating a second tactile output corresponding to the second input; and

in accordance with a determination, by the application-independent module, that the second input does not meet the tactile output criteria specified by the second user interaction model, forgoing generation of the second tactile output.

5. The method of claim 1, wherein the first element includes a slider object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating tactile outputs in accordance with user manipulation of the slider object with touch inputs.

6. The method of claim 5, including generating a predefined modulation selection tactile output in response to a touch input that selects a modulation type for the slider object.

7. The method of claim 5, wherein the first user interaction model specifies generating the first tactile output in accordance with a modulation type of the slider object and a parameter of the first software application modulated in accordance with user manipulation of the slider object with touch inputs.

8. The method of claim 5, wherein the first user interaction model specifies generating a distinct tactile output in accordance with a moveable portion of the slider object reaching an end position.

9. The method of claim 1, wherein the first element includes a switch object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a state of the switch object.

10. The method of claim 1, wherein the first element includes a value selection object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a value selected using the value selection object.

11. The method of claim 1, wherein the first element includes content that is refreshed in response to a pull down gesture reaching a refresh threshold, and the first user interaction model specifies generating a predefined impact tactile output in accordance with a determination, by the application-independent module, that the input corresponds to a pull down gesture reaching the refresh threshold.

12. The method of claim 1, wherein the first element includes a region or object, the size of which changes in response to the input comprising a zoom gesture, and the first user interaction model specifies generating a predefined size limit tactile output in accordance with a determination, by the application-independent module, that the size of the region or object has reached a predefined limit.

13. The method of claim 1, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input moves from any cell in the table to any other cell in the table, or moves from outside the table to a cell inside the table.

14. The method of claim 1, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that two or more rows of the cells are reordered in response to the input.

15. The method of claim 1, wherein the input causes the first software application to select an item, in a collection of distinct items, to be moved to a predefined position within the first element, and the first user interaction model specifies generating the first tactile output in conjunction with the selected item being moved to the predefined position within the first element.

16. The method of claim 1, wherein the first user interaction model specifies generating the first tactile output in accordance with a determination that the input or an object manipulated in accordance with the input approaches a next predefined state, and generating a second tactile output, distinct from the first tactile output, in accordance with a determination that the input or an object manipulated in accordance with the input reaches the next predefined state.

17. An electronic device, comprising:

a display;

one or more input devices;

one or more tactile output generators;

one or more processors; and

memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for;

displaying, on the display, a user interface for a first software application, wherein;

the user interface includes a plurality of elements to which user interaction models from a plurality of user interaction models provided by an application-independent module have been assigned, the plurality of elements including a first element to which a first user interaction model has been assigned, the plurality of elements having content provided by an application-specific module for the first software application; and

the first user interaction model defines how the user interface responds to inputs directed to the first element of the user interface;

while displaying the user interface, detecting, via the one or more input devices, an input directed to the first element of the user interface; and

in response to detecting the input;

updating the user interface on the display based on characteristics of the input; and

in accordance with a determination, by the application-independent module, that the input meets tactile output criteria specified by the first user interaction model, generating a first tactile output corresponding to the input; and

in accordance with a determination, by the application-independent module, that the input does not meet the tactile output criteria specified by the first user interaction model, forgoing generation of the first tactile output.

18. The electronic device of claim 17, wherein the one or more programs include instructions for updating the user interface on the display based on characteristics of the input and the first user interaction model.

19. The electronic device of claim 17, wherein the one or more programs include instructions for:

assigning the first user interaction model to the first element using a predefined application programming interface (API).

20. The electronic device of claim 17, wherein the plurality of elements includes a second element to which a second user interaction model from the plurality of user interaction models provided by the application-independent module has been assigned, and the one or more programs include instructions for:

while displaying the user interface, detecting, via the one or more input devices, a second input directed to the second element of the user interface; and

,

in response to detecting the second input directed to the second element of the user interface;

updating the user interface on the display based on characteristics of the detected second input; and

in accordance with a determination, by the application-independent module, that the second input meets tactile output criteria specified by the second user interaction model, generating a second tactile output corresponding to the second input; and

in accordance with a determination, by the application-independent module, that the second input does not meet the tactile output criteria specified by the second user interaction model, forgoing generation of the second tactile output.

21. The electronic device of claim 17, wherein the first element includes a slider object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating tactile outputs in accordance with user manipulation of the slider object with touch inputs.

22. The electronic device of claim 21, wherein the one or more programs include instructions for generating a predefined modulation selection tactile output in response to a touch input that selects a modulation type for the slider object.

23. The electronic device of claim 21, wherein the first user interaction model specifies generating the first tactile output in accordance with a modulation type of the slider object and a parameter of the first software application modulated in accordance with user manipulation of the slider object with touch inputs.

24. The electronic device of claim 21, wherein the first user interaction model specifies generating a distinct tactile output in accordance with a moveable portion of the slider object reaching an end position.

25. The electronic device of claim 17, wherein the first element includes a switch object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a state of the switch object.

26. The electronic device of claim 17, wherein the first element includes a value selection object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a value selected using the value selection object.

27. The electronic device of claim 17, wherein the first element includes content that is refreshed in response to a pull down gesture reaching a refresh threshold, and the first user interaction model specifies generating a predefined impact tactile output in accordance with a determination, by the application-independent module, that the input corresponds to a pull down gesture reaching the refresh threshold.

28. The electronic device of claim 17, wherein the first element includes a region or object, the size of which changes in response to the input comprising a zoom gesture, and the first user interaction model specifies generating a predefined size limit tactile output in accordance with a determination, by the application-independent module, that the size of the region or object has reached a predefined limit.

29. The electronic device of claim 17, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input moves from any cell in the table to any other cell in the table, or moves from outside the table to a cell inside the table.

30. The electronic device of claim 17, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that two or more rows of the cells are reordered in response to the input.

31. The electronic device of claim 17, wherein the input causes the first software application to select an item, in a collection of distinct items, to be moved to a predefined position within the first element, and the first user interaction model specifies generating the first tactile output in conjunction with the selected item being moved to the predefined position within the first element.

32. The electronic device of claim 17, wherein the first user interaction model specifies generating the first tactile output in accordance with a determination that the input or an object manipulated in accordance with the input approaches a next predefined state, and generating a second tactile output, distinct from the first tactile output, in accordance with a determination that the input or an object manipulated in accordance with the input reaches the next predefined state.

33. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with a display, one or more input devices, and one or more tactile output generators, cause the device to:

display, on the display, a user interface for a first software application, wherein;

the user interface includes a plurality of elements to which user interaction models from a plurality of user interaction models provided by an application-independent module have been assigned, the plurality of elements including a first element to which a first user interaction model has been assigned, the plurality of elements having content provided by an application-specific module for the first software application; and

the first user interaction model defines how the user interface responds to inputs directed to the first element of the user interface;

while displaying the user interface, detect, via the one or more input devices, an input directed to the first element of the user interface; and

in response to detecting the input;

update the user interface on the display based on characteristics of the input; and

in accordance with a determination, by the application-independent module, that the input meets tactile output criteria specified by the first user interaction model, generate a first tactile output corresponding to the input; and

in accordance with a determination, by the application-independent module, that the input does not meet the tactile output criteria specified by the first user interaction model, forgo generation of the first tactile output.

34. The non-transitory computer readable storage medium of claim 33, wherein the one or more programs include instructions, which when executed by the electronic device, cause the device to update the user interface on the display based on characteristics of the input and the first user interaction model.

35. The non-transitory computer readable storage medium of claim 33, wherein the one or more programs include instructions, which when executed by the electronic device, cause the device to:

assign the first user interaction model to the first element using a predefined application programming interface (API).

36. The non-transitory computer readable storage medium of claim 33, wherein the plurality of elements includes a second element to which a second user interaction model from the plurality of user interaction models provided by the application-independent module has been assigned, and the one or more programs include instructions, which when executed by the electronic device, cause the device to:

while displaying the user interface, detect, via the one or more input devices, a second input directed to the second element of the user interface; and

,

in response to detecting the second input directed to the second element of the user interface;

update the user interface on the display based on characteristics of the detected second input; and

in accordance with a determination, by the application-independent module, that the second input meets tactile output criteria specified by the second user interaction model, generate a second tactile output corresponding to the second input; and

in accordance with a determination, by the application-independent module, that the second input does not meet the tactile output criteria specified by the second user interaction model, forgo generation of the second tactile output.

37. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a slider object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating tactile outputs in accordance with user manipulation of the slider object with touch inputs.

38. The non-transitory computer readable storage medium of claim 37, wherein the one or more programs include instructions, which when executed by the electronic device, cause the device to generate a predefined modulation selection tactile output in response to a touch input that selects a modulation type for the slider object.

39. The non-transitory computer readable storage medium of claim 37, wherein the first user interaction model specifies generating the first tactile output in accordance with a modulation type of the slider object and a parameter of the first software application modulated in accordance with user manipulation of the slider object with touch inputs.

40. The non-transitory computer readable storage medium of claim 37, wherein the first user interaction model specifies generating a distinct tactile output in accordance with a moveable portion of the slider object reaching an end position.

41. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a switch object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a state of the switch object.

42. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a value selection object generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input corresponds to an action that changes a value selected using the value selection object.

43. The non-transitory computer readable storage medium of claim 33, wherein the first element includes content that is refreshed in response to a pull down gesture reaching a refresh threshold, and the first user interaction model specifies generating a predefined impact tactile output in accordance with a determination, by the application-independent module, that the input corresponds to a pull down gesture reaching the refresh threshold.

44. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a region or object, the size of which changes in response to the input comprising a zoom gesture, and the first user interaction model specifies generating a predefined size limit tactile output in accordance with a determination, by the application-independent module, that the size of the region or object has reached a predefined limit.

45. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that the input moves from any cell in the table to any other cell in the table, or moves from outside the table to a cell inside the table.

46. The non-transitory computer readable storage medium of claim 33, wherein the first element includes a table, having cells organized in rows and columns, generated at least in part based on information provided by the application-specific module for the first software application, and the first user interaction model specifies generating the first tactile output in accordance with a determination that two or more rows of the cells are reordered in response to the input.

47. The non-transitory computer readable storage medium of claim 33, wherein the input causes the first software application to select an item, in a collection of distinct items, to be moved to a predefined position within the first element, and the first user interaction model specifies generating the first tactile output in conjunction with the selected item being moved to the predefined position within the first element.

48. The non-transitory computer readable storage medium of claim 33, wherein the first user interaction model specifies generating the first tactile output in accordance with a determination that the input or an object manipulated in accordance with the input approaches a next predefined state, and generating a second tactile output, distinct from the first tactile output, in accordance with a determination that the input or an object manipulated in accordance with the input reaches the next predefined state.