Semantic framework for variable haptic output

• US 10,504,340 B2
• Filed: 11/27/2017
• Issued: 12/10/2019
• Est. Priority Date: 09/02/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Moussette, Camille, Butcher, Gary I., Verweij, Hugo D., Ive, Jonathan
• Primary Examiner(s): Swarthout, Brent
• Application Number:
  US15/823,436
• Publication Number:
  US 20180082552A1
• Time in Patent Office: 743 Days
• Field of Search: 3404071, 3404072, 345156, 345173
• CPC Class Codes: G06F 11/0745 in an input/output transactions management context input/output processing in general G06F13/00
• CPC Class Codes: G06F 11/0763 by bit configuration check, e.g. of formats or tags
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G08B 7/06 using electric transmission , e.g. involving audible and visible signalling through the use of sound and light sources
• CPC Class Codes: H04L 51/224 providing notification on incoming messages, e.g. pushed notifications of received messages
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 19/04 the ringing-current being generated at the substations

Abstract

Methods and apparatus organize a plurality of haptic output variations into a cohesive semantic framework that uses various information about the alert condition and trigger, application context, and other conditions to provide a system of haptic outputs that share characteristics between related events. In some embodiments, an event class or application class provides the basis for a corresponding haptic output. In some embodiments, whether an alert-salience setting is on provides the basis for adding an increased salience haptic output to the standard haptic output for the alert. In some embodiments, consistent haptics provide for branding of the associated application class, application, and/or context.

Claims

1. A processor-implemented method performed on a computing device, the method comprising:

detecting an alert condition associated with an application executing on the computing device; and

responsive to detecting the alert condition;

determining a state associated with the application at a time associated with the alert condition, wherein determining the state associated with the application at the time associated with the alert condition comprises determining whether a user interface for the application was displayed on a touch screen of the computing device at the time associated with the alert condition;

in accordance with a determination that the application was in an active state at the time associated with the alert condition, providing a first haptic output that represents the occurrence of the alert condition, the first haptic output having a first set of output characteristics, wherein the active state of the application corresponds to the user interface for the application being displayed on the touch screen of the computing device; and

in accordance with a determination that the application was in an inactive state at the time associated with the alert condition, providing a second haptic output that represents the occurrence of the alert condition, the second haptic output having a second set of output characteristics, wherein the second haptic output is different from the first haptic output, wherein the inactive state of the application corresponds to the user interface for the application not being displayed on the touch screen of the computing device.

2. The method of claim 1, wherein:

providing the first haptic output in accordance with the determination that the application was in the active state at the time associated with the alert condition further includes;

determining a level of user engagement associated with the user interaction at the time of the alert condition; and

determining one or more of the first set of output characteristics for the first haptic output based on the determined level of engagement.

3. The method of claim 1, wherein:

one or more user interface windows are concurrently displayed on a multi-application-window user interface on the computing device at the time associated with the alert condition; and

determining the state associated with the application at the time associated with the alert condition includes determining whether a respective user interface window corresponding to the application was displayed in the multi-application-window user interface at the time associated with the alert condition.

4. The method of claim 1, wherein the alert condition corresponds to an event that is automatically triggered by the application, an automatically initiated event notification received by the application from a source external to the computing device, or a manually initiated event notification received by the application from a human user other than a user operating the computing device.

5. The method of claim 1, wherein:

the first haptic output is accompanied by a first audio output;

the second haptic output is accompanied by a second audio output; and

the first audio output is different from the second audio output.

6. The method of claim 1, wherein:

the first haptic output is accompanied by an audio output;

the second haptic output is accompanied by the audio output; and

the first haptic output is different from the second haptic output.

7. The method of claim 1, wherein:

the second haptic output is accompanied by an audio output, a waveform of the second haptic output generated based on, mirroring, and synchronized to a waveform of the accompanying audio output.

8. The method of claim 1, wherein:

the first haptic output is not accompanied by an audio output; and

the second haptic output is accompanied by an audio output.

9. The method of claim 1, wherein the second haptic output has a greater intensity than the first haptic output.

10. The method of claim 1, wherein the first set of characteristics comprises one or more of:

an amplitude, a duration, a regularity, a repetition frequency, or a selection of haptic features of the first haptic output.

11. The method of claim 1, wherein the second haptic output is different and has a greater intensity than the first haptic output.

12. The method of claim 1, wherein the first and second sets of output characteristics correspond to a device type of the computing device.

13. The method of claim 1, further comprising:

at a first point in time, while the application is in the active state, detecting a first alert condition associated with the application;

responsive to detecting the first alert condition while the first application is in the active state, providing a first output representing the first alert condition;

at a second point in time, while the first application is in an inactive state, detecting a second alert condition;

responsive to detecting the second alert condition while that the application is in the inactive state, providing the second haptic output representing the second alert condition, the second haptic output different from the first haptic output.

14. The method of claim 1, wherein:

the first haptic output and the second haptic output are provided via the touch screen of the computing device.

15. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by a computing device, cause the computing device to perform operations comprising:

detecting an alert condition associated with an application executing on the computing device; and

responsive to detecting the alert condition;

determining a state associated with the application at a time associated with the alert condition, wherein determining the state associated with the application at the time associated with the alert condition comprises determining whether a user interface for the application was displayed on a touch screen of the computing device at the time associated with the alert condition;

in accordance with a determination that the application was in an active state at the time associated with the alert condition, providing a first haptic output that represents the occurrence of the alert condition, the first haptic output having a first set of output characteristics, wherein the active state of the application corresponds to the user interface for the application being displayed on the touch screen of the computing device; and

in accordance with a determination that the application was in an inactive state at the time associated with the alert condition, providing a second haptic output that represents the occurrence of the alert condition, the second haptic output having a second set of output characteristics, wherein the second haptic output is different from the first haptic output, wherein the inactive state of the application corresponds to the user interface for the application not being displayed on the touch screen of the computing device.

17. The non-transitory computer readable storage medium of claim 15, wherein:

providing the first haptic output in accordance with the determination that the application was in the active state at the time associated with the alert condition further includes;

determining a level of user engagement associated with the user interaction at the time of the alert condition; and

determining one or more of the first set of output characteristics for the first haptic output based on the determined level of engagement.

18. The non-transitory computer readable storage medium of claim 15, wherein:

one or more user interface windows are concurrently displayed on a multi-application-window user interface on the computing device at the time associated with the alert condition; and

determining the state associated with the application at the time associated with the alert condition includes determining whether a respective user interface window corresponding to the application was displayed in the multi-application-window user interface at the time associated with the alert condition.

19. The non-transitory computer readable storage medium of claim 15, wherein the alert condition corresponds to an event that is automatically triggered by the application, an automatically initiated event notification received by the application from a source external to the computing device, or a manually initiated event notification received by the application from a human user other than a user operating the computing device.

20. The non-transitory computer readable storage medium of claim 15, wherein:

the first haptic output is accompanied by a first audio output;

the second haptic output is accompanied by a second audio output; and

the first audio output is different from the second audio output.

21. The non-transitory computer readable storage medium of claim 15, wherein:

the first haptic output is accompanied by an audio output;

the second haptic output is accompanied by the audio output; and

the first haptic output is different from the second haptic output.

22. The non-transitory computer readable storage medium of claim 15, wherein:

the second haptic output is accompanied by an audio output, a waveform of the second haptic output generated based on, mirroring, and synchronized to a waveform of the accompanying audio output.

23. The non-transitory computer readable storage medium of claim 15, wherein:

the first haptic output is not accompanied by an audio output; and

the second haptic output is accompanied by an audio output.

24. The non-transitory computer readable storage medium of claim 15, wherein the second haptic output has a greater intensity than the first haptic output.

25. The non-transitory computer readable storage medium of claim 15, wherein the first set of characteristics comprises one or more of:

an amplitude, a duration, a regularity, a repetition frequency, or a selection of haptic features of the first haptic output.

26. The non-transitory computer readable storage medium of claim 15, wherein the second haptic output is different and has a greater intensity than the first haptic output.

27. The non-transitory computer readable storage medium of claim 15, wherein the first and second sets of output characteristics correspond to a device type of the computing device.

28. The non-transitory computer readable storage medium of claim 15, wherein the one or more programs include instructions that, when executed by the computing device, cause the computing device to perform operations comprising:

at a first point in time, while the application is in the active state, detecting a first alert condition associated with the application;

responsive to detecting the first alert condition while the first application is in the active state, providing a first output representing the first alert condition;

at a second point in time, while the first application is in an inactive state, detecting a second alert condition; and

responsive to detecting the second alert condition while that the application is in the inactive state, providing the second haptic output representing the second alert condition, the second haptic output different from the first haptic output.

29. The non-transitory computer readable storage medium of claim 15, wherein:

the first haptic output and the second haptic output are provided via the touch screen of the computing device.

16. A computing device comprising:

a display;

a touch screen; and

memory storing one or more programs that include instructions for;

detecting an alert condition associated with an application executing on the computing device; and

responsive to detecting the alert condition;

determining a state associated with the application at a time associated with the alert condition, wherein determining the state associated with the application at the time associated with the alert condition comprises determining whether a user interface for the application was displayed on the touch screen of the computing device at the time associated with the alert condition;

in accordance with a determination that the application was in an active state at the time associated with the alert condition, providing a first haptic output that represents the occurrence of the alert condition, the first haptic output having a first set of output characteristics, wherein the active state of the application corresponds to the user interface for the application being displayed on the touch screen of the computing device; and

in accordance with a determination that the application was in an inactive state at the time associated with the alert condition, providing a second haptic output that represents the occurrence of the alert condition, the second haptic output having a second set of output characteristics, wherein the second haptic output is different from the first haptic output, wherein the inactive state of the application corresponds to the user interface for the application not being displayed on the touch screen of the computing device.

30. The computing device of claim 16, wherein:

providing the first haptic output in accordance with the determination that the application was in the active state at the time associated with the alert condition further includes;

determining a level of user engagement associated with the user interaction at the time of the alert condition; and

determining one or more of the first set of output characteristics for the first haptic output based on the determined level of engagement.

31. The computing device of claim 16, wherein:

one or more user interface windows are concurrently displayed on a multi-application-window user interface on the computing device at the time associated with the alert condition; and

determining the state associated with the application at the time associated with the alert condition includes determining whether a respective user interface window corresponding to the application was displayed in the multi-application-window user interface at the time associated with the alert condition.

32. The computing device of claim 16, wherein the alert condition corresponds to an event that is automatically triggered by the application, an automatically initiated event notification received by the application from a source external to the computing device, or a manually initiated event notification received by the application from a human user other than a user operating the computing device.

33. The computing device of claim 16, wherein:

the first haptic output is accompanied by a first audio output;

the second haptic output is accompanied by a second audio output; and

the first audio output is different from the second audio output.

34. The computing device of claim 16, wherein:

the first haptic output is accompanied by an audio output;

the second haptic output is accompanied by the audio output; and

the first haptic output is different from the second haptic output.

35. The computing device of claim 16, wherein:

the second haptic output is accompanied by an audio output, a waveform of the second haptic output generated based on, mirroring, and synchronized to a waveform of the accompanying audio output.

36. The computing device of claim 16, wherein:

the first haptic output is not accompanied by an audio output; and

the second haptic output is accompanied by an audio output.

37. The computing device of claim 16, wherein the second haptic output has a greater intensity than the first haptic output.

38. The computing device of claim 16, wherein the first set of characteristics comprises one or more of:

an amplitude, a duration, a regularity, a repetition frequency, or a selection of haptic features of the first haptic output.

39. The computing device of claim 16, wherein the second haptic output is different and has a greater intensity than the first haptic output.

40. The computing device of claim 16, wherein the first and second sets of output characteristics correspond to a device type of the computing device.

41. The computing device of claim 16, wherein the one or more programs include instructions for:

at a first point in time, while the application is in the active state, detecting a first alert condition associated with the application;

responsive to detecting the first alert condition while the first application is in the active state, providing a first output representing the first alert condition;

at a second point in time, while the first application is in an inactive state, detecting a second alert condition; and

responsive to detecting the second alert condition while that the application is in the inactive state, providing the second haptic output representing the second alert condition, the second haptic output different from the first haptic output.

42. The computing device of claim 16, wherein:

the first haptic output and the second haptic output are provided via the touch screen of the computing device.