ATTENTION AWARE VIRTUAL ASSISTANT DISMISSAL

• US 20200285327A1
• Filed: 05/27/2020
• Published: 09/10/2020
• Est. Priority Date: 06/01/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: HINDI, Musa M., DASARI, Rohit, TRAN, Trungtin
• Granted Patent: US 11,009,970 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/3697 Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

Systems and processes for operating an intelligent automated assistant are provided. An example process includes initiating a virtual assistant session responsive to receiving user input. In accordance with initiating the virtual assistant session, the process includes determining, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing expressed user disinterest are satisfied. In accordance with determining that the one or more criteria representing expressed user disinterest are satisfied prior to a first time, the process includes automatically deactivating the virtual assistant session prior to the first time. The first time is defined by a setting of the electronic device. In accordance with determining that the one or more criteria representing expressed user disinterest are not satisfied prior to the first time, the process includes automatically deactivating the virtual assistant session at the first time.

Claims

1. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

initiate a virtual assistant session responsive to receiving user input;

in accordance with initiating the virtual assistant session, determine, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing expressed user engagement are satisfied, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining that a direction of a user gaze is directed to the electronic device;

in accordance with determining that the one or more criteria representing expressed user engagement are satisfied prior to a first time, forgo deactivating the virtual assistant session at the first time, wherein the first time is a predetermined duration after a second time at which a final result for the virtual assistant session is presented; and

in accordance with determining that the one or more criteria representing expressed user engagement are not satisfied prior to the first time, deactivate the virtual assistant session at the first time.

2. The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:

determine a type of the user input; and

wherein determining whether the one or more criteria representing expressed user engagement are satisfied is performed in accordance with determining that the type of the user input is a second type of user input.

3. The non-transitory computer-readable storage medium of claim 2, wherein the second type of user input includes input received from an external electronic device or audio input.

4. The non-transitory computer-readable storage medium of claim 1, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes detecting a touch on a button or on a screen of the electronic device.

5. The non-transitory computer-readable storage medium of claim 1, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining a raising of the electronic device.

6. The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the electronic device to:

determine whether a predetermined application is executing when the user input was received; and

in accordance with a determination that the predetermined application is executing when the user input was received, deactivate the virtual assistant session a second predetermined duration after the second time.

7. The non-transitory computer-readable storage medium of claim 1, wherein initiating the virtual assistant session includes providing audio output.

8. The non-transitory computer-readable storage medium of claim 1, wherein initiating the virtual assistant session includes displaying a user interface associated with the virtual assistant session.

9. The non-transitory computer-readable storage medium of claim 8, wherein deactivating the virtual assistant session includes ceasing the display of the user interface associated with the virtual assistant session.

10. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

initiating a virtual assistant session responsive to receiving user input;

in accordance with initiating the virtual assistant session, determining, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing expressed user engagement are satisfied, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining that a direction of a user gaze is directed to the electronic device;

in accordance with determining that the one or more criteria representing expressed user engagement are satisfied prior to a first time, forgoing deactivating the virtual assistant session at the first time, wherein the first time is a predetermined duration after a second time at which a final result for the virtual assistant session is presented; and

in accordance with determining that the one or more criteria representing expressed user engagement are not satisfied prior to the first time, deactivating the virtual assistant session at the first time.

12. The electronic device of claim 10, the one or more programs further including instructions for:

determining a type of the user input; and

wherein determining whether the one or more criteria representing expressed user engagement are satisfied is performed in accordance with determining that the type of the user input is a second type of user input.

13. The electronic device of claim 12, wherein the second type of user input includes input received from an external electronic device or audio input.

14. The electronic device of claim 10, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes detecting a touch on a button or on a screen of the electronic device.

15. The electronic device of claim 10, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining a raising of the electronic device.

16. The electronic device of claim 10, the one or more programs further including instructions for:

determining whether a predetermined application is executing when the user input was received; and

in accordance with a determination that the predetermined application is executing when the user input was received, deactivating the virtual assistant session a second predetermined duration after the second time.

17. The electronic device of claim 10, wherein initiating the virtual assistant session includes providing audio output.

18. The electronic device of claim 10, wherein initiating the virtual assistant session includes displaying a user interface associated with the virtual assistant session.

19. The electronic device of claim 18, wherein deactivating the virtual assistant session includes ceasing the display of the user interface associated with the virtual assistant session.

11. A method for operating a virtual assistant, comprising:

at an electronic device;

initiating a virtual assistant session responsive to receiving user input;

in accordance with initiating the virtual assistant session, determining, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing expressed user engagement are satisfied, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining that a direction of a user gaze is directed to the electronic device;

in accordance with determining that the one or more criteria representing expressed user engagement are satisfied prior to a first time, forgoing deactivating the virtual assistant session at the first time, wherein the first time is a predetermined duration after a second time at which a final result for the virtual assistant session is presented; and

in accordance with determining that the one or more criteria representing expressed user engagement are not satisfied prior to the first time, deactivating the virtual assistant session at the first time.

20. The method of claim 11, further comprising:

determining a type of the user input; and

wherein determining whether the one or more criteria representing expressed user engagement are satisfied is performed in accordance with determining that the type of the user input is a second type of user input.

21. The method of claim 20, wherein the second type of user input includes input received from an external electronic device or audio input.

22. The method of claim 11, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes detecting a touch on a button or on a screen of the electronic device.

23. The method of claim 11, wherein determining that the one or more criteria representing expressed user engagement are satisfied includes determining a raising of the electronic device.

24. The method of claim 11, further comprising:

determining whether a predetermined application is executing when the user input was received; and

in accordance with a determination that the predetermined application is executing when the user input was received, deactivating the virtual assistant session a second predetermined duration after the second time.

25. The method of claim 11, wherein initiating the virtual assistant session includes providing audio output.

26. The method of claim 11, wherein initiating the virtual assistant session includes displaying a user interface associated with the virtual assistant session.

27. The method of claim 26, wherein deactivating the virtual assistant session includes ceasing the display of the user interface associated with the virtual assistant session.