Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant

• US 11,048,473 B2
• Filed: 11/29/2018
• Issued: 06/29/2021
• Est. Priority Date: 06/09/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Carson, David A., Keen, Daniel, Dibiase, Evan, Saddler, Harry J., Iacono, Marco, Lemay, Stephen O., Pitschel, Donald W., Gruber, Thomas R.
• Primary Examiner(s): Kumar, Anil N
• Application Number:
  US16/204,178
• Publication Number:
  US 20190095171A1
• Time in Patent Office: 943 Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

An electronic device with one or more processors and memory includes a procedure for enabling conversation persistence across two or more instances of a digital assistant. In some embodiments, the device displays a first dialogue in a first instance of a digital assistant user interface. In response to a request to display a user interface different from the digital assistant user interface, the device displays the user interface different from the digital assistant user interface. In response to a request to invoke the digital assistant, the device displays a second instance of the digital assistant user interface, including displaying a second dialogue in the second instance of the digital assistant user interface, where the first dialogue remains available for display in the second instance of the digital assistant user interface.

Claims

1. A method of operating a digital assistant, performed at an electronic device having one or more processors and memory storing one or more programs for execution by the one or more processors, the method comprising:

while displaying a first user interface other than a digital assistant user interface, receiving a user request to invoke the digital assistant;

in response to receiving the user request to invoke the digital assistant, initiating a current interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a spoken query;

determining whether the user request to invoke the digital assistant satisfies a first predetermined condition, wherein the user request to invoke the digital assistant does not satisfy the first predetermined condition if the electronic device is in a locked state while receiving the user request to invoke the digital assistant; and

in accordance with a determination that the user request to invoke the digital assistant satisfies the first predetermined condition;

using context data from a previous interactive session with the digital assistant to determine a response to the spoken query, wherein the previous interactive session was initiated on the electronic device prior to displaying the first user interface; and

displaying dialogue in the displayed digital assistant user interface, wherein the displayed dialogue includes first text corresponding to the spoken query and second text corresponding to the response to the spoken query.

2. The method of claim 1, wherein satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a predetermined time period after the previous interactive session with the digital assistant ends.

3. The method of claim 2, wherein the previous interactive session with the digital assistant ends upon detecting, prior to displaying the first user interface, a user request to end the previous interactive session with the digital assistant.

4. The method of claim 3, wherein initiating the previous interactive session includes displaying the digital assistant user interface, and wherein detecting the user request to end the previous interactive session with the digital assistant causes the electronic device to:

cease display of the digital assistant user interface; and

display the first user interface.

5. The method of claim 1, further comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

determining a second response to the spoken query without using the context data; and

displaying a second dialogue in the displayed digital assistant user interface, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to the second response to the spoken query.

6. The method of claim 1, wherein prior to displaying the first user interface, the electronic device causes the context data to be stored in a context history of the digital assistant, and wherein the method further comprises:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition, causing the stored context data to be removed from the context history.

7. The method of claim 1, further comprising:

determining whether the user request to invoke the digital assistant satisfies a second predetermined condition;

in accordance with a determination that the user request to invoke the digital assistant satisfies the second predetermined condition and does not satisfy the first predetermined condition;

enabling the displayed digital assistant user interface to display previous dialogue from the previous interactive session; and

displaying a second dialogue in the displayed digital assistant user interface without displaying the previous dialogue, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to a second response to the spoken query, and wherein the second response to the spoken query is determined without using the context data.

8. The method of claim 7, further comprising:

detecting user input via the displayed digital assistant user interface;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant satisfies the second predetermined condition, displaying the previous dialogue in the displayed digital assistant user interface;

while displaying the previous dialogue, receiving a user request to activate a microphone of the electronic device;

in accordance with receiving the user request to activate the microphone, obtaining second audio data containing a second spoken query;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

using the context data from the previous interactive session to determine a response to the second spoken query; and

displaying a third dialogue in the displayed digital assistant user interface, wherein the third dialogue includes first text corresponding to the second spoken query and second text corresponding to the response to the second spoken query.

25. The method of claim 5, wherein:

displaying the second dialogue in the displayed digital assistant user interface is performed further in accordance with a determination that the electronic device is in the locked state while receiving the user request to invoke the digital assistant; and

previous dialogue from the previous interactive session is not available for display in the displayed digital assistant user interface when the displayed digital assistant user interface displays the second dialogue.

26. The method of claim 5, further comprising:

after displaying the second dialogue in the displayed digital assistant user interface, and while the electronic device is in the locked state;

receiving a user input to unlock the electronic device from the locked state;

in response to receiving the user input to unlock the electronic device, changing a state of the electronic device from the locked state to an unlocked state;

while the electronic device is in the unlocked state;

receiving a second user request to invoke the digital assistant; and

in response to receiving the second user request to invoke the digital assistant, initiating a second interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a second spoken query; and

using the context data from the previous interactive session with the digital assistant to determine a third response to the second spoken query.

27. The method of claim 7, wherein prior to displaying the first user interface, the electronic device causes the previous dialogue to be stored in a conversation history of the digital assistant, the method further comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the second predetermined condition, causing the previous dialog to be removed from the conversation history.

28. The method of claim 7, wherein:

satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a first predetermined time period after the previous interactive session with the digital assistant ends;

satisfying the second predetermined condition requires that the user request to invoke the digital assistant is received within a second predetermined time period after the previous interactive session with the digital assistant ends; and

the second predetermined time period is longer than the first predetermined time period.

29. The method of claim 28, wherein the first predetermined time period and the second predetermined time period are each user defined.

9. A non-transitory computer-readable storage medium storing instructions for operating a digital assistant, the instructions, when executed by one or more processors of an electronic device, cause the device to perform operations comprising:

while displaying a first user interface other than a digital assistant user interface, receiving a user request to invoke the digital assistant;

in response to receiving the user request to invoke the digital assistant, initiating a current interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a spoken query;

determining whether the user request to invoke the digital assistant satisfies a first predetermined condition, wherein the user request to invoke the digital assistant does not satisfy the first predetermined condition if the device is in a locked state while receiving the user request to invoke the digital assistant; and

in accordance with a determination that the user request to invoke the digital assistant satisfies the first predetermined condition;

using context data from a previous interactive session with the digital assistant to determine a response to the spoken query, wherein the previous interactive session was initiated on the electronic device prior to displaying the first user interface; and

displaying dialogue in the displayed digital assistant user interface, wherein the displayed dialogue includes first text corresponding to the spoken query and second text corresponding to the response to the spoken query.

10. The computer-readable storage medium of claim 9, wherein satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a predetermined time period after the previous interactive session with the digital assistant ends.

11. The computer-readable storage medium of claim 10, wherein the previous interactive session with the digital assistant ends upon detecting, prior to displaying the first user interface, a user request to end the previous interactive session with the digital assistant.

12. The computer-readable storage medium of claim 11, wherein initiating the previous interactive session includes displaying the digital assistant user interface, and wherein detecting the user request to end the previous interactive session with the digital assistant causes the electronic device to:

cease display of the digital assistant user interface; and

display the first user interface.

13. The computer-readable storage medium of claim 9, wherein the instructions further cause the device to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

determining a second response to the spoken query without using the context data; and

displaying a second dialogue in the displayed digital assistant user interface, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to the second response to the spoken query.

14. The computer-readable storage medium of claim 9, wherein prior to displaying the first user interface, the device causes the context data to be stored in a context history of the digital assistant, and wherein the instructions further cause the device to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition, causing the stored context data to be removed from the context history.

15. The computer-readable storage medium of claim 9, wherein the instructions further cause the device to perform operations comprising:

determining whether the user request to invoke the digital assistant satisfies a second predetermined condition;

in accordance with a determination that the user request to invoke the digital assistant satisfies the second predetermined condition and does not satisfy the first predetermined condition;

enabling the displayed digital assistant user interface to display previous dialogue from the previous interactive session; and

displaying a second dialogue in the displayed digital assistant user interface without displaying the previous dialogue, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to a second response to the spoken query, and wherein the second response to the spoken query is determined without using the context data.

16. The computer-readable storage medium of claim 15, wherein the instructions further cause the device to perform operations comprising:

detecting user input via the displayed digital assistant user interface;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant satisfies the second predetermined condition, displaying the previous dialogue in the displayed digital assistant user interface;

while displaying the previous dialogue, receiving a user request to activate a microphone of the electronic device;

in accordance with receiving the user request to activate the microphone, obtaining second audio data containing a second spoken query;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

using the context data from the previous interactive session to determine a response to the second spoken query; and

displaying a third dialogue in the displayed digital assistant user interface, wherein the third dialogue includes first text corresponding to the second spoken query and second text corresponding to the response to the second spoken query.

30. The computer-readable storage medium of claim 13, wherein:

displaying the second dialogue in the displayed digital assistant user interface is performed further in accordance with a determination that the electronic device is in the locked state while receiving the user request to invoke the digital assistant; and

previous dialogue from the previous interactive session is not available for display in the displayed digital assistant user interface when the displayed digital assistant user interface displays the second dialogue.

31. The computer-readable storage medium of claim 13, wherein the instructions further cause the device to perform operations comprising:

after displaying the second dialogue in the displayed digital assistant user interface, and while the electronic device is in the locked state;

receiving a user input to unlock the electronic device from the locked state;

in response to receiving the user input to unlock the electronic device, changing a state of the electronic device from the locked state to an unlocked state;

while the electronic device is in the unlocked state;

receiving a second user request to invoke the digital assistant; and

in response to receiving the second user request to invoke the digital assistant, initiating a second interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a second spoken query; and

using the context data from the previous interactive session with the digital assistant to determine a third response to the second spoken query.

32. The computer-readable storage medium of claim 15, wherein prior to displaying the first user interface, the electronic device causes the previous dialogue to be stored in a conversation history of the digital assistant, and wherein the instructions further cause the device to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the second predetermined condition, causing the previous dialog to be removed from the conversation history.

33. The computer-readable storage medium of claim 15, wherein:

satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a first predetermined time period after the previous interactive session with the digital assistant ends;

satisfying the second predetermined condition requires that the user request to invoke the digital assistant is received within a second predetermined time period after the previous interactive session with the digital assistant ends; and

the second predetermined time period is longer than the first predetermined time period.

34. The computer-readable storage medium of claim 33, wherein the first predetermined time period and the second predetermined time period are each user defined.

17. An electronic device comprising:

one or more processors; and

memory storing instructions for operating a digital assistant, the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

while displaying a first user interface other than a digital assistant user interface, receiving a user request to invoke the digital assistant;

in response to receiving the user request to invoke the digital assistant, initiating a current interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a spoken query;

determining whether the user request to invoke the digital assistant satisfies a first predetermined condition, wherein the user request to invoke the digital assistant does not satisfy the first predetermined condition if the electronic device is in a locked state while receiving the user request to invoke the digital assistant; and

in accordance with a determination that the user request to invoke the digital assistant satisfies the first predetermined condition;

using context data from a previous interactive session with the digital assistant to determine a response to the spoken query, wherein the previous interactive session was initiated on the electronic device prior to displaying the first user interface; and

displaying dialogue in the displayed digital assistant user interface, wherein the displayed dialogue includes first text corresponding to the spoken query and second text corresponding to the response to the spoken query.

18. The device of claim 17, wherein satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a predetermined time period after the previous interactive session with the digital assistant ends.

19. The device of claim 18, wherein the previous interactive session with the digital assistant ends upon detecting, prior to displaying the first user interface, a user request to end the previous interactive session with the digital assistant.

20. The device of claim 19, wherein initiating the previous interactive session includes displaying the digital assistant user interface, and wherein detecting the user request to end the previous interactive session with the digital assistant causes the device to:

cease display of the digital assistant user interface; and

display the first user interface.

21. The device of claim 17, wherein the instructions further cause the one or more processors to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

determining a second response to the spoken query without using the context data; and

displaying a second dialogue in the displayed digital assistant user interface, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to the second response to the spoken query.

22. The device of claim 17, wherein prior to displaying the first user interface, the one or more processors cause the context data to be stored in a context history of the digital assistant, and wherein the instructions further cause the one or more processors to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition, causing the stored context data to be removed from the context history.

23. The device of claim 17, wherein the instructions further cause the one or more processors to perform operations comprising:

determining whether the user request to invoke the digital assistant satisfies a second predetermined condition;

in accordance with a determination that the user request to invoke the digital assistant satisfies the second predetermined condition and does not satisfy the first predetermined condition;

enabling the displayed digital assistant user interface to display previous dialogue from the previous interactive session; and

displaying a second dialogue in the displayed digital assistant user interface without displaying the previous dialogue, wherein the second dialogue includes first text corresponding to the spoken query and second text corresponding to a second response to the spoken query, and wherein the second response to the spoken query is determined without using the context data.

24. The device of claim 23, wherein the instructions further cause the one or more processors to perform operations comprising:

detecting user input via the displayed digital assistant user interface;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant satisfies the second predetermined condition, displaying the previous dialogue in the displayed digital assistant user interface;

while displaying the previous dialogue, receiving a user request to activate a microphone of the electronic device;

in accordance with receiving the user request to activate the microphone, obtaining second audio data containing a second spoken query;

in accordance with detecting the user input and in accordance with the determination that the user request to invoke the digital assistant does not satisfy the first predetermined condition;

using the context data from the previous interactive session to determine a response to the second spoken query; and

displaying a third dialogue in the displayed digital assistant user interface, wherein the third dialogue includes first text corresponding to the second spoken query and second text corresponding to the response to the second spoken query.

35. The electronic device of claim 21, wherein the instructions further cause the one or more processors to perform operations comprising:

displaying the second dialogue in the displayed digital assistant user interface is performed further in accordance with a determination that the electronic device is in the locked state while receiving the user request to invoke the digital assistant; and

previous dialogue from the previous interactive session is not available for display in the displayed digital assistant user interface when the displayed digital assistant user interface displays the second dialogue.

36. The electronic device of claim 21, wherein the instructions further cause the one or more processors to perform operations comprising:

after displaying the second dialogue in the displayed digital assistant user interface, and while the electronic device is in the locked state;

receiving a user input to unlock the electronic device from the locked state;

in response to receiving the user input to unlock the electronic device, changing a state of the electronic device from the locked state to an unlocked state;

while the electronic device is in the unlocked state;

receiving a second user request to invoke the digital assistant; and

in response to receiving the second user request to invoke the digital assistant, initiating a second interactive session with the digital assistant, including;

displaying the digital assistant user interface; and

obtaining audio data containing a second spoken query; and

using the context data from the previous interactive session with the digital assistant to determine a third response to the second spoken query.

37. The electronic device of claim 23, wherein prior to displaying the first user interface, the electronic device causes the previous dialogue to be stored in a conversation history of the digital assistant, and wherein the instructions further cause the one or more processors to perform operations comprising:

in accordance with a determination that the user request to invoke the digital assistant does not satisfy the second predetermined condition, causing the previous dialog to be removed from the conversation history.

38. The electronic device of claim 23, wherein:

satisfying the first predetermined condition requires that the user request to invoke the digital assistant is received within a first predetermined time period after the previous interactive session with the digital assistant ends;

satisfying the second predetermined condition requires that the user request to invoke the digital assistant is received within a second predetermined time period after the previous interactive session with the digital assistant ends; and

the second predetermined time period is longer than the first predetermined time period.

39. The electronic device of claim 38, wherein the first predetermined time period and the second predetermined time period are each user defined.