SYSTEM AND METHOD FOR DETECTING ERRORS IN INTERACTIONS WITH A VOICE-BASED DIGITAL ASSISTANT

• US 20140365226A1
• Filed: 06/05/2014
• Published: 12/11/2014
• Est. Priority Date: 06/07/2013
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: SINHA, Anoop K.
• Granted Patent: US 9,633,674 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 2250/74 with voice recognition means voice activated dialling H04M1/271; voice recognition algorithms G10L15/00

Abstract

The method is performed at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors. A speech input containing a request is received from a user. At least one action in furtherance of satisfying the request is performed. A user interaction is detected, such as a speech input to a digital assistant or a physical interaction with a device. It is determined whether the user interaction is indicative of a problem in the performing of the at least one action. Upon determining that the user interaction is indicative of a problem, information relating to the request is stored in a repository for error analysis.

Claims

1. A method for operating an intelligent automated assistant, comprising:

at an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors;

receiving, from a user, a speech input containing a request;

performing at least one action in furtherance of satisfying the request;

detecting a user interaction;

determining whether the user interaction is indicative of a problem in the performing of the at least one action;

upon determining that the user interaction is indicative of a problem, storing information relating to the request in a repository for error analysis.

2. The method of claim 1, wherein detecting the user interaction comprises detecting an additional speech input, and determining whether the user interaction is indicative of a problem comprises determining that the additional speech input indicates dissatisfaction with the at least one action.

3. The method of claim 2, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes at least one predefined word.

4. The method of claim 2, wherein determining whether the additional speech input indicates dissatisfaction includes determining a volume of the additional speech input.

5. The method of claim 4, wherein the additional speech input indicates dissatisfaction when the volume of the additional speech input is above a predefined threshold.

6. The method of claim 1, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input contains prosodic indications of frustration.

7. The method of claim 2, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes substantially the same words as the first speech input.

8. The method of claim 1, wherein the user interaction comprises one or more of:

a predefined motion of the device;

a selection of an affordance;

a termination of a dialog session with the intelligent automated assistant;

or

a rejection of a proposed task.

9. The method of claim 1, further comprising, upon determining that the user interaction is indicative of a problem:

providing a prompt requesting the user to confirm whether there was a problem in the performing of the at least one action; and

receiving, from the user, a confirmation or a disconfirmation of whether there was a problem in the performing of the at least one action.

10. The method of claim 1, wherein the repository includes a plurality of entries from a plurality of users, the method further comprising:

analyzing the repository to identify a set of entries, each entry of the set of entries having one or more similar characteristics indicative of an error; and

adjusting one or more of a speech-to-text module and a natural language processing module based on the set of entries so as to reduce reproduction of the error.

11. An electronic device, comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving, from a user, a speech input containing a request;

performing at least one action in furtherance of satisfying the request;

detecting a user interaction;

determining whether the user interaction is indicative of a problem in the performing of the at least one action;

upon determining that the user interaction is indicative of a problem, storing information relating to the request in a repository for error analysis.

12. The electronic device of claim 11, wherein detecting the user interaction comprises detecting an additional speech input, and determining whether the user interaction is indicative of a problem comprises determining that the additional speech input indicates dissatisfaction with the at least one action.

13. The electronic device of claim 12, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes at least one predefined word.

14. The electronic device of claim 12, wherein determining whether the additional speech input indicates dissatisfaction includes determining a volume of the additional speech input.

15. The electronic device of claim 14, wherein the additional speech input indicates dissatisfaction when the volume of the additional speech input is above a predefined threshold.

16. The electronic device of claim 11, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input contains prosodic indications of frustration.

17. The electronic device of claim 12, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes substantially the same words as the first speech input.

18. The electronic device of claim 11, wherein the user interaction comprises one or more of:

a predefined motion of the device;

a selection of an affordance;

a termination of a dialog session with the intelligent automated assistant;

or

a rejection of a proposed task.

19. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with one or more processors and memory, cause the device to:

receive, from a user, a speech input containing a request;

perform at least one action in furtherance of satisfying the request;

detect a user interaction;

determine whether the user interaction is indicative of a problem in the performing of the at least one action;

upon determining that the user interaction is indicative of a problem, store information relating to the request in a repository for error analysis.

20. The non-transitory computer readable storage medium of claim 19, wherein detecting the user interaction comprises detecting an additional speech input, and determining whether the user interaction is indicative of a problem comprises determining that the additional speech input indicates dissatisfaction with the at least one action.

21. The non-transitory computer readable storage medium of claim 20, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes at least one predefined word.

22. The non-transitory computer readable storage medium of claim 20, wherein determining whether the additional speech input indicates dissatisfaction includes determining a volume of the additional speech input.

23. The non-transitory computer readable storage medium of claim 22, wherein the additional speech input indicates dissatisfaction when the volume of the additional speech input is above a predefined threshold.

24. The non-transitory computer readable storage medium of claim 19, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input contains prosodic indications of frustration.

25. The non-transitory computer readable storage medium of claim 20, wherein determining whether the additional speech input indicates dissatisfaction includes determining whether the additional speech input includes substantially the same words as the first speech input.