AUTOMATIC SPEECH RECOGNITION BASED ON USER FEEDBACK

• US 20160063998A1
• Filed: 01/07/2015
• Published: 03/03/2016
• Est. Priority Date: 08/28/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: KRISHNAMOORTHY, Mahesh, PAULIK, Matthias
• Granted Patent: US 10,446,141 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

Systems and processes for processing speech in a digital assistant are provided. In one example process, a first speech input can be received from a user. The first speech input can be processed using a first automatic speech recognition system to produce a first recognition result. An input indicative of a potential error in the first recognition result can be received. The input can be used to improve the first recognition result. For example, the input can include a second speech input that is a repetition of the first speech input. The second speech input can be processed using a second automatic speech recognition system to produce a second recognition result.

Claims

1. A method for processing speech in a digital assistant, the method comprising:

at an electronic device with a processor and memory storing one or more programs for execution by the processor;

receiving, from a user, a first speech input;

processing the first speech input using a first automatic speech recognition system to produce a first recognition result;

receiving, from the user, an input indicative of a potential error in the first recognition result, wherein the input includes a second speech input; and

processing the second speech input using a second automatic speech recognition system to produce a second recognition result.

2. The method of claim 1, wherein the second speech input is a repetition of at least a portion of the first speech input.

3. The method of claim 1, further comprising:

determining whether the second speech input comprises a repetition of at least a portion of the first speech input, wherein the second speech input is processed using the second automatic speech recognition system to produce the second recognition result in response to determining that the second speech input comprises a repetition of at least a portion of the first speech input.

4. The method of claim 3, wherein determining whether the second speech input comprises a repetition of at least a portion of the first speech input comprises:

determining whether an error rate of a phonemic transcription of the second speech input with respect to a phonemic transcription of a corresponding portion of the first speech input is less than a predetermined value.

5. The method of claim 3, wherein determining whether the second speech input comprises a repetition of at least a portion of the first speech input comprises:

comparing an audio waveform of the second speech input with an audio waveform of a corresponding portion of the first speech input.

6. The method of claim 1, further comprising:

determining a combined result based on the first recognition result and the second recognition result.

7. A method for processing speech in a digital assistant, the method comprising:

at an electronic device with a processor and memory storing one or more programs for execution by the processor;

receiving, from a user, a first speech input;

processing the first speech input using a first automatic speech recognition system to produce a first recognition result;

receiving, from the user, an input indicative of a potential error in the first recognition result;

prompting the user to repeat at least a portion of the first speech input;

receiving, from the user, a second speech input; and

processing the second speech input using a second automatic speech recognition system to produce a second recognition result.

8. The method of claim 7, wherein the input is a speech input that includes a predetermined utterance.

9. The method of claim 7, wherein the input is a selection of an affordance.

10. The method of claim 7, wherein at least a portion of text of the first recognition result is displayed on the electronic device, and wherein the input is a selection of at least a portion of the displayed text.

11. The method of claim 7, wherein the input is associated with a rejection of a proposed task.

12. The method of claim 7, further comprising:

identifying a portion of the first speech input corresponding to the potential error in the first recognition result.

13. The method of claim 12, wherein processing the first speech input using the first automatic speech recognition system includes determining a confidence measure of each word in a text of the first recognition result, and wherein the portion of the first speech input associated with the potential error is identified based on the confidence measure of each word in the text.

14. The method of claim 12, wherein the user is prompted to repeat the identified portion of the first speech input corresponding to the potential error.

15. The method of claim 7, further comprising:

determining a combined result based on the first recognition result and the second recognition result.

16. A method for processing speech in a digital assistant, the method comprising:

at an electronic device with a processor and memory storing one or more programs for execution by the processor;

receiving, from a user, a speech input;

processing the speech input using a first automatic speech recognition system to produce a first recognition result;

receiving, from the user, an input indicative of a potential error in the first recognition result; and

processing at least a portion of the speech input using a second automatic speech recognition system to produce a second recognition result.

17. The method of claim 16, wherein an error rate of the second automatic speech recognition system is lower than an error rate of the first automatic speech recognition system.

18. The method of claim 16, wherein a latency of the second automatic speech recognition system is greater than a latency of the first automatic speech recognition system.

19. The method of claim 16, further comprising:

identifying a portion of the speech input corresponding to the potential error in the first recognition result, wherein the identified portion of the speech input corresponding to the potential error is processed using the second automatic speech recognition system to produce the second recognition result.

20. The method of claim 16, further comprising:

performing an first action based on the first recognition result; and

performing an second action based on the second recognition result.

21. The method of claim 16, further comprising:

determining a combined result based on the first recognition result and the second recognition result.

22. The method of claim 21, wherein the combined result is determined by performing automatic speech recognition system combination using the first recognition result and the second recognition result.

23. The method of claim 22, wherein performing automatic speech recognition system combination comprises implementing at least one of recognition output voting error reduction, cross-adaptation, confusion network combination, and lattice combination.

24. A non-transitory computer-readable storage medium comprising instructions for causing one or more processor to:

receiving, from a user, a first speech input;

processing the first speech input using a first automatic speech recognition system to produce a first recognition result;

receiving, from the user, an input indicative of a potential error in the first recognition result, wherein the input includes a second speech input; and

processing the second speech input using a second automatic speech recognition system to produce a second recognition result.

25. An electronic device comprising:

one or more processors;

memory;

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving, from a user, a first speech input;

processing the first speech input using a first automatic speech recognition system to produce a first recognition result;

receiving, from the user, an input indicative of a potential error in the first recognition result, wherein the input includes a second speech input; and

processing the second speech input using a second automatic speech recognition system to produce a second recognition result.