CORRECTIVE FEEDBACK LOOP FOR AUTOMATED SPEECH RECOGNITION

• US 20090240488A1
• Filed: 03/19/2009
• Published: 09/24/2009
• Est. Priority Date: 03/19/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: YAP INC. (Amazon.com, Inc.)
• Inventors: White, Marc, Jablokov, Igor Roditis, Jablokov, Victor Roditis
• Granted Patent: US 8,352,264 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering

Abstract

A method for facilitating the updating of a language model includes receiving, at a client device, via a microphone, an audio message corresponding to speech of a user; communicating the audio message to a first remote server; receiving, that the client device, a result, transcribed at the first remote server using an automatic speech recognition system (“ASR”), from the audio message; receiving, at the client device from the user, an affirmation of the result; storing, at the client device, the result in association with an identifier corresponding to the audio message; and communicating, to a second remote server, the stored result together with the identifier.

Claims

1. A method for facilitating the updating of a language model, comprising:

(a) receiving, at a client device, via a microphone, an audio message corresponding to speech of a user;

(b) communicating, to a first remote server, the audio message;

(c) receiving, at the client device, a result, transcribed at the first remote server using an automatic speech recognition system (ASR), from the audio message;

(d) receiving, at the client device from the user, an affirmation of the result;

(e) storing, at the client device, the result in association with an identifier corresponding to the audio message; and

(f) communicating, to a second remote server, the stored result together with the identifier.

2. The method of claim 1, wherein the method further includes, subsequent to said step of communicating the stored result to the second remote server, updating, using the stored result and the identifier, a language model.

3. The method of claim 1, wherein the first remote server and the second remote server are the same remote server.

4. The method of claim 1, wherein said step of storing, at the client device, the result and the identifier comprises storing, at the client device, the result and identifier as a data stream.

5. The method of claim 4, wherein said step of communicating the stored result to the second remote server comprises communicating the data stream to the second remote server.

6. The method of claim 1, wherein the identifier corresponding to the audio message comprises the audio message.

7. The method of claim 3, wherein the identifier corresponding to the audio message is configured to allow the remote server to locate the original audio stream.

8. The method of claim 7, wherein the identifier comprises a reference that allows the remote server to locate the audio message.

9. The method of claim 1, wherein the method further includes transcribing, at the first remote server using an automatic speech recognition system (ASR), a result from the audio message.

10. The method of claim 1, wherein said step of receiving, at the client device, a result comprises receiving, at the client device, a result and alternative result matrix transcribed, at the first remote server using an automatic speech recognition system (ASR), from the audio message.

11. The method of claim 1, wherein said step of communicating, to the second remote server, the stored result together with the identifier comprises waiting, to communicate the stored result to the second remote server, until the next time the client device contacts the second remote server.

12. The method of claim 1, wherein said step of communicating, to the second remote server, the stored result together with the identifier comprises waiting for user instruction to communicate the stored result to the second remote server.

13. The method of claim 1, wherein said step of communicating, to the second remote server, the stored result together with the identifier comprises waiting, for an API call, to communicate the stored result to the second remote server.

14. A method for facilitating the updating of a language model, comprising:

(a) receiving, at a client device, via a microphone, an audio message corresponding to speech of a user;

(b) communicating, to a first remote server, the audio message;

(c) receiving, at the client device, a result and alternative result matrix transcribed, at the first remote server using an automatic speech recognition system (ASR), from the audio message;

(d) receiving, at the client device from the user, a manual correction of the result;

(e) storing, at the client device, the corrected result in association with an identifier corresponding to the audio message; and

(f) communicating, to a second remote server, the stored result together with the identifier.

15. The method of claim 14, wherein said step of receiving, at the client device, a manual correction comprises receiving, at the client device, an affirmation of an alternative fragment result of the alternative result matrix.

16. The method of claim 14, wherein said step of receiving, at the client device, a manual correction comprises receiving, at the client device, text input manually by the user using a keypad.

17. The method of claim 14, wherein said step of receiving, at the client device, a manual correction comprises receiving, at the client device, text input manually by the user using a touchscreen.

18. The method of claim 14, wherein the identifier corresponding to the audio message comprises the audio message.

19. The method of claim 18, wherein the identifier corresponding to the audio message is configured to allow the remote server to locate the original audio message.

20. A method for facilitating the updating of a language model, comprising:

(a) receiving, at a client device, via a microphone, a first audio message corresponding to speech of a user;

(b) communicating, to a first remote server, the first audio message;

(c) receiving, at the client device, a first result, transcribed at the first remote server using an automatic speech recognition system (ASR), from the first audio message;

(d) receiving, at the client device from the user, a disapproval of the first result;

(e) receiving, at a client device, via a microphone, a second audio message corresponding to speech of the user;

(f) communicating, to the first remote server, the second audio message;

(g) receiving, at the client device, a result, transcribed at the first remote server using an automatic speech recognition system (ASR), from the second audio message;

(h) receiving, at the client device from the user, an affirmation of the second result;

(i) storing, at the client device, the second result in association with an identifier corresponding to the audio message; and

(j) communicating, to a second remote server, the stored result together with the identifier.