CONFIGURABLE SPEECH RECOGNITION SYSTEM USING MULTIPLE RECOGNIZERS

• US 20130132089A1
• Filed: 01/22/2013
• Published: 05/23/2013
• Est. Priority Date: 01/07/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Newman, Michael J., Gillet, Anthony, Krowitz, David Mark, Edgington, Michael D., Fanty, Mark, Lynch, Timothy
• Granted Patent: US 9,183,843 B2
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

Techniques for combining the results of multiple recognizers in a distributed speech recognition architecture. Speech data input to a client device is encoded and processed both locally and remotely by different recognizers configured to be proficient at different speech recognition tasks. The client/server architecture is configurable to enable network providers to specify a policy directed to a trade-off between reducing recognition latency perceived by a user and usage of network resources. The results of the local and remote speech recognition engines are combined based, at least in part, on logic stored by one or more components of the client/server architecture. An indication of the availability of the remote speech recognition to perform speech recognition at a point in time may be provided to a user of the client device via a user interface of the client device.

Claims

1. A client device for use in a client/server speech recognition system comprising the client device and a server device, wherein the server device includes a remote speech recognizer remote from the client device, the client device comprising:

a user interface, the user interface comprising an input interface configured to receive input audio comprising speech; and

at least one processor programmed to;

determine an availability status, at a point in time, of the remote speech recognizer for performing speech recognition on input audio received via the input interface; and

provide via the user interface, an indication of the availability status.

2. The client device of claim 1, wherein the user interface is configured to support interacting with a user using any of a plurality of different interaction techniques;

and

wherein providing the indication of the availability status comprises;

selecting one of the plurality of different interaction techniques based, at least in part, on the availability status; and

providing the indication of the availability status using the selected one of the plurality of different interaction techniques.

3. The client device of claim 2, wherein the at least one processor is further programmed to implement a plurality of digital agent personas;

wherein selecting one of the plurality of different interaction techniques comprises selecting one of the plurality of digital agent personas; and

wherein providing the indication comprises providing the indication using the selected one of the digital agent personas.

4. The client device of claim 2, wherein the providing the indication comprises providing the indication using audio output and/or visual output.

5. The client device of claim 4, wherein the plurality of different interaction techniques includes a first audio output and a second audio output;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first audio output or the second audio output; and

wherein providing the indication comprises providing the indication by outputting the selected first audio output or second audio output.

6. The client device of claim 4, wherein the plurality of different interaction techniques includes a first visual appearance of the user interface and a second visual appearance of the user interface;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first visual appearance or the second visual appearance; and

wherein providing the indication comprises providing the indication by displaying the selected first visual appearance or second visual appearance.

7. The client device of claim 6, wherein the first visual appearance of the user interface includes a first icon displayed on the user interface to indicate the availability status.

8. The client device of claim 7, wherein the second visual appearance of the user interface includes a second icon displayed on the user interface to indicate the availability status, wherein the second icon is different than the first icon.

9. A method of providing an indication of availability status of a server device in a client/server speech recognition system comprising a client device and the server device, wherein the server device includes a remote speech recognizer remote from the client device, the method comprising:

providing a user interface, the user interface comprising an input interface configured to receive input audio comprising speech;

determining an availability status of the remote speech recognizer for performing speech recognition on input audio received via the input interface; and

providing via the user interface, an indication of the availability status.

10. The method of claim 9, wherein the user interface is configured to support interacting with a user using any of a plurality of different interaction techniques;

and

wherein providing the indication of the availability status comprises;

selecting one of the plurality of different interaction techniques based, at least in part, on the availability status; and

providing the indication of the availability status using the selected one of the plurality of different interaction techniques.

11. The method of claim 10, wherein the client device is configured to implement a plurality of digital agent personas;

wherein selecting one of the plurality of different interaction techniques comprises selecting one of the plurality of digital agent personas; and

wherein providing the indication comprises providing the indication using the selected one of the digital agent personas.

12. The method of claim 11, wherein the plurality of different interaction techniques includes a first audio output and a second audio output;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first audio output or the second audio output; and

wherein providing the indication comprises providing the indication by outputting the selected first audio output or second audio output.

13. The method of claim 11, wherein the plurality of different interaction techniques includes a first visual appearance of the user interface and a second visual appearance of the user interface;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first visual appearance or the second visual appearance; and

wherein providing the indication comprises providing the indication by displaying the selected first visual appearance or second visual appearance.

14. The method of claim 13, wherein the first visual appearance of the user interface includes a first icon displayed on the user interface to indicate the availability status;

and

wherein the second visual appearance of the user interface includes a second icon displayed on the user interface to indicate the availability status, wherein the second icon is different than the first icon.

15. A non-transitory computer-readable storage medium encoded with a plurality of instructions that, when executed by at least one processor on a client device in a client/server speech recognition system comprising the client device and a server device, wherein the server device has a remote speech recognizer remote from the client device, perform a method comprising:

providing a user interface, the user interface comprising an input interface configured to receive input audio comprising speech;

determining an availability status of the remote speech recognizer for performing speech recognition on input audio received via the input interface; and

providing via the user interface, an indication of the availability status.

16. The computer-readable storage medium of claim 15, wherein the user interface is configured to support interacting with a user using any of a plurality of different interaction techniques;

and

wherein providing the indication of the availability status comprises;

selecting one of the plurality of different interaction techniques based, at least in part, on the availability status; and

providing the indication of the availability status using the selected one of the plurality of different interaction techniques.

17. The computer-readable storage medium of claim 16, wherein the client device is configured to implement a plurality of digital agent personas;

wherein selecting one of the plurality of different interaction techniques comprises selecting one of the plurality of digital agent personas; and

wherein providing the indication comprises providing the indication using the selected one of the digital agent personas.

18. The computer-readable storage medium of claim 17, wherein the plurality of different interaction techniques includes a first audio output and a second audio output;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first audio output or the second audio output; and

wherein providing the indication comprises providing the indication by outputting the selected first audio output or second audio output.

19. The computer-readable storage medium of claim 17, wherein the plurality of different interaction techniques includes a first visual appearance of the user interface and a second visual appearance of the user interface;

wherein selecting one of the plurality of different interaction techniques comprises selecting the first visual appearance or the second visual appearance; and

wherein providing the indication comprises providing the indication by displaying the selected first visual appearance or second visual appearance.

20. The computer-readable storage medium of claim 29, wherein the first visual appearance of the user interface includes a first icon displayed on the user interface to indicate the availability status;

and

wherein the second visual appearance of the user interface includes a second icon displayed on the user interface to indicate the availability status, wherein the second icon is different than the first icon.