Combining N-best lists from multiple speech recognizers

• US 20020193991A1
• Filed: 06/13/2001
• Published: 12/19/2002
• Est. Priority Date: 06/13/2001
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Anderson, Andrew V., Bennett, Steven M.
• Granted Patent: US 6,701,293 B2
• Time in Patent Office: Days
• US Class Current: 704/247
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

A method and system for utilizing multiple speech recognizers. The speech system includes a port through which an input audio stream may be received, at least two recognizers that may convert the input stream to text or commands, and a combiner able to combine lists of possible results from each recognizer into a combined list. The method includes receiving an input audio stream, routing the stream to one or more recognizers, receiving a list of possible results from each of the recognizers, combining the lists into a combined list and returning at least a subset of the list to the application.

Claims

1. A method for utilizing multiple speech recognizers, the method comprising:

a) receiving an input stream;

b) routing the input stream to one or more recognizers;

c) receiving individual result sets from each of the recognizers, wherein at least one individual result set comprises an N-best list of dynamic length;

d) combining the individual results into a combined list; and

e) returning at least a subset of the combined list to the application.

2. The method of claim 1, wherein the subset of the combined list further comprises an N-best list of dynamic length.

3. The method of claim 1, wherein the individual result sets from the recognizers include at least one associated utility metric.

4. The method of claim 3, where the utility metric is comprised of confidence values for the individual results.

5. The method of claim 1, wherein the method further comprises normalizing the individual results prior to combining the individual results into a combined list.

6. The method of claim 1, wherein the individual results comprise N-best lists from the recognizers.

7. The method of claim 5, the method further comprising weighting the individual results from each recognizer with a performance value associated with each recognizer.

8. The method of claim 7, wherein the performance value associated with each recognizer varies based upon at least one the group comprising:

a user identity, a user gender, a user ethnicity, a user native language, contextual information, an active dialog state of the application, a communication channel used, and a communication device used.

9. The method of claim 7, wherein the performance value associated with each recognizer is updated dynamically.

10. The method of claim 1, the method further comprising analyzing redundant results in the combined list for results that are substantially the same and raising a confidence level for those results.

11. The method of claim 1, wherein the method further comprises reducing the size of the combined list.

12. The method of claim 1, wherein the method further comprises tracking and mapping locations of each entry in the combined list from the individual results.

13. The method of claim 1, wherein the method further comprises:

a) receiving a query specific to an entry in the combined list after returning at least a subset of the combined list to the application,

b) determining a location for the entry in the individual results received from the recognizers; and

c) satisfying the query

15. The method of claim 14, wherein the N-best lists have accompanying confidence levels for each entry in each N-best list.

16. The method of claim 15, wherein the method further comprises normalizing the confidence levels from each N-best list.

18. The speech recognition system of claim 17, wherein the port further comprises a telephone receiver.

19. The speech recognition system of claim 17, wherein the port further comprises a microphone connected to a computing device.

20. The speech recognition system of claim 17, wherein the port connects to the recognizers by a network.

21. The speech recognition system of claim 17, wherein the combiner is further operable to act as a proxy for the recognizers.

27. The method of claim 1, wherein at least one individual results sets further comprises an N-best list of dynamic length.

14. A method for combining N-best lists from multiple recognizers, the method comprising:

a) receiving an N-best list from one or more recognizers

b) combining all N-best entries from the N-best lists received from the recognizers into an initial N-best list;

c) reducing the N-best list size, resulting in an intermediate N-best;

d) sorting the intermediate N-best list according to at least one sorting criteria, resulting in a sorted N-best list; and

e) selecting a subset of entries from the sorted N-best list.

17. A speech recognition system, comprising:

a) a port operable to receive an input audio stream;

b) at least two recognizers operable to convert the input stream into a converted stream and to provide results;

c) a combiner operable to combine the results from each of the recognizers into a combined result and to send at least a subset of the combined result to the application, wherein the combiner processes an N-best list.

22. An article containing machine-readable code that, when executed, causes the machine to:

a) receive an input stream;

b) route the input stream to one or more recognizers;

c) receive individual result sets from each of the recognizers, wherein at least one individual result set is further comprises an N-best list of dynamic length;

d) combine the individual result sets into a combined list; and

e) return at least a subset of the combined list to the application.

23. The article of claim 22, wherein the code, when executed, further causes the machine to normalize the individual result sets received from the recognizers prior to combining the individual result sets into a combined list.

24. The article of claim 22, wherein the code, when executed, further causes the machine to:

a) combine all individual result sets from the recognizers into an initial N-best list;

b) reduce the initial N-best list size, resulting in an intermediate N-best list;

c) sort the intermediate N-best list according to at least one sorting criteria, resulting in a sorted N-best list; and

d) select a predetermined number of entries from the sorted N-best list.

25. The article of claim 24, the code, when executed, further causing the machine to weight the N-best entries from each recognizer with a performance value associated with each recognizer.

29. The article of claim 28, wherein at least one individual result sets further comprises an N-best list of dynamic length.

26. A method for utilizing multiple speech recognizers, the method comprising:

a) receiving an input stream;

b) routing the input stream to one or more recognizers;

c) receiving individual result sets from each of the recognizers;

d) combining the individual results into a combined list; and

e) returning at least a subset of the combined list to the application, wherein the subset further comprises an N-best list of dynamic length.

28. An article containing machine-readable code that, when executed, causes the machine to:

a) receive an input stream;

b) route the input stream to one or more recognizers;

c) receive individual result sets from each of the recognizers;

d) combine the individual result sets into a combined list; and

e) return at least a subset of the combined list to the application, wherein the subset further comprises an N-best list of dynamic length.