Method and apparatus for generating and displaying N-Best alternatives in a speech recognition system

• US 7,162,423 B2
• Filed: 11/23/2004
• Issued: 01/09/2007
• Est. Priority Date: 07/20/2000
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Alleva, Fileno A., Thrasher, Chris
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US10/996,317
• Publication Number:
  US 20050091054A1
• Time in Patent Office: 777 Days
• Field of Search: None
• US Class Current: 704/251
• CPC Class Codes: G06F 9/4488 Object-oriented
• CPC Class Codes: G10L 15/083 Recognition networks G10L15/142, G10L15/16 take precedence
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems
• CPC Class Codes: H04L 65/1101 Session protocols

Abstract

The present invention is directed to a generating alternatives to words indicative of recognized speech. A reference path of recognized words is generated, based upon input speech data. An operator selection input is received and is indicative of a selected portion of the recognized speech, for which alternatives are to be generated. Boundary conditions for alternatives to be generated are calculated based upon bounds of a reference subpath corresponding to the selected portion of the recognized speech. Alternate subpaths satisfying the boundary conditions are constructed from a hypothesis store which corresponds to the input speech data.

Claims

1. A computer readable medium storing instructions which, when executed by a computer, cause the computer to perform a method of generating alternatives to words indicative of recognized speech, recognized based on input speech data, comprising:

generating a reference path of recognized words based on the input speech data;

receiving an operator selection input indicative of a selected portion of the recognized speech for which alternatives are to be generated;

calculating boundary conditions for alternatives based on position bounds of a reference subpath corresponding to the selected portion of the recognized speech; and

constructing from a hypothesis store, corresponding to the input speech data, alternate subpaths satisfying the boundary conditions.

2. The computer readable medium of claim 1 wherein calculating the boundary conditions comprises:

determining the position bounds of the reference subpath by identifying, as a beginning boundary condition, a beginning time in the reference path of a first boundary word, the first boundary word preceding the selected portion of the recognized speech.

3. The computer readable medium of claim 2 wherein calculating the boundary conditions comprises:

determining the temporal bounds of the reference subpath by identifying, as an ending boundary condition, an ending time in the reference path of a second boundary word, the second boundary word following the selected portion of the recognized speech.

4. The computer readable medium of claim 3 wherein the hypothesis store includes a word hypothesis lattice indicative of entries for a plurality of alternate words corresponding to utterances in the input speech data, each entry including a lexical word, a beginning time in the input speech data corresponding to the lexical word and an ending time in the speech data corresponding to the lexical word, and wherein constructing alternate subpaths comprises:

obtaining from the hypothesis lattice instances of the first boundary word that satisfy the beginning boundary condition.

5. The computer readable medium of claim 4 wherein constructing alternate subpaths comprises:

concatenating to each of the instances of the first boundary word a string of one or more additional words, wherein the string of one or more additional words satisfies the ending boundary condition.

6. The computer readable medium of claim 5 wherein constructing alternate subpaths comprises:

concatenating to each of the instances of the first boundary word a string of one or more additional words, wherein the string of one or more additional words ends in an instance of the second boundary word that satisfies the ending boundary condition.

7. The computer readable medium of claim 5 wherein concatenating comprises:

for each instance of the first boundary word, obtaining from the hypothesis lattice one or more instances of a first intermediate word having a beginning time corresponding to an ending time of the first boundary word and having an ending time.

8. The computer readable medium of claim 7 wherein concatenating further comprises:

continuously concatenating additional words until an alternate path is formed satisfying the boundary conditions, each of the additional words having a beginning time corresponding to an ending time of a preceding word and an ending time corresponding to a beginning time of a subsequent word.

9. The computer readable medium of claim 1 wherein constructing alternate subpaths comprises:

beginning construction of up to a predetermined number, X, of hypothesis alternate subpaths; and

assigning each hypothesis alternate subpath a score that is updated as the hypothesis alternate subpath is constructed.

10. The computer readable medium of claim 9 wherein assigning a score comprises:

computing an actual score for the alternate subpath through a temporal point to which the hypothesis alternate subpath is constructed; and

computing an estimated score, estimating a score associated with a remainder of the hypothesis alternate subpath to be computed.

11. The computer readable medium of claim 10 wherein computing an actual score comprises:

computing the actual score as a combination of acoustic scores for words in the hypothesis alternate subpath and language model scores for the words in the hypothesis alternate subpath.

12. The computer readable medium of claim 11 wherein computing an estimated score comprises:

computing the estimated score as a portion of a reference score assigned to the reference path, the portion of the reference score corresponding to an amount of the hypothesis alternate subpath left to be computed in relation to an overall length of the hypothesis alternate subpath.

13. The computer readable medium of claim 9 wherein constructing alternate subpaths comprises:

after beginning construction of each of the X hypothesis alternate subpaths, beginning construction of a hypothesis alternate subpath only if its score is one of the top X scores; and

discontinuing construction of all hypothesis alternate subpaths having a score not in the top X scores.

14. The computer readable medium of claim 13 wherein constructing alternate subpaths comprises:

when construction of X hypothesis alternate subpaths has begun, sorting the hypothesis alternate subpaths by score.

15. The computer readable medium of claim 14 wherein constructing alternate subpaths comprises:

retaining hypothesis alternate subpaths having the top X scores as the alternate subpaths.

16. The computer readable medium of claim 15 wherein the method further comprises:

presenting the alternate subpaths to the user.

17. The computer readable medium of claim 1 wherein constructing alternate subpaths comprises:

constructing alternate subpaths that are lexically different from the reference subpath.

18. The computer readable medium of claim 17 wherein, when fewer than a predetermined number of lexically different alternate subpaths can be constructed, extending the boundary conditions and constructing alternate subpaths satisfying the extended boundary conditions.

19. The computer readable medium of claim 1 wherein constructing the alternate subpaths begins in response to the operator selection input.

20. A computer readable medium storing instructions which, when executed by a computer, cause the computer to perform a method of generating alternate speech recognitions for speech data input by an application, comprising:

receiving the speech data at a speech recognition engine;

generating a reference speech path and a hypothesis lattice based on the speech data, the hypothesis lattice representing alternate recognition paths corresponding to the speech data, the hypothesis lattice including a plurality of entries, each entry including an instance of a lexical word, a starting point in the speech data of the instance of the lexical word, an ending point in the speech data of the instance of the lexical word, and a score associated with the instance of the lexical word;

storing the hypothesis lattice with the application;

receiving a user selection input indicative of a portion of the reference speech path to be corrected;

in response to the user selection input, accessing the retrieved hypothesis lattice with the speech recognition engine; and

constructing, at the engine, alternate subpaths to replace portions of the reference speech path based on the hypothesis lattice.

21. The computer readable medium of claim 20 wherein the method further comprises:

presenting the alternate subpaths to the user.

22. The computer readable medium of claim 20 wherein constructing alternate subpaths comprises:

calculating boundary conditions for alternatives based on temporal bounds of a reference subpath corresponding to the selected portion of the reference speech path; and

constructing from the hypothesis lattice alternate subpaths satisfying the boundary conditions.

23. The computer readable medium of claim 22 wherein calculating the boundary conditions comprises:

determining the temporal bounds of the reference subpath by identifying, as a beginning boundary condition, a beginning time in the reference speech path of a first boundary word, the first boundary word preceding the selected portion of the reference speech path.

24. The computer readable medium of claim 23 wherein calculating the boundary conditions comprises:

determining the temporal bounds of the reference subpath by identifying, as an ending boundary condition, an ending time in the reference speech path of a second boundary word, the second boundary word following the selected portion of the reference speech path.

25. A computer readable medium storing instructions which, when executed by a computer, cause the computer to perform a method of generating correction alternative to misrecognized words in input speech data, comprising:

generating a reference path of recognized words based on the input speech data;

receiving an operator selection input indicative of a selected misrecongnized portion of the recognized speech for which alternatives are to be generated;

calculating boundary conditions for alternatives based on position bounds of a reference subpath corresponding to the selected misrecognized portion of the recognized speech; and

constructing from a hypothesis store, corresponding to the input speech data, alternate subpaths satisfying the boundary conditions.