Method of producing alternate utterance hypotheses using auxiliary information on close competitors

• US 7,676,367 B2
• Filed: 02/20/2004
• Issued: 03/09/2010
• Est. Priority Date: 02/21/2003
• Status: Active Grant

• Current Assignee: Cerence Inc., Cerence Operating Company (Cerence Inc.)
• Original Assignee: Voice Signal Technologies Incorporated (Microsoft Corporation)
• Inventors: Roth, Robert, Khasin, Arkady, Gillick, Laurence S.
• Primary Examiner(s): Opsasnick; Michael N
• Assistant Examiner(s): GODBOLD, DOUGLAS
• Application Number:
  US10/783,518
• Publication Number:
  US 20050108012A1
• Time in Patent Office: 2,209 Days
• Field of Search: 704/231, 704/251, 704/252, 704/254, 704/257
• US Class Current: 704/252
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/10 using distance or distortion measures between unknown speech and reference templates
• CPC Class Codes: G10L 2015/085 Methods for reducing search complexity, pruning

Abstract

A method of constructing a list of alternate transcripts from a recognized transcript includes generating a list of close call records, matching partial sub-histories from the recognized transcript with one of the history pairs stored in each of the records, and substituting the other of the history pairs for the partial sub-history of the recognized transcript. A close call record is generated each time a pair of partial hypotheses attempt to seed a common word. Each close call record includes history information and scoring information associated with a particular pair of partial hypotheses seeding a common word. Alternate transcripts are constructed by substituting close call histories for partial histories of the recognized transcripts, and also by substituting close call histories for partial histories of other alternate transcript.

Claims

1. A method of constructing a list of alternate versions of a recognized transcript, said method comprising:

during speech recognition, generating a list of close call records, wherein each record includes histories for each of two competing partial hypotheses that seed a common word;

adding the recognized transcript to a choice list;

by using a processor selecting a record from the close call list;

for each entry on the choice list,

(a) determining whether one of the two histories for the selected record matches a partial subhistory of that entry on the choice list;

(b) if one of the two histories for the selected record matches a partial subhisory of that entry, substituting the other of the two histories for the partial subhistory of that entry to generate an alternative version of the transcript; and

,

(c) adding the alternative version of the transcript to the choice list.

2. The method of claim 1, further comprising:

selecting another record from the close call list; and

,

for each entry on the choice list, repeating steps (a) (b) and (c).

3. A method of constructing a list of alternate transcript from a recognized utterance, said method comprising

generating a list of close call records, wherein each record includes history information and scoring information associated with a particular pair of partial hypotheses seeding a common word;

generating one or more alternate transcripts from the list of close call records by evaluating each record in the list for a match between a partial sub-history of the recognized utterance and one of the histories stored in the record, and upon finding such a match, substituting the other of the histories stored in the record for the partial sub-history in the recognized transcript;

by using a processor generating additional alternate transcripts by evaluating each record in the list of close call records for a match between a partial sub-history of each alternate utterance in the choice list and one of the histories stored in the record, and upon finding a match, substituting the other of the histories stored in the record for the partial sub-history in the alternate transcript; and

,

storing the one or more alternate transcripts and the additional alternate transcripts in a choice list.

4. A method according to claim 3, further including limiting the list of close call records to a preset maximum number of records.

5. A method of constructing a choice list of alternate versions of a recognized transcript from a speech recognition system, said method comprising:

during speech recognition, generating a list of close call records, wherein each record includes histories for each of two competing partial hypotheses;

initializing the choice list from at least one output of the speech recognition system;

by using a processor selecting one of the close call records from the list of close call records;

selecting a transcript from the choice list;

determining whether one of the two histories for the selected record matches a partial subhistory of the transcript from the choice list;

if one of the two histories for the selected close call record matches a partial subhistory of the transcript, substituting the other of the two histories for the partial subhistory of the transcript to generate an alternative version of the transcript; and

,

adding the alternative version of the transcript to the choice list.

6. The method of claim 5, wherein initializing the choice list involves initializing the choice list with the recognized transcript.

7. The method of claim 5, wherein initializing the choice list involves initializing the choice list with all active, legal word ending hypotheses.

8. The method of claim 5, further including comparing the close call record selected from the close call list against each transcript in the choice list.

9. The method of claim 5, further including limiting the list of close call records to a preset maximum number of close call records.

10. The method of claim 9, further including with each of the close call records, a close call score difference between the competitor hypothesis and the score of the globally best hypothesis at the time the close call record is added, the close call score difference being used to determine which close calls to keep if the preset number of close call records is reached.

11. The method of claim 5, further including generating a list of close call records, wherein each record includes histories for each of two competing word-ending partial hypotheses.

12. The method of claim 11, further including generating a list of close call records, wherein each record includes histories for each of two competing word-ending partial hypotheses, both seeding a common word.

13. The method of claim 5, further including generating a list of close call records wherein each of the close call records includes a close call score difference between the competing hypotheses, the score difference being used to construct the choice list.

14. The method of claim 13, further including computing a transcript score for each transcript in the choice list by adding the close call score difference to the transcript score of the transcript from the choice list into which the other of the two histories was substituted.

15. The method of claim 14, further including adding a difference between a language model score of the new transcript and the language model score of the transcript from the choice list into which the other of the two histories was substituted.

16. The method of claim 5, further including with each of the close call records, a first score and a second score, the first score being a close call score difference between the competing hypotheses, the second score being a global score difference between the competitor hypothesis and the score of a globally best hypothesis at the time the record is added, wherein the close call difference is used to construct the choice list, and the global score difference is used to determine which close calls to keep if the preset number of close call records is reached.

17. A method of constructing a list of alternate transcript from a recognized utterance, said method comprising:

generating a list of close call records, wherein each record includes history information and scoring information associated with a particular pair of partial hypotheses seeding a common word;

by using a processor generating one or more alternate transcripts from the list of close call records by evaluating each record in the list for a match between a partial sub-history of the recognized utterance and one of the histories stored in the record, and upon finding such a match, substituting the other of the histories stored in the record for the partial sub-history in the recognized transcript;

storing the one or more alternate transcripts in a choice list; and

storing in the close call list for each pair of partial hypotheses seeding a common word (i) a history of a first partial hypothesis, (ii) a history of a second partial hypothesis, (iii) a score difference being a difference between a score of the first partial hypothesis and a score of the second partial hypothesis, and (iv) a global score difference being a difference between the score of a current best overall scoring hypothesis and the score of the second partial hypothesis.

18. A computer readable medium storing instructions adapted for execution on a processor, said instructions comprising:

instructions for generating, during speech recognition, a close call list of close call records, wherein each record includes histories for each of two competing partial hypotheses;

instructions for initializing the choice list from at least one output of the speech recognition system;

instructions for selecting one of the close call records from the close call list;

instructions for selecting a transcript from the choice list;

instructions for determining whether one of the two histories for the selected record matches a partial subhistory of the transcript from the choice list;

instructions for substituting the other of the two histories for the partial subhistory of the transcript to generate an alternative version of the transcript, if one of the two histories for the selected close call record matches a partial subhistory of the transcript; and

,

instructions for adding the alternative version of the transcript to the choice list.

19. A method of constructing a list of alternate transcripts from a recognized transcript, comprising:

providing two or more partial hypotheses of an acoustic transcript;

for each pair of partial hypotheses characterized by a first partial hypothesis having an associated first score and a second partial hypothesis having an associated second score being less than the first score, both ending at a common time and both seeding a common continuation word, evaluating by using a processor the first partial hypothesis and the second partial hypothesis at each acoustic frame following the seeding of the common continuation word, and storing in a close call list a record of the first and second partial hypotheses, the record corresponding to the acoustic frame resulting in a smallest score difference between a current best overall scoring hypothesis and the second score, wherein the record includes at least (i) a history of the first partial hypothesis, (ii) a history of the second partial hypothesis, (iii) a score difference being a difference between the first score and the second score, and (iv) a global score difference being a difference between the current best overall scoring hypothesis and the second score; and

,

generating one or more alternate hypotheses by combining information from at least one record in the close call list with the recognized transcript.

20. A method of constructing a list of alternate utterance hypotheses from a complete utterance hypothesis, comprising:

providing two or more partial hypotheses of an acoustic utterance;

for each pair of partial hypotheses characterized by a first partial hypothesis having an associated first score and a second partial hypothesis having an associated second score being less than the first score, both ending at a common time and both seeding a common continuation word, evaluating by using a processor the first partial hypothesis and the second partial hypothesis at each acoustic frame following the seeding of the common continuation word, and storing in a close call list a record of the first and second partial hypotheses, the record corresponding to the acoustic frame resulting in a smallest score difference between a current best overall scoring hypothesis and the second score, wherein the record includes at least (i) a history of the first partial hypothesis, (ii) a history of the second partial hypothesis, (iii) a score difference being a difference between the first score and the second score, and (iv) a global score difference being a difference between the current best overall scoring hypothesis and the second score;

for each acoustic frame, updating the two or more partial hypotheses until the acoustic utterance ends, and selecting a best scoring complete hypothesis;

evaluating the records in the close call list for potential alternate utterance hypotheses, beginning with a record in the close call list having a smallest score difference and subsequently with each record in the close call list in an order of ascending score difference, by;

(i) comparing a set of first words from the first hypothesis and a set of first words from one or more complete hypotheses from a choice list;

(ii) if a set of first words from a history of the first partial hypothesis matches a set of first words from one or more complete hypotheses from the choice list, substituting the history of the second partial hypothesis for the history of the first partial hypothesis within the one or more complete hypotheses from the choice list so as to generate one or more alternate utterance hypotheses, and placing the alternate hypotheses in the choice list; and

,

(iii) continuing evaluating the records in the close call list until filling the choice list.

21. A method according to claim 20, further including providing a close call list having a capacity for a preset maximum number of records.

22. A method according to claim 20, further including providing a choice list having a capacity for a finite number of alternate transcripts.

23. A method according to claim 20, further including comparing the set of first words of the history of the first partial hypothesis to the set of words from the complete utterance hypothesis, wherein the set of words from the complete utterance hypothesis includes all of the words from the first partial hypothesis.

24. A method according to claim 20, further including comparing the global score of a highest ranking record remaining in the close call list with the lowest scoring hypothesis in the choice list, and generating at least one further alternate utterance hypothesis if the global score of the highest ranking record remaining in the close call list is better than the score difference between the best and worst scoring hypotheses in the choice list.

25. A method of creating an alternate utterance hypothesis from a complete utterance hypothesis, comprising:

for a first partial hypothesis having an associated first score and a second partial hypothesis having an associated second score being less than the first score, both ending at a common time and both seeding a common continuation word, storing information characterizing the first partial hypothesis and the second partial hypothesis at each frame following the seeding of the common continuation word, the information including at least a history of the first partial hypothesis and a history of the second partial hypothesis;

by using a processor comparing a set of first words from the first hypothesis and a set of first words from the complete utterance hypothesis;

if a set of first words from the history of the first partial hypothesis matches a set of first words from the complete utterance hypothesis, substituting the history of the second partial hypothesis for the history of the first partial hypothesis within the complete utterance hypothesis; and

storing information characterizing the first and second partial hypotheses that include (i) a history of the first partial hypothesis, (ii) a history of the second partial hypothesis, (iii) a score difference being a difference between the first score and the second score, and (iv) a global score difference being a difference between the score of a globally best hypothesis and the second score.

26. A method according to claim 25, further including generating the first score and the second score based at least upon input acoustic data and a set of language models.

27. A method according to claim 25, further including comparing the set of first words of the history of the first partial hypothesis to the set of words from the complete utterance hypothesis, wherein the set of words from the complete utterance hypothesis includes all of the words from the first partial hypothesis.

28. A computer readable medium storing instructions adapted for execution on a processor, said instructions comprising:

instructions for generating a list of close call records, wherein each record includes history information and scoring information associated with a particular pair of partial hypotheses seeding a common word;

instructions for generating one or more alternate transcripts from the list of close call records by evaluating each record in the list for a match between a partial sub-history of the recognized transcript and one of the histories stored in the record, and upon finding such a match, substituting the other of the histories stored in the record for the partial sub-history in the recognized transcript;

instructions for generating additional alternate transcripts by evaluating each record in the list of close call records for a match between a partial sub-history of each alternate utterance in the choice list and one of the histories stored in the record, and upon finding a match, substituting the other of the histories stored in the record for the partial sub-history in the alternate transcript; and

instructions for storing the one or more alternate transcripts and the additional alternate transcripts in a choice list.

29. The computer readable medium of claim 28, wherein the medium is disposed within a mobile telephone apparatus and operates in conjunction with a user interface.

30. The computer readable medium of claim 28, wherein the medium is disposed within a handheld electronic apparatus and operates in conjunction with a user interface.