Phonetically based speech recognition system and method

• US 7,146,319 B2
• Filed: 03/31/2003
• Issued: 12/05/2006
• Est. Priority Date: 03/31/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Novauris Technologies Ltd. (Apple Inc.)
• Inventors: Hunt, Melvyn J.
• Primary Examiner(s): CHAWAN, VIJAY B
• Application Number:
  US10/401,572
• Publication Number:
  US 20040193408A1
• Time in Patent Office: 1,345 Days
• Field of Search: 704/254, 704/243, 704/256, 704/255, 704/249, 704/1, 704/241, 704/270.1, 704/9, 704/2, 704/231, 704/240, 704/245, 704/257, 704/239, 704/268
• US Class Current: 704/254
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A speech recognition method includes a step of receiving a phonetic sequence output by a phonetic recognizer. The method also includes a step of matching the phonetic sequence with one of a plurality of reference phoneme sequences stored in a reference list that matches closest thereto. At least one of the plurality of reference phoneme sequences stored in the reference list includes additional information with respect to a phonetic sequence that is capable of being output by the phonetic recognizer.

Claims

1. A method for performing speech recognition, the method comprising:

receiving at least a phonetic sequence from a phonetic recognizer, wherein the phonetic sequence comprises at least a portion of an output of the phonetic recognizer, the output corresponding to a spoken input received by the phonetic recognizer;

selecting one of a plurality of reference phoneme sequences stored in a reference list based on a comparison of the phonetic sequence with one or more of the plurality of reference phoneme sequences, wherein selecting one of the plurality of reference phoneme sequences includes using additional information provided independently of the phonetic recognizer and with at least one of the plurality of reference phoneme sequences, wherein the additional information is indicative of whether a given one or more of the plurality of reference phoneme sequences corresponds to at least a portion of the phonetic sequence.

2. The method of claim 1, wherein selecting one of the plurality of reference phoneme sequences includes using additional information that is indicative of phoneme stress level.

3. The method of claim 2, wherein the additional information is stored as a particular ASCII character.

4. The method of claim 2, wherein using the additional information includes using:

a first value that is indicative of an unstressed level, a second value that is indicative of a first stress level that is greater than the unstressed level, and a third value that is indicative of a second stress level that is greater than the first stress level.

5. The method of claim 2, wherein using the additional information includes using additional information that is indicative of the phoneme stress level for at least one vowel in each of the reference phoneme sequences.

6. The method of claim 1, wherein the step of selecting one of the plurality of reference phoneme sequences includes using additional information provided with a match cost matrix that includes information indicative of phoneme stress levels of the plurality of reference phoneme sequences.

7. The method of claim 1, wherein selecting one of the plurality of reference phoneme sequences includes using additional information that is indicative of the phoneme stress level to enable selection of the one of the plurality of reference phoneme sequences when the output of the phonetic recognizer does not distinguish between different phonetic stress levels occurring in a corresponding spoken input.

8. The method of claim 1, wherein selecting one of the plurality of reference phoneme sequences includes using additional information that identifies one or both of prevocalic and postvocalic consonants from a given reference phoneme sequence in the plurality of reference phoneme sequences.

9. The method of claim 8, wherein selecting one of a plurality of reference phoneme sequences includes:

using the additional information from a match cost matrix that includes information that is indicative of occurrences of postvocalic consonants and/or prevocalic consonants in one or more of the plurality of reference phoneme sequences.

10. The method of claim 9, wherein selecting one of the plurality of reference phoneme sequences includes using additional information that is indicative of one or both of prevocalic and postvocalic consonants when the output of the phonetic recognizer does not distinguish between postvocalic consonants and prevocalic consonants occurring in a corresponding spoken input.

11. The method of claim 1, wherein using additional information includes using, in one or more of the reference phoneme sequences, a multi-phoneme representation that provides alternative acceptable pronunciations corresponding to two or more alternative phonemes.

12. The method of claim 11, wherein the multi-phoneme representation can be represented by a particular ASCII character sequence in the reference list.

13. The method of claim 12, wherein the multi-phoneme representations are included in a match cost matrix, and wherein a cost corresponding to a particular multi-phoneme representation corresponds to a cost of the two or more alternative phonemes and of the phonetic sequence being considered for the comparison.

14. The method of claim 1, wherein selecting one of the plurality of reference phoneme sequences includes using, for a given phoneme in a reference phoneme sequence in the plurality of reference phoneme sequences, information about one or more phonemes that are adjacent to the given phoneme sequence in the plurality of reference phoneme sequences.

15. The method of claim 1, wherein selecting one of a plurality of reference phoneme sequences stored in a reference list includes using additional information about whether a syllable boundary affects a given phoneme that precedes or follows a given phoneme in a reference phoneme sequence in the plurality of reference phone sequences.

16. The method of claim 15, wherein the additional information identifies the syllable boundary that most closely precedes or follows the given phoneme.

17. The method of claim 1, wherein selecting one of a plurality of reference phoneme sequences stored in a reference list includes using a probability value that a particular reference phoneme sequence corresponds to a given phonetic sequence that is outputted from the phonetic recognizer.

18. The method of claim 1, wherein using additional information includes using, in one or more of the reference phoneme sequences, a multi-phoneme representation that includes a deletable phoneme, wherein the multi-phoneme representation provides alternative acceptable pronunciations corresponding to use and non-use of the deletable phoneme.

19. A speech recognition apparatus, comprising:

an input unit configured to receive an output of a phonetic recognizer, wherein the output includes at least a phonetic sequence that corresponds to a speech-utterance received by the phonetic recognizer;

a reference list configured to store a plurality of reference phoneme sequences;

a matching unit configured to select one of the plurality of reference phoneme sequences from the reference list based on a comparison of the phonetic sequence with one or more phoneme sequences in the reference list, wherein the matching unit is configured to select one of the plurality of reference phoneme sequences using additional information that is (i) provided with at least one of the plurality of reference phoneme sequences independently of the output of the phonetic recognizer, and (ii) is indicative of whether a given one or more of the plurality of reference phoneme sequences corresponds to at least a portion of the phonetic sequence.

20. The speech recognition apparatus of claim 19,

wherein the matching unit is configured to use the additional information—

that includes information which is indicative of phoneme stress levels of one or more of the reference phoneme sequences in the reference list.

21. The speech recognition apparatus of claim 19, wherein the matching unit is configured to select one of the reference phoneme sequences using additional information that is indicative of the phoneme stress level when the phonetic recognizer does not distinguish between different phonetic stress levels occurring in a corresponding acoustic sequence input.

22. The speech recognition apparatus of claim 19, wherein the matching unit is configured to select one of the plurality of reference phoneme sequences using additional information that is indicative of prevocalic and postvocalic consonants.

23. The speech recognition apparatus of claim 19, wherein the speech recognition apparatus corresponds to a lexical interpreter that provides output by a display monitor unit of a speech recognition system.

24. The speech recognition apparatus of claim 19, further comprising:

a match cost matrix that stores at least some of the additional information, including information corresponding to occurrences of postvocalic consonants and prevocalic consonants for one or more of the plurality of reference phoneme sequences in the reference list.

25. The speech recognition apparatus of claim 19, wherein the matching unit is configured to use additional information that is indicative of postvocalic consonants and prevocalic consonants when the phonetic recognizer does not distinguish between postvocalic and prevocalic consonants occurring in a corresponding spoken input.

26. The speech recognition apparatus of claim 19, wherein, for one or more of the reference phoneme sequences, the additional information corresponds to a multi-phoneme representation that provides alternative acceptable pronunciations corresponding to two or more alternative phonemes.

27. The speech recognition apparatus of claim 26, wherein the multi-phoneme representation can be represented by a particular ASCII character in the reference list.

28. The apparatus of claim 19, wherein the matching unit is further configured to use, for a given phoneme in a reference phoneme sequence in the plurality of reference phoneme sequences, information about one or more phonemes that are adjacent to the given phoneme.

29. The speech recognition apparatus of claim 19, wherein, for one or more of the reference phoneme sequences, the additional information corresponds to a multi-phoneme representation that includes a deletable phoneme, wherein the multi-phoneme representation provides alternative acceptable pronunciations corresponding to use and non-use of the deletable phoneme.

30. A program product having machine-readable program code for performing speech recognition, the program code, when executed, causing a machine to perform steps that include:

receiving a phonetic sequence, wherein the phonetic sequence comprises at least a portion of an output of a phonetic recognizer, the output corresponding to a speech utterance received by the phonetic recognizer;

selecting one of a plurality of reference phoneme sequences stored in a reference list based on a comparison of the phonetic sequence with one or more phoneme sequences in the plurality of reference phoneme sequences, wherein selecting one of the plurality of reference phoneme sequences includes using additional information, the additional information being provided independently of the output of the phonetic recognizer and with at least one of the plurality of reference phoneme sequences, wherein the additional information is indicative of whether a given one or more of the plurality of reference phoneme sequences corresponds to at least a portion of the phonetic sequence.

31. The program product of claim 30, wherein the additional information is indicative of phoneme stress level and is stored in the reference list.

32. The program product of claim 31, wherein the additional information that is indicative of the phoneme stress level is stored as a particular ASCII character.

33. The program product of claim 31, wherein the additional information comprises:

a first value that is indicative of an unstressed level, a second value that is indicative of a first stress level that is greater than the unstressed level, and a third value that is indicative of a second stress level that is greater than the first stress level.

34. The program product of claim 33, wherein the additional information that is indicative of the phoneme stress level is stored in the reference list for at least one vowel in each of the reference phoneme sequences.

35. The program product of claim 33, wherein the additional information is indicative of one or both of prevocalic and postvocalic consonants.

36. The program product of claim 30, wherein the program code, when executed, causes the machine to use, for a selected phoneme in a reference phoneme sequence, information about one or more phonemes that are adjacent to the selected phoneme in a listing of the plurality of reference phoneme sequences.

37. The program product of claim 31, wherein the additional information includes, for a given phoneme in a reference phoneme sequence in the plurality of reference phoneme sequences, information about one or more phonemes that are adjacent to the given phoneme.