System and method for speech recognition using an enhanced phone set

• US 7,139,708 B1
• Filed: 08/04/1999
• Issued: 11/21/2006
• Est. Priority Date: 03/24/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Olorenshaw, Lex S., Amador-Hernandez, Mariscela
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US09/369,031
• Time in Patent Office: 2,666 Days
• Field of Search: 704251-257, 704/235
• US Class Current: 704/254
• CPC Class Codes: G10L 15/065 Adaptation
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units

Abstract

A system and method for speech recognition using an enhanced phone set comprises speech data, an enhanced phone set, and a transcription generated by a transcription process. The transcription process selects appropriate phones from the enhanced phone set to represent acoustic-phonetic content of the speech data. The enhanced phone set includes base-phones and composite-phones. A phone dataset includes the speech data and the transcription. The present invention also comprises a transformer that applies transformation rules to the phone dataset to produce a transformed phone dataset. The transformed phone dataset may be utilized in training a speech recognizer, such as a Hidden Markov Model. Various types of transformation rules may be applied to the phone dataset of the present invention to find an optimum transformed phone dataset for training a particular speech recognizer.

Claims

1. A system for speech processing, comprising:

speech data generated from one or more speech sources;

an enhanced phone set that includes acoustic-phonetic symbols and connectors for extending said enhanced phone set, said enhanced phone set including a TIMIT base-phone set and an extended base-phone set, said extended base-phone set including a base phone for representing an articulator noise;

a transcription generated by a transcription process that selects appropriate phones from said enhanced phone set to represent said speech data; and

transformation rules applied to said enhanced phone set to produce a transformed phone dataset, said transformed phone dataset being used in building a phonetic dictionary.

2. The system of claim 1, further comprising a phone dataset that includes said speech data and said transcription.

3. The system of claim 2, wherein said phone dataset is utilized in training a speech recognizer.

4. The system of claim 2, wherein said phone dataset is utilized in building a phonetic dictionary.

5. The system of claim 2, wherein said transformation rules are applied to said phone dataset to produce said transformed phone dataset, said transformed phone dataset being for use in training a speech recognizer.

6. The system of claim 5, wherein said transformation rules include merge-type transformation rules that combine two adjacent phones in said phone dataset into a single phone selected from said enhanced phone set.

7. The system of claim 5, wherein said transformation rules include split-type transformation rules that separate one phone in said phone dataset into two different phones selected from said enhanced phone set.

8. The system of claim 5, wherein said transformation rules include replace-type transformation rules that replace one phone in said phone dataset with a different phone selected from said enhanced phone set.

9. The system of claim 5, wherein said transformation rules include change in context-type transformation rules that change one phone in said phone dataset to a different phone selected from said enhanced phone set depending on context.

10. The system of claim 1, wherein said enhanced phone set includes said acoustic-phonetic symbols, said acoustic-phonetic symbols being utilized in said transcription process to represent acoustic-phonetic processes of said speech data.

11. The system of claim 10, wherein said enhanced phone set further includes said connectors used in said transcription process to connect said acoustic-phonetic symbols to base-phones affected by acoustic-phonetic processes, thereby producing composite-phones.

12. The system of claim 11, wherein said connectors indicate how and where said acoustic-phonetic processes affect said base-phones.

13. The system of claim 12, wherein said connectors include a character “

>

”

that is placed to the left of one of said base-phones to indicate that one of said acoustic-phonetic processes affects a beginning of said one of said base-phones.

14. The system of claim 13, wherein said character “

>

”

is placed to the left of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects a beginning of said one of said composite-phones.

15. The system of claim 12, wherein said connectors include a character “

<

”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes affects an ending of said one of said base-phones.

16. The system of claim 15, wherein said character “

<

”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects an ending of said one of said composite-phones.

17. The system of claim 12, wherein said connectors include a character “

=”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes affects an entirety of said one of said base-phones.

18. The system of claim 17, wherein said character “

=”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects an entirety of said one of said composite-phones.

19. The system of claim 12, wherein said connectors include a character “

^”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes occurred completely within said one of said base-phones.

20. The system of claim 19, wherein said character “

^”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes occurred completely within said one of said composite-phones.

21. A system for speech processing, comprising:

speech data generated from one or more speech sources;

an enhanced phone set that includes acoustic-phonetic symbols and connectors for extending said enhanced phone set, said enhanced phone set including a TIMIT base-phone set and an extended base-phone set;

a transcription generated by a transcription process that selects appropriate phones from said enhanced phone set to represent said speech data, said acoustic-phonetic symbols being utilized in said transcription process to represent acoustic-phonetic processes of said speech data, said acoustic-phonetic processes represented by said acoustic-phonetic symbols including an epenthetic vowel; and

transformation rules applied to said enhanced phone set to produce a transformed phone dataset, said transformed phone dataset being used in building a phonetic dictionary.

22. A method for speech processing, comprising:

generating speech data from one or more speech sources;

providing an enhanced phone set that includes acoustic-phonetic symbols and connectors for extending said enhanced phone set, said enhanced phone set including a TIMIT base-phone set and an extended base-phone set, said extended base-phone set including a base phone for representing an articulator noise;

producing a transcription using a transcription process that selects appropriate phones from said enhanced phone set to represent said speech data; and

applying transformation rules to said enhanced phone set to produce a transformed phone dataset, said transformed phone dataset being used in building a phonetic dictionary.

23. The method of claim 22, further comprising the step of combining said speech data and said transcription to produce a phone dataset.

24. The method of claim 23, wherein said phone dataset is utilized in training a speech recognizer.

25. The method of claim 23, wherein said phone dataset is utilized in building a phonetic dictionary.

26. The method of claim 23, further comprising the step of applying said transformation rules to said phone dataset to produce said transformed phone dataset, said transformed phone dataset being for use in training a speech recognizer.

27. The method of claim 26, wherein said transformation rules include merge-type transformation rules that combine two adjacent phones in said phone dataset into a single phone selected from said enhanced phone set.

28. The method of claim 26, wherein said transformation rules include split-type transformation rules that separate one phone in said phone dataset into two different phones selected from said enhanced phone set.

29. The method of claim 26, wherein said transformation rules include replace-type transformation rules that replace one phone in said phone dataset with a different phone selected from said enhanced phone set.

30. The method of claim 26, wherein said transformation rules include change in context-type transformation rules that change one phone in said phone dataset to a different phone selected from said enhanced phone set depending on context.

31. The method of claim 22, wherein said enhanced phone set includes said acoustic-phonetic symbols, said acoustic-phonetic symbols being utilized in said transcription process to represent acoustic-phonetic processes of said speech data.

32. The method of claim 31, wherein said enhanced phone set further includes said connectors used in said transcription process to connect said acoustic-phonetic symbols to base-phones affected by acoustic-phonetic processes, thereby producing composite-phones.

33. The method of claim 32, wherein said connectors indicate how and where said acoustic-phonetic processes affect said base-phones.

34. The method of claim 33, wherein said connectors include a character “

>

”

that is placed to the left of one of said base-phones to indicate that one of said acoustic-phonetic processes affects a beginning of said one of said base-phones.

35. The method of claim 34, wherein said character “

>

”

is placed to the left of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects a beginning of said one of said composite-phones.

36. The method of claim 33, wherein said connectors include a character “

<

”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes affects an ending of said one of said base-phones.

37. The method of claim 36, wherein said character “

<

”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects an ending of said one of said composite-phones.

38. The method of claim 33, wherein said connectors include a character “

=”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes affects an entirety of said one of said base-phones.

39. The method of claim 38, wherein said character “

=”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes affects an entirety of said one of said composite-phones.

40. The method of claim 33, wherein said connectors include a character “

^”

that is placed to the right of one of said base-phones to indicate that one of said acoustic-phonetic processes occurred completely within said one of said base-phones.

41. The method of claim 40, wherein said character “

^”

is placed to the right of one of said composite-phones to indicate that one of said acoustic-phonetic processes occurred completely within said one of said composite-phones.

42. A method for speech processing, comprising:

generating speech data from one or more speech sources;

providing an enhanced phone set that includes acoustic-phonetic symbols and connectors for extending said enhanced phone set, said enhanced phone set including a TIMIT base-phone set and an extended base-phone set;

producing a transcription using a transcription process that selects appropriate phones from said enhanced phone set to represent said speech data, said acoustic-phonetic symbols being utilized in said transcription process to represent acoustic-phonetic processes of said speech data, said acoustic-phonetic processes represented by said acoustic-phonetic symbols including an epenthetic vowel; and

applying transformation rules to said enhanced phone set to produce a transformed phone dataset, said transformed phone dataset being used in building a phonetic dictionary.

43. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones, said enhanced base-phones including articulator noises l#, b#, hh#, w#, g#, ly#, ll#, and lq#;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process.

44. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data, said acoustic-phonetic processes represented by said acoustic-phonetic symbols including an epenthetic vowel; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process.

45. The method of claim 44, wherein said connectors include a character “

>

”

that is placed left of one of said enhanced base-phones to indicate that one of said acoustic-phonetic processes affects a beginning of said one of said enhanced base-phones, said connectors including a character “

<

”

that is placed right of said one of said base-phones to indicate that said one of said acoustic-phonetic processes affects an ending of said one of said enhanced base-phones, said connectors including a character “

=”

that is placed right of said one of said base-phones to indicate that said one of said acoustic-phonetic processes affects an entirety of said one of said enhanced base-phones, said connectors include a character “

^”

that is placed right of said one of said base-phones to indicate that said one of said acoustic-phonetic processes occurred completely within said one of said enhanced base-phones.

46. The method of claim 44, wherein said transformation rules include merge-type transformation rules that combine two adjacent phones in said detailed transcription into a single phone selected from said enhanced phone set, said transformation rules including split-type transformation rules that separate a first phone in said detailed transcription into two different phones selected from said enhanced phone set, said transformation rules including replace-type transformation rules that replace a second phone in said detailed transcription with a first different phone selected from said enhanced phone set, said transformation rules including change-in-context transformation rules that change a third phone in said detailed transcription to a second different phone selected from said enhanced phone set depending on a surrounding context of said input speech data.

47. The method of claim 44 wherein at least one of said composite enhanced phones includes a plurality of said connectors that extend a corresponding one of said enhanced base phones with a plurality of said acoustic-phonetic symbols.

48. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process, said transformation rules including a first merge rule that merges an original phone bcl and an original phone b into a merged phone b, said transformation rules including a second merge rule that merges an original phone tcl and an original phone t into a merged phone t, said transformation rules including a third merge rule that merges an original phone kcl and an original phone k into a merged phone k.

49. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process, said transformation rules including a first split rule that splits an original phone em into a first split phone ah and a second split phone m, said transformation rules including a second split rule that splits an original phone or into a first split phone ao and a second split phone r, said transformation rules including a third split rule that splits an original phone al into a first split phone aa and a second split phone l, said transformation rules including a fourth split rule that splits an original phone aa=n into a first split phone aa and a second split phone n.

50. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process, said transformation rules including a first replace rule that replaces an original phone gg with a replacement phone g, said transformation rules including a second replace rule that replaces an original phone qclq with a replacement phone q, said transformation rules include a third replace rule that replaces an original phone p=v with a replacement phone b.

51. A method for speech processing, comprising:

providing an enhanced phone set that includes enhanced base-phones for representing input speech data, acoustic-phonetic symbols that represent acoustic-phonetic content of said input speech data, and connectors for extending said enhanced base-phones by selectively connecting said acoustic-phonetic symbols to said enhanced base-phones to create composite enhanced phones;

producing a detailed transcription of said input speech data using a transcription process that selects appropriate phones from said enhanced phone set to represent said input speech data; and

applying transformation rules to said transcription to produce a transformed transcription, said transformed transcription being used to create a phonetic dictionary for a speech recognition process, said transformation rules including a change-in-context rule that replaces an original phone aa=n with a changed-context phone aa<

n m ng.