Method and apparatus for obtaining a transcription of phrases through text and spoken utterances

• US 6,243,680 B1
• Filed: 06/15/1998
• Issued: 06/05/2001
• Est. Priority Date: 06/15/1998
• Status: Expired due to Term

• Current Assignee: Avaya Incorporated
• Original Assignee: Nortel Networks Limited (Nortel Networks Corporation)
• Inventors: Sabourin, Michael, Gupta, Vishwa Nath
• Primary Examiner(s): Tsang, Fan
• Assistant Examiner(s): Opsasnick, Michael N.
• Application Number:
  US09/094,653
• Time in Patent Office: 1,086 Days
• Field of Search: 704/260
• US Class Current: 704/260
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams

Abstract

A method and an apparatus for adding a new entry to a speech recognition dictionary, and more particularly a system and method for generating transcriptions from multiple utterances and a textual representation of a given word is provided. The method and apparatus combine acoustic information obtained from each of the training utterances to generate the most likely transcriptions. It also provides a method and apparatus for transcribing multiple utterances into a single transcription that can be added to a speech recognition dictionary. In a first step, a graph representative of possible pronunciation of the word or combination of words is generated on the basis of the textual representation of the word and a letter to phoneme set of rules. Phonemes perceptually close to the phonemes in the transcriptions obtained using the letter to phoneme rules are added on parallel branches in the generated graph. Following this, each branch in the graph is replaced with its allophones and the complete graph is scored using training utterances to obtain the most likely allophone sequence on the basis of the multiple utterances of the word.

Claims

1. An apparatus for creating a transcription capable of being used to generate an entry in a speech recognition dictionary for a certain word, said apparatus comprising:

a first input for receiving a set of audio signals derived from respective utterances of the certain word;

a second input for receiving a data element representative of an orthographic representation of the certain word;

a processing unit coupled to said first input and to said second input for processing the set of audio signals and the data element to provide a transcription of the certain word, the transcription of the certain word combining acoustic information originating from at least two audio signals of said set of audio signals.

2. An apparatus as defined in claim 1, wherein said processing unit is operative for generating at least in part on the basis of the data element a data structure representative of different pronunciations of the certain word.

3. An apparatus as defined in claim 2, wherein said data structure is a graph having branches, each branch of said graph being associated with a certain acoustic symbol.

4. An apparatus as defined in claim 3, wherein said certain acoustic symbol is a phoneme.

5. An apparatus as defined in claim 3, wherein said processing unit is operative for processing the data structure and the set of audio signals to generate the transcription.

6. An apparatus as defined in claim 3, wherein said processing unit is operative for scoring said graph on the basis of said set of audio signals to compute a likelihood score and generate a transcription on the basis of the likelihood score.

7. An apparatus as defined in claim 2, wherein said processing unit is operative for generating a plurality of transcriptions on the basis of the data element representative of an orthographic representation of the certain word.

8. An apparatus as defined in claim 7, wherein said processing unit is operative for generating a graph on the basis of said plurality of transcriptions, said graph being representative of different pronunciations of the certain word.

9. An apparatus as defined in claim 1, wherein the utterances are vocally expressed by the same individual.

10. An apparatus as defined in claim 1, wherein the utterances are vocally expressed by different individuals.

11. An apparatus as defined in claim 1, wherein said processing unit is operative for grouping the utterances of the certain word into utterance clusters on the basis of acoustic properties of the utterances.

12. An apparatus as defined in claim 11, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.

13. An apparatus as defined in claim 1, wherein said processing unit is operative for grouping into utterance clusters the utterances of the certain word on the basis of heuristic properties of the utterances.

14. An apparatus as defined in claim 13, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.

15. A method for creating a transcription capable of being used to generate an entry in a speech recognition dictionary for a certain word, said method comprising the steps of:

receiving a set of audio signals derived from respective utterances of the certain word;

receiving a data element representative of an orthographic representation of the certain word;

processing the set of audio signals and the data element to provide a transcription of the certain word, the transcription of the certain word combining acoustic information originating from at least two audio signals of said set of audio signals;

utilizing the transcription to create an entry in a speech recognition dictionary implemented on a computer readable medium.

16. A method as defined in claim 15, wherein said processing step comprises the step of generating at least in part on the basis of the the data element a data structure representative of different pronunciations of the certain word.

17. A method as defined in claim 16, wherein said data structure is a graph having branches, each branch of said graph being associated with a certain acoustic symbol.

18. A method as defined in claim 17, wherein said certain acoustic symbol is a phoneme.

19. A method as defined in claim 18, wherein said processing step includes the step of processing the data structure and the set of audio signals to generate the transcription.

20. A method as defined in claim 17, wherein said processing step includes the step of:

scoring said graph on the basis of said set of audio signals to generate a likelihood score and generate a transcription on the basis of the likelihood score.

21. A method as defined in claim 16, said method further comprising the step of generating a plurality of transcriptions on the basis of the data element representative of an orthographic representation of the certain word.

22. A method as defined in claim 21, said method further comprising the step of generating a graph on the basis of said plurality of transcriptions, said graph being representative of different pronunciations of the certain word.

23. A method as defined in claim 15, wherein the utterances are vocally expressed by the same individual.

24. A method as defined in claim 15, wherein the utterances are vocally expressed by different individuals.

25. A method as defined in claim 15, wherein said processing step includes the step of grouping the utterances of the certain word into utterance clusters on the basis of acoustic properties of the utterances.

26. A method as defined in claim 25, said method further comprising generating a transcription of the certain word for each utterance cluster.

27. A method as defined in claim 15, said method further comprising the step of grouping into utterance clusters the utterances of the certain word on the basis of heuristic properties of the utterances.

28. A method as defined in claim 27, said method comprising generating a transcription of the certain word for each utterance cluster.

29. A machine-readable storage medium containing a program element to direct a computer to generate a transcription capable of being used to generate an entry in a speech recognition dictionary for a certain word, said program element implementing functional blocks, said functional blocks comprising:

a first input for receiving a set of audio signals derived from respective utterances of the certain word;

a second input for receiving a data element representative of an orthographic representation of the certain word;

a processing unit coupled to said first input and to said second input for processing the set of audio signals and the data element to provide a transcription of the certain word, the transcription of the certain word combining acoustic information originating from at least two audio signals of said set of audio signals.

30. A machine-readable storage medium as defined in claim 29, wherein said processing unit is operative for generating at least in part on the basis of the data element a data structure representative of different pronunciations of the certain word.

31. A machine-readable storage medium as defined in claim 30, wherein said data structure is a graph having branches, each branch of said graph being associated with a certain acoustic symbol.

32. A machine-readable storage medium as defined in claim 31, wherein said certain acoustic symbol is a phoneme.

33. A machine-readable storage medium as defined in claim 31, wherein said processing unit is operative for processing the data structure and the set of audio signals to generate the transcription.

34. A machine-readable storage medium as defined in claim 31, wherein said processing unit is operative for scoring said graph on the basis of said set of audio signals to compute a likelihood score and generate a transcription on the basis of the likelihood score.

35. A machine-readable storage medium as defined in claim 30, wherein said processing unit is further operative for generating a plurality of transcriptions on the basis of the data element representative of an orthographic representation of the certain word.

36. A machine-readable storage medium as defined in claim 35, wherein said processing unit is further operative for generating a graph on the basis of said plurality of transcriptions, said graph being representative of different pronunciations of the certain word.

37. A machine-readable storage medium as defined in claim 29, wherein the utterances are vocally expressed by the same individual.

38. A machine-readable storage medium as defined in claim 29, wherein the utterances are vocally expressed by different individuals.

39. A machine-readable storage medium as defined in claim 29, wherein said processing unit is operative for grouping the utterances of the certain word into utterance clusters on the basis of acoustic properties of the utterances.

40. A machine-readable storage medium as defined in claim 39, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.

41. A machine-readable storage medium as defined in claim 29, wherein said processing unit is operative for grouping into utterance clusters the utterances of the certain word on the basis of heuristic properties of the utterances.

42. A machine-readable storage medium as defined in claim 41, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.

43. An apparatus for creating an entry for a certain word in a speech recognition dictionary, said apparatus including:

a first input for receiving a set of audio signals derived from respective utterances of the certain word;

a second input for receiving a data element representative of an orthographic representation of the certain word;

a processing unit operative for;

a) generating on the basis of the data element a data structure representative of different pronunciations of the certain word;

b) processing the data structure on a basis of the set of audio signals to derive a transcription of the certain word, the transcription of the certain word combining acoustic information originating from at least two signals of said set of audio signals;

c) generating at least in part on the basis of said transcription an entry in the speech recognition dictionary for the certain word.

44. An apparatus as defined in claim 43, wherein said data structure is a graph having branches, each branch of said graph being associated with a certain acoustic symbol.

45. An apparatus as defined in claim 44, wherein said certain acoustic symbol is a phoneme.

46. An apparatus as defined in claim 44, wherein said processing unit is operative for scoring said graph on the basis of the set of audio signal to compute a likelihood score and generate a transcription on the basis of the likelihood score.

47. An apparatus as defined in claim 43, wherein said processing unit is further operative for generating a plurality of transcriptions on the basis of the data element representative of an orthographic representation of the certain word.

48. An apparatus as defined in claim 47, wherein said processing unit is further operative for generating a graph on the basis of said plurality of transcriptions, said graph being representative of different pronunciations of the certain word.

49. An apparatus as defined in claim 43, wherein the utterances are vocally expressed by the same individual.

50. An apparatus as defined in claim 43, wherein the utterances are vocally expressed by different individuals.

51. An apparatus as defined in claim 43, wherein said processing unit is operative for grouping the utterances of the certain word into utterance clusters on the basis of acoustic properties of the utterances.

52. An apparatus as defined in claim 51, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.

53. An apparatus as defined in claim 43, wherein said processing unit is operative for grouping into utterance clusters the utterances of the certain word on the basis of heuristic properties of the utterances.

54. An apparatus as defined in claim 53, wherein said processing unit is operative for generating a transcription of the certain word for each utterance cluster.