Method and system for building a phonotactic model for domain independent speech recognition

• US 8,392,188 B1
• Filed: 09/21/2001
• Issued: 03/05/2013
• Est. Priority Date: 11/05/1999
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Riccardi, Giuseppe
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US09/956,907
• Time in Patent Office: 4,183 Days
• Field of Search: 704/244, 704/231, 704/270, 704/250, 704/243, 704/245, 704/247, 704/249
• US Class Current: 704/249
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/07 to the speaker
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: G10L 25/27 characterised by the analysis technique

Abstract

The invention concerns a method and corresponding system for building a phonotactic model for domain independent speech recognition. The method may include recognizing phones from a user'"'"'s input communication using a current phonotactic model, detecting morphemes (acoustic and/or non-acoustic) from the recognized phones, and outputting the detected morphemes for processing. The method also updates the phonotactic model with the detected morphemes and stores the new model in a database for use by the system during the next user interaction. The method may also include making task-type classification decisions based on the detected morphemes from the user'"'"'s input communication.

Claims

1. A method for building a phonotactic model for domain independent speech recognition, comprising:

recognizing phones from a user'"'"'s input communication using a current phonotactic model stored in a database;

detecting morphemes from the recognized phones;

creating, via a new processor, a new phonotactic model using the detected morphemes, the creating the new phonotactic model comprising transforming a prior probability distribution associated with a first domain to a prior probability distribution associated with a second domain;

replacing the current phonotactic model with the new phonotactic model in the database; and

outputting the detected morphemes for processing.

2. The method of claim 1, wherein the morphemes include at least one of verbal speech or non-verbal speech.

3. The method of claim 2, wherein the non-verbal speech includes use of at least one of gestures, body movements, head movements, non-responses, text, keyboard entries, keypad entries, mouse clicks, DTMF codes, pointers, stylus, cable set-top box entries, graphical user interface entries or touchscreen entries.

4. The method of claim 2, wherein the user'"'"'s input communication is derived from the verbal and non-verbal speech and an environment of the user.

5. The method of claim 1, wherein the morphemes are expressed in multimodal form.

6. The method of claim 1, wherein the morphemes in the user'"'"'s input communication are derived from actions of the user, including a focus of attention of the user.

7. The method of claim 1, further comprising:

mapping the detected acoustic morphemes into words.

8. The method of claim 1, wherein the method operates in conjunction with one or more communication networks, the communication networks including a telephone network, the Internet, an intranet, cable TV network, a local area network (LAN), and a wireless communication network.

9. The method of claim 1, wherein the method is used for customer care purposes.

10. The method of claim 1, wherein the morphemes are detected using a measure of commonality within a language of the morphemes.

11. The method of claim 10, wherein the commonality measure is a mutual information measure.

12. The method of claim 11, wherein each of the plurality of detected morphemes has a mutual information measure exceeding a predetermined threshold.

13. The method of claim 1, wherein detecting acoustic morphemes includes using a confidence function.

14. The method of claim 1, wherein the new phonotactic model is used when recognizing phones for a future input communication from a user.

15. The method of claim 1, wherein the method is iterative.

16. The method of claim 1, wherein the morphemes are at least one of acoustic morphemes or non-acoustic morphemes.

17. The method of claim 1, wherein creating a new phonotactic model using the detected morphemes further comprises:

mapping phone sequence statistics into word statistics; and

transforming a word probability vector of the prior probability distribution of the first domain using the mapped word statistics.

18. A method of task classification using a phonotactic model built for domain independent speech recognition, comprising:

recognizing phones from a user'"'"'s input communication using a current phonotactic model stored in a database;

detecting morphemes from the recognized phones;

creating, via a processor, a new phonotactic model using the detected morpheme, the creating the new phonotactic model comprising transforming a prior probability distribution associated with a first domain to a prior probability distribution associated with a second domain;

replacing the current phonotactic model with the new phonotactic model in the database; and

making a task-type classification decision based on the detected morphemes from the user'"'"'s input communication.

19. The method of claim 18, wherein the morphemes include at least one of verbal speech or non-verbal speech.

20. The method of claim 19, wherein the non-verbal speech includes the use of at least one of gestures, body movements, head movements, non-responses, text, keyboard entries, keypad entries, mouse clicks, DTMF codes, pointers, stylus, cable set-top box entries, graphical user interface entries or touchscreen entries.

21. The method of claim 19, wherein the user'"'"'s input communication is derived from the verbal and non-verbal speech and an environment of the user.

22. The method of claim 19, wherein each of the verbal and non-verbal speech are directed to one of a set of predetermined task objectives and each of the verbal and non-verbal speech is labeled with one of the predetermined task objectives to which it is directed.

23. The method of claim 18, wherein the morphemes are expressed in multimodal form.

24. The method of claim 18, wherein the morphemes in the user'"'"'s input communication are derived from actions of the user, including a focus of attention of the user.

25. The method of claim 18, further comprising entering into a dialog with the user to obtain a feedback response from the user.

26. The method of claim 25, wherein entering into a dialog with the user to obtain a feedback response from the user, further prompting the user to provide a feedback response including additional information with respect to an initial input communication of the user.

27. The method of claim 25, wherein the user is prompted to provide a feedback response that includes confirmation with respect to at least one set of task objectives determined in the task-type classification decision.

28. The method of claim 18, wherein the input communication is routed based on the task-type classification decision.

29. The method of claim 28, wherein a task objective is performed after the input communication is routed.

30. The method of claim 18, wherein the method operates in conjunction with one or more communication networks, the communication networks including a telephone network, the Internet, an intranet, cable TV network, a local area network (LAN), and a wireless communication network.

31. The method of claim 18, wherein the method is used for customer care purposes.

32. The method of claim 18, wherein the classification decisions decision and corresponding user input communications communication are collected for automated learning purposes.

33. The method of claim 18, wherein a relationship between the detected morphemes and a predetermined set of task objectives includes a measure of usefulness of one of the morphemes to a specified one of the predetermined set of task objectives.

34. The method of claim 18, wherein a relationship between the detected morphemes and a predetermined set of task objectives includes a measure of commonality within a language of the morphemes.

35. The method of claim 34, wherein the commonality measure is a mutual information measure.

36. The method of claim 35, wherein each of the detected morphemes has a mutual information measure exceeding a predetermined threshold.

37. The method of claim 18, wherein making a task-type classification decision includes using a confidence function.

38. The method of claim 18, wherein the input communication from the user represents a request for at least one of a set of predetermined task objectives.

39. The method of claim 18, wherein the input communication is responsive to a query of a of form “

How may I help you?”

.

40. The method of claim 18, wherein the new phonotactic model is used when recognizing phones for a future input communication from a user.

41. The method of claim 18, wherein the method is iterative.

42. The method of claim 18, wherein the morphemes are at least one of acoustic morphemes or non-acoustic morphemes.

43. The method of claim 18, wherein creating a new phonotactic model using the detected morphemes further comprises:

mapping phone sequence statistics into word statistics; and

transforming a word probability vector of the prior probability distribution of the first domain using the mapped word statistics.

44. A system that builds a phonotactic model for domain independent speech recognition, comprising:

a database that stores phonotactic models;

a recognizer that recognizes phones from a user'"'"'s input communication using a current phonotactic model stored in the database;

a morpheme detector that detects morphemes from the recognized phones and outputs the detected morphemes for processing; and

a learning unit that creates a new phonotactic model using the detected morphemes and transforms a prior probability distribution associated with a first domain to a prior probability distribution associated with a second domain, wherein the learning unit replaces the current phonotactic model with the new phonotactic model in the database.

45. The system of claim 44, further comprising:

a task classification processor that makes task classification decisions based on the detected morphemes from the user'"'"'s input communication.

46. The system of claim 44, wherein the learning unit is configured to perform:

mapping phone sequence statistics into word statistics; and

transforming a word probability vector of the prior probability distribution of the first domain using the mapped word statistics.