Phrase-based dialogue modeling with particular application to creating a recognition grammar for a voice-controlled user interface

• US 8,442,812 B2
• Filed: 04/05/2004
• Issued: 05/14/2013
• Est. Priority Date: 05/28/1999
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Fluential, LLC
• Inventors: Ehsani, Farzad, Knodt, Eva M., Master, Demitrios L.
• Primary Examiner(s): Neway, Samuel G
• Application Number:
  US10/818,219
• Publication Number:
  US 20040199375A1
• Time in Patent Office: 3,326 Days
• Field of Search: 704/4, 704/9, 704231-257
• US Class Current: 704/9
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

The invention enables creation of grammar networks that can regulate, control, and define the content and scope of human-machine interaction in natural language voice user interfaces (NLVUI). The invention enables phrase-based modeling of generic structures of verbal interaction to be used for the purpose of automating part of the design of such grammar networks. Most particularly, the invention enables such grammar networks to be used in providing a voice-controlled user interface to human readable text data that is also machine-readable (such as a Web page, a word processing document, a PDF document, or a spreadsheet).

Claims

1. A method for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the method comprising the steps of:

providing access to a phrase thesaurus database comprising a plurality of classes of phrases, wherein any two phrases that are semantic equivalent of each other are assigned to a same class;

formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression is constructed as one or more combinations of one or more phrases in the phrase thesaurus database; and

by a computing device, automatically using the phrase thesaurus database to construct one or more equivalent expressions of each formulated expression based on assigned classes of the one or more phrases of each formulated expression, wherein the recognition grammar comprises the collection of all of the expressions.

2. The method as in claim 1, wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing an interactive part of the text data.

3. The method as in claim 1, wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing a non interactive part of the text data.

4. The method as in claim 1, wherein the text data represents one or more Web pages.

5. The method as in claim 4, wherein the step of formulating an expression further comprises the step of automatically parsing code representing the one or more Web pages to identify the one or more parts.

6. The method as in claim 5, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the step of parsing further comprising the step of identifying the system prompt and the plurality of option values.

7. The method as in claim 6, wherein the step of formulating an expression further comprises the steps of

automatically identifying one or more phrases that correspond to a system prompt;

and

automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

8. The method as in claim 4, wherein the code representing the one or more Web pages is expressed in a markup language.

9. The method as in claim 8, wherein the code representing the one or more Web pages is expressed in HTML.

10. The method as in claim 8, wherein the code representing the one or more Web pages is expressed in XML.

11. The method as in claim 4, wherein the recognition grammar is expressed in VXML.

12. The method as in claim 1, wherein the step of automatically using the phrase thesaurus database to construct one or more equivalent expressions further comprises the steps of:

selecting a combination of one or more phrases representing the formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.

13. The method as in claim 12, wherein:

each phrases in the phrase thesaurus database is associated with a probability of occurrence;

one or more original phrases has a plurality of equivalent phrases; and

the step of identifying an equivalent phrase further comprises the step of selecting an equivalent phrase having the highest probability of occurrence.

14. The method as in claim 12, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the method further comprising the step of tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

15. The method as in claim 1, further comprising the step of translating the recognition grammar into a form that can be processed by a speech recognition system.

16. The method as in claim 1, further comprising the step of manually editing the recognition grammar.

17. The method as in claim 1, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus database is used to identify equivalent expressions for a plurality of formulated expressions.

18. A system for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the system comprising:

means for providing access to a phrase thesaurus database comprising a plurality of classes of phrases, wherein any two phrases that are semantic equivalent of each other are assigned to a same class;

means for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression is constructed as one or more combinations of one or more phrases in the phrase thesaurus database; and

means for automatically using the phrase thesaurus database to construct one or more equivalent expressions of each formulated expression based on assigned classes of the one or more phrases of each formulated expression, wherein the recognition grammar comprises the collection of all of the expressions.

19. The system as in claim 18, wherein the means for formulating an expression of each of one or more anticipated spoken inputs to the interface further comprises a graphical user interface device.

20. The system as in claim 18, wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing an interactive part of the text data.

21. The system as in claim 18, wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing a non-interactive part of the text data.

22. The system as in claim 18, wherein the text data represents one or more Web pages.

23. The system as in claim 22, wherein the means for formulating an expression further comprises means for automatically parsing code representing the one or more Web pages to identify the one or more parts.

24. The system as in claim 23, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, and wherein the means for parsing further comprises means for identifying the system prompt and the plurality of option values.

25. The system as in claim 24, wherein the means for formulating an expression further comprises:

means for automatically identifying one or more phrases that correspond to a system prompt; and

means for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

26. The system as in claim 22, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in a markup language.

27. The system as in claim 26, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in HTML.

28. The system as in claim 26, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in XML.

29. The system as in claim 22, wherein the recognition grammar is expressed in VXML.

30. The system as in claim 18, further comprising data storage means for storing data representing the phrase thesaurus database and the recognition grammar.

31. The system as in claim 30, wherein the data storage means further stores data representing lexical items that can be used to complete a phrase template.

32. The system as in claim 30, wherein the data storage means further stores data representing a probability of occurrence of phrases.

33. The system as in claim 32, wherein the phrases are stored in the data storage means in accordance with the corresponding probability of occurrence.

34. The system as in claim 32, wherein the means for automatically using the phrase thesaurus database to construct equivalent phrases further comprises means for using the data representing a probability of occurrence of phrases to construct a probabilistic grammar.

35. The system as in claim 18, wherein the means for automatically using the phrase thesaurus database to construct equivalent phrases further comprises:

means for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

means for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

means for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.

36. The system as in claim 25, wherein:

each phrase in the phrase thesaurus database is associated with a probability of occurrence;

one or more original phrases has a plurality of equivalent phrases; and

the means for identifying an equivalent phrase further comprises means for selecting an equivalent phrase having the highest probability of occurrence.

37. The system as in claim 35, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the system further comprising means for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

38. The system as in claim 18, further comprising means for translating the recognition grammar into a form that can be processed by a speech recognition system.

39. The system as in claim 18, further comprising means for manually editing the recognition grammar.

40. The system as in claim 18, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus database is used to identify equivalent expressions for a plurality of formulated expressions.

41. A non-transitory computer readable storage medium encoded with one or more computer programs executable by at least one processor, the one or more computer programs for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the computer programs comprising:

instructions for providing access to a phrase thesaurus database comprising a plurality of classes of phrases, wherein any two phrases that are semantic equivalent of each other are assigned to a same class;

instructions for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression is constructed as one or more combinations of one or more phrases in the phrase thesaurus database; and

instructions for automatically using the phrase thesaurus database to construct one or more equivalent expressions of each formulated expression based on assigned classes of the one or more phrases of each formulated expression, wherein the recognition grammar comprises the collection of all of the expressions.

42. The non-transitory computer readable storage medium as in claim 41, wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing an interactive part of the text data.

43. The non-transitory computer readable storage medium as in claim 41, wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing a non-interactive part of the text data.

44. The non-transitory computer readable storage medium as in claim 41, wherein the text data represents one or more Web pages.

45. The non-transitory computer readable storage medium as in claim 44, wherein the instructions for formulating an expression further comprise instructions for automatically parsing code representing the one or more Web pages to identify the one or more parts.

46. The non-transitory computer readable storage medium as in claim 45, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the instructions for parsing further comprising instructions for identifying the system prompt and the plurality of option values.

47. The non-transitory computer readable storage medium as in claim 46, wherein the instructions for formulating an expression further comprise:

instructions for automatically identifying one or more phrases that correspond to a system prompt; and

instructions for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

48. The non-transitory computer readable storage medium as in claim 44, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in a markup language.

49. The non-transitory computer readable storage medium as in claim 48, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in HTML.

50. The non-transitory computer readable storage medium as in claim 48, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in XML.

51. The non-transitory computer readable storage medium as in claim 44, wherein the recognition grammar is expressed in VXML.

52. The non-transitory computer readable storage medium as in claim 41, wherein the instructions for automatically using the phrase thesaurus database to construct equivalent phrases further comprise:

instructions for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

instructions for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

instructions for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination the equivalent expression.

53. The non-transitory computer readable storage medium as in claim 52, wherein:

each phrase in the phrase thesaurus database is associated with a probability of occurrence;

one or more original phrases has a plurality of equivalent phrases; and

the instructions for identifying an equivalent phrase further comprise instructions for selecting an equivalent phrase having the highest probability of occurrence.

54. The non-transitory computer readable storage medium as in claim 52, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the one or more computer programs further comprising instructions for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

55. The non-transitory computer readable storage medium as in claim 41, further comprising instructions for translating the recognition grammar into a form that can be processed by a speech recognition system.

56. The non-transitory computer readable storage medium as in claim 41, further comprising instructions for manually editing the recognition grammar.

57. The non-transitory computer readable storage medium as in claim 41, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus database is used to identify equivalent expressions for a plurality of formulated expressions.