Phrase-based dialogue modeling with particular application to creating a recognition grammar for a voice-controlled user interface

• US 20040199375A1
• Filed: 04/05/2004
• Published: 10/07/2004
• Est. Priority Date: 05/28/1999
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Fluential, LLC
• Inventors: Ehsani, Farzad, Master, Demitrios L., Knodt, Eva M.
• Granted Patent: US 8,442,812 B2
• Time in Patent Office: Days
• US Class Current: 704/4
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

The invention enables creation of grammar networks that can regulate, control, and define the content and scope of human-machine interaction in natural language voice user interfaces (NLVUI). The invention enables phrase-based modeling of generic structures of verbal interaction to be used for the purpose of automating part of the design of such grammar networks. Most particularly, the invention enables such grammar networks to be used in providing a voice-controlled user interface to human readable text data that is also machine-readable (such as a Web page, a word processing document, a PDF document, or a spreadsheet).

Claims

1. A method for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the method comprising the steps of:

formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and

automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.

2. A method as in claim 1, wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing an interactive part of the text data.

3. A method as in claim 1, wherein the step of formulating an expression representing a part of the text data further comprises the step of formulating an expression representing a non-interactive part of the text data.

4. A method as in claim 1, wherein the text data represents one or more Web pages.

5. A method as in claim 4, wherein the step of formulating an expression further comprises the step of automatically parsing code representing the one or more Web pages to identify the one or more parts.

6. A method as in claim 5, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the step of parsing further comprising the step of identifying the system prompt and the plurality of option values.

7. A method as in claim 6, wherein the step of formulating an expression further comprises the steps of:

automatically identifying one or more phrases that correspond to a system prompt; and

automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

8. A method as in claim 4, wherein the code representing the one or more Web pages is expressed in a markup language.

9. A method as in claim 8, wherein the code representing the one or more Web pages is expressed in HTML.

10. A method as in claim 8, wherein the code representing the one or more Web pages is expressed in XML.

11. A method as in claim 4, wherein the recognition grammar is expressed in VXML.

12. A method as in claim 1, wherein the step of automatically using the phrase thesaurus to construct one or more equivalent expressions further comprises the steps of:

selecting a combination of one or more phrases representing the formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.

13. A method as in claim 12, wherein:

phrases in the phrase thesaurus have a probability of occurrence associated therewith;

one or more original phrases has a plurality of equivalent phrases; and

the step of identifying an equivalent phrase further comprises the step of selecting an equivalent phrase having the highest probability of occurrence.

14. A method as in claim 12, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the method further comprising the step of tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

15. A method as in claim 1, further comprising the step of translating the recognition grammar into a form that can be processed by a speech recognition system.

16. A method as in claim 1, further comprising the step of manually editing the recognition grammar.

17. A method as in claim 1, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.

35. A system as in claim 1, wherein the means for automatically using the phrase thesaurus to construct equivalent phrases further comprises:

means for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

means for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

means for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.

37. A system as in claim 35, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the system further comprising means for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

18. A system for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the system comprising:

means for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and

means for automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.

19. A system as in claim 18, wherein the means for formulating an expression of each of one or more anticipated spoken inputs to the interface further comprises a graphical user interface device.

20. A system as in claim 18, wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing an interactive part of the text data.

21. A system as in claim 18, wherein the means for formulating an expression representing a part of the text data further comprises means for formulating an expression representing a non-interactive part of the text data.

22. A system as in claim 18, wherein the text data represents one or more Web pages.

23. A system as in claim 22, wherein the means for formulating an expression further comprises means for automatically parsing code representing the one or more Web pages to identify the one or more parts.

24. A system as in claim 23, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, and wherein the means for parsing further comprises means for identifying the system prompt and the plurality of option values.

25. A system as in claim 24, wherein the means for formulating an expression further comprises:

means for automatically identifying one or more phrases that correspond to a system prompt; and

means for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

26. A system as in claim 22, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in a markup language.

27. A system as in claim 26, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in HTML.

28. A system as in claim 26, wherein the means for automatically parsing further comprises means for automatically parsing code expressed in XML.

29. A system as in claim 22, wherein the recognition grammar is expressed in VXML.

30. A system as in claim 18, further comprising data storage means for storing data representing the phrase thesaurus and the recognition grammar.

31. A system as in claim 30, wherein the data storage means further stores data representing lexical items that can be used to complete a phrase template.

32. A system as in claim 30, wherein the data storage means further stores data representing a probability of occurrence of phrases.

33. A system as in claim 32, wherein the phrases are stored in the data storage means in accordance with the corresponding probability of occurrence.

34. A system as in claim 32, wherein the means for automatically using the phrase thesaurus to construct equivalent phrases further comprises means for using the data representing a probability of occurrence of phrases to construct a probabilistic grammar.

36. A system as in claim 25, wherein:

phrases in the phrase thesaurus have a probability of occurrence associated therewith;

one or more original phrases has a plurality of equivalent phrases; and

the means for identifying an equivalent phrase further comprises means for selecting an equivalent phrase having the highest probability of occurrence.

38. A system as in claim 18, further comprising means for translating the recognition grammar into a form that can be processed by a speech recognition system.

39. A system as in claim 18, further comprising means for manually editing the recognition grammar.

40. A system as in claim 18, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.

41. A computer readable storage medium encoded with one or more computer programs for creating a recognition grammar for use with an interactive user interface to human readable text data that is also machine readable, the interactive user interface being responsive to spoken input, the computer programs comprising:

instructions for formulating an expression representing a part of the text data for each of one or more parts of the text data, wherein each formulated expression can be constructed as one or more combinations of one or more phrases in a phrase thesaurus; and

instructions for automatically using the phrase thesaurus to construct one or more equivalent expressions of one or more formulated expressions, wherein the recognition grammar comprises the collection of all of the expressions.

42. A computer readable storage medium as in claim 41, wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing an interactive part of the text data.

43. A computer readable storage medium as in claim 41, wherein the instructions for formulating an expression representing a part of the text data further comprise instructions for formulating an expression representing a non-interactive part of the text data.

44. A computer readable storage medium as in claim 41, wherein the text data represents one or more Web pages.

45. A computer readable storage medium as in claim 44, wherein the instructions for formulating an expression further comprise instructions for automatically parsing code representing the one or more Web pages to identify the one or more parts.

46. A computer readable storage medium as in claim 45, wherein one or more of the parts of the one or more Web pages comprise a system prompt indicating a type of interaction with an interactive part of a web page and a plurality of option values each representing a possible input to the interactive user interface for that type of interaction, the instructions for parsing further comprising instructions for identifying the system prompt and the plurality of option values.

47. A computer readable storage medium as in claim 46, wherein the instructions for formulating an expression further comprise:

instructions for automatically identifying one or more phrases that correspond to a system prompt; and

instructions for automatically identifying, for each of a plurality of option values, one or more phrases that correspond to the option value.

48. A computer readable storage medium as in claim 44, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in a markup language.

49. A computer readable storage medium as in claim 48, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in HTML.

50. A computer readable storage medium as in claim 48, wherein the instructions for automatically parsing further comprise instructions for automatically parsing code expressed in XML.

51. A computer readable storage medium as in claim 44, wherein the recognition grammar is expressed in VXML.

52. A computer readable storage medium as in claim 41, wherein the instructions for automatically using the phrase thesaurus to construct equivalent phrases further comprise:

instructions for selecting a combination of one or more phrases representing a formulated expression, wherein the phrases of the selected combination of one or more phrases are original phrases of the formulated expression;

instructions for identifying an equivalent phrase for each of one or more original phrases of the formulated expression; and

instructions for producing a new combination of one or more phrases representing the formulated expression, the new combination including at least one of the identified equivalent phrases, wherein the new combination represents the equivalent expression.

53. A computer readable storage medium as in claim 52, wherein:

phrases in the phrase thesaurus have a probability of occurrence associated therewith;

one or more original phrases has a plurality of equivalent phrases; and

the instructions for identifying an equivalent phrase further comprise instructions for selecting an equivalent phrase having the highest probability of occurrence.

54. A computer readable storage medium as in claim 52, wherein equivalent phrases are grouped in classes and each class of equivalent phrases has associated therewith a descriptor denoting a conceptual representation of the phrases contained in that phrase class, the one or more computer programs further comprising instructions for tagging each equivalent expression with the descriptor or descriptors associated with phrases of the equivalent expression.

55. A computer readable storage medium as in claim 41, further comprising instructions for translating the recognition grammar into a form that can be processed by a speech recognition system.

56. A computer readable storage medium as in claim 41, further comprising instructions for manually editing the recognition grammar.

57. A computer readable storage medium as in claim 41, wherein expressions representing a plurality of parts of the text data are formulated and the phrase thesaurus is used to identify equivalent expressions for a plurality of formulated expressions.