Dialog manager for interactive dialog with computer user

• US 20030137537A1
• Filed: 03/14/2002
• Published: 07/24/2003
• Est. Priority Date: 12/28/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Bo, Shum, Heung-Yeung, Guo, Baining
• Granted Patent: US 7,610,556 B2
• Time in Patent Office: Days
• US Class Current: 345/751
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A dialog manager module accesses a dialog model, a discourse model and a domain model to render information to a computer user through the course of a dialog exchange with the computer user. Actions are performed according to a set of rules during the dialog. Each of the rules comprise an action to perform as a function of the current dialog state and a filled form of slots of semantic information of the user input.

Claims

1. A method of providing a dialog interface in a computer system, the method comprising:

(a) receiving user input;

(b) ascertaining semantic information of the user input;

(c) ascertaining a current dialog state as a function of the semantic information; and

(d) repeating at least steps (a) and (b) and rendering information to the user according to an action when a rule of a set of rules has been met, each rule of the set of rules comprising an action to be performed as a function of the current dialog state and a filled form of slots of semantic information of the user input.

2. The computer-implemented method of claim 1 and further comprising rendering information to the user if a dialog does not exist with the user.

3. The computer-implemented method of claim 2 wherein rendering information to the user if a dialog does not exist with the user includes accessing a task model defining tasks in a hierarchical structure wherein at least some tasks comprises sub-tasks, and wherein rendering includes sequentially rendering sub-tasks to complete a larger task.

4. The computer-implemented method of claim 3 wherein rendering information to the user if a dialog does not exist includes suspending rendering of information according to a task when a dialog begins with the user and resuming rendering according to the task when the dialog ends.

5. The computer-implemented method of claim 1 wherein rendering information according to an action includes rendering information as a function of a user preference.

6. The computer-implemented method of claim 5 and further comprising adjusting the user preference as a function of the dialog.

7. The computer-implemented method of claim 6 wherein adjusting the user preference includes monitoring a number of interruptions made by the user during rendering.

8. A computer readable medium having instructions, which when executed on a computer provide a user interface, the instructions comprising:

a user input module adapted to receive input from a user and provide semantic information as a function of the input;

a dialog model defining a set of rules, each rule comprising an action to be performed as function of received input when a dialog exists with the user;

a discourse model defining if a dialog exists with the user as a function of received input;

a domain model defining information to be rendered to the user;

a task model defining some of the information to be rendered to the user as tasks; and

a module receiving the user input and accessing the discourse model, the dialog model, the task model and the domain model to render information according to a rule if dialog exists with the user and according to a task if dialog does not exist with the user.

9. The computer readable medium of claim 8 and further comprising a parser receiving the user input and ascertaining semantic information pertaining to the user input, the module rendering information as a function of the semantic information.

10. The computer readable medium of claim 8 wherein the task model is organized in a hierarchical manner wherein at least some tasks comprise a plurality of sub-tasks related to actions for rendering information, and wherein the module sequentially renders information defined by the sub-tasks in order to complete a larger task.

11. The computer readable medium of claim 10 wherein the module suspends rendering information defined by a task in the task model when using a rule of the dialog model, and wherein the module resumes rendering information defined by the task when at least some of the rules of the dialog model do not apply.

12. The computer readable medium of claim 11 wherein each sub-task comprises at least one of an action for rendering information and another sub-task.

13. The computer readable medium of claim 8 and further comprising a user model defining user preferences related to rendering information, the module accessing the user model and rendering information as a function of user preferences.

14. The computer readable medium of claim 13 wherein module adjusts the user preferences as a function of the dialog with the user.

15. The computer readable medium of claim 14 wherein information to be rendered is prioritized and wherein one of the user preferences indicates a level of priority of information to be rendered.

16. The computer readable medium of claim 15 wherein the module adjusts the user preference related to the level of priority as a function of interruptions made by the user.

17. The computer readable medium of claim 14 wherein the information to be rendered includes stories, and wherein one of the user preferences relates to a frequency of stories to be rendered.

18. The computer readable medium of claim 17 wherein the module adjusts the user preference related to the frequency of stories to be rendered as a function of at least one of responses received from the user as to whether a story should be rendered and interruptions made by the user during telling a story.

19. The computer readable medium of claim 14 wherein at least some of the actions comprise rendering information to provoke a question from the user, and wherein one of the user preferences relates to a frequency for rendering information to provoke a question from the user.

20. The computer readable medium of claim 8 wherein the rules of the dialog model are hierarchical.

21. The computer readable medium of claim 20 wherein at least some of the rules pertain to sub-dialogs.

22. The computer readable medium of claim 8 wherein information of the domain is hierarchically organized and classified into portions, wherein at least some of the portions include a plurality of properties about the information of the portions.

23. A computer readable medium having instructions, which when executed on a computer provide a user interface, the instructions comprising:

a user input module adapted to receive input from a user and provide semantic information as a function of the input;

a dialog model defining a set of rules, each rule comprising an action to be performed as function of received input pertaining to semantic information and a dialog state with the user;

a discourse model defining a current dialog state with the user as a function of received input pertaining to semantic information;

a domain model defining information to be rendered to the user; and

a module receiving the semantic information and corresponding input and accessing the discourse model, the dialog model and the domain model to ascertain the current dialog state and perform an action to render selected information to the user.

24. The computer readable medium of claim 23 wherein the user input module comprises a speech recognizer and a parser.

25. The computer readable medium of claim 23 and further comprising a task model defining some of the information to be rendered to user as a function of a topic, and wherein the module accesses the task model to ascertain information to be rendered if a rule of the dialog model is not being used.

26. The computer readable medium of claim 25 wherein the task model is organized in a hierarchical manner wherein at least some tasks comprise a plurality of sub-tasks related to actions for rendering information, and wherein the module sequentially renders information defined by the sub-tasks in order to complete a larger task.

27. The computer readable medium of claim 26 wherein the module suspends rendering information defined by a task in the task model when using a rule of the dialog model, and wherein the module resumes rendering information defined by the task when at least some of the rules of the dialog model do not apply.

28. The computer readable medium of claim 27 wherein each sub-task comprises at least one of an action for rendering information and another sub-task.

29. The computer readable medium of claim 23 and further comprising a user model defining user preferences related to rendering information, the module accessing the user model and rendering information as a function of user preferences.

30. The computer readable medium of claim 29 wherein module adjusts the user preferences as a function of the dialog with the user.

31. The computer readable medium of claim 30 wherein information to be rendered is prioritized and wherein one of the user preferences indicates a level of priority of information to be rendered.

32. The computer readable medium of claim 31 wherein the module adjusts the user preference related to the level of priority as a function of interruptions made by the user.

33. The computer readable medium of claim 30 wherein the information to be rendered includes stories, and wherein one of the user preferences relates to a frequency of stories to be rendered.

34. The computer readable medium of claim 33 wherein the module adjusts the user preference related to the frequency of stories to be rendered as a function of at least one of responses received from the user as to whether a story should be rendered and interruptions made by the user during telling a story.

35. The computer readable medium of claim 30 wherein at least some of the actions comprise rendering information to provoke a question from the user, and wherein one of the user preferences relates to a frequency for rendering information to provoke a question from the user.

36. The computer readable medium of claim 23 wherein the semantic information of at least some of the rules comprises a plurality of semantic types.

37. The computer readable medium of claim 36 wherein the rules of the dialog model are hierarchical.

38. The computer readable medium of claim 37 wherein at least some of the rules pertain to sub-dialogs.

39. The computer readable medium of claim 23 wherein information of the domain is hierarchically organized and classified into portions, wherein at least some of the portions include a plurality of properties about the information of the portions.

40. The computer readable medium of claim 39 and further comprising:

a store having data related to the properties of the portions;

a speech synthesizer; and

wherein the module accesses the store as a function of access of the domain model to obtain data to be provided to the speech synthesizer for speech synthesis.

41. The computer readable medium of claim 40 and further comprising:

a video rendering module receiving information related to the data, the video rendering module rendering a representation of a talking head having a talking state with mouth movements in accordance with an audio output signal from the speech synthesizer.

42. The computer readable medium of claim 41 wherein the video rendering module renders the representation of the talking head in a waiting state with movements in accordance with listening.

43. The computer readable medium of claim 42 wherein the video rendering module renders a sequence of video frames having the talking head.

44. The computer readable medium of claim 43 wherein the video rendering module continuously renders the video frames having the talking head with non-talking mouth movements during the waiting state and adds a talking mouth position to each of the frames during the talking state.