Intelligent portal engine

• US 8,027,945 B1
• Filed: 08/14/2006
• Issued: 09/27/2011
• Est. Priority Date: 07/31/2000
• Status: Expired due to Term

• Current Assignee: Longhorn Automotive Group LLC (Alpha Alpha Intellectual Partners LLC)
• Original Assignee: Kalano Meui Hi LLC (Intellectual Ventures LLC)
• Inventors: Decker, Keith S., Philips, Irene H., Roper, Thomas A., Elad, Joseph B., Chester, Daniel L., Cleaver, David S., Johnson, Apperson H.
• Primary Examiner(s): Gaffin; Jeffrey A
• Assistant Examiner(s): Wong; Lut
• Application Number:
  US11/503,826
• Time in Patent Office: 1,870 Days
• Field of Search: 706/47, 706/12
• US Class Current: 706/47
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A human-computer interface system and methods for providing intelligent, adaptive, multimodal interaction with users while accomplishing tasks on their behalf in some particular domain or combination of domains. Specifically, this system accepts user input via natural language text, mouse actions, human speech, whistles, gestures, pedal movements, facial or postural changes, and conveys results via natural language text, automatically-generated speech, and displays of graphs, tables, animation, video, and mechanical and chemical effectors that convey heat, tactile sensation, taste and smell.

Claims

1. A method for performing at least one task using execution of a software component out of a plurality of software components on a computing device for at least one user involving at least one domain, the method comprising:

representing by the computing device at least one user goal with respect to the domain, the goal represented being independent of tasks which the plurality of software components can perform;

representing by the computing device at least one task which can be accomplished by the software component in the domain independent of stated user goals;

establishing by the computing device an association between at least one linguistic or pragmatic item and the at least one user goal, the at least one linguistic or pragmatic item having an association with the task;

inferring by the computing device using the associations that the software component which can accomplish the at least one task is appropriate out of the plurality of software components for achieving the at least one user goal; and

performing the at least one task for the user by the computing device executing the software component.

2. The method of claim 1, further comprising inferring by the computing device parameters of the software component, using the associations.

3. The method of claim 1, wherein the software component provides the at least one linguistic or pragmatic item corresponding with the task.

4. The method of claim 1, wherein said establishing an association between the at least one linguistic or pragmatic item and the at least one user goal comprises employing at least one model of the user with respect to the domain.

5. The method of claim 1, wherein inferring by the computing device using the associations that the software component which can accomplish the at least one task is appropriate comprises:

attempting to perform two or more tasks, using two or more software components, or using two or more sets of parameters;

rating feasibility of the two or more tasks and the software components and the parameters; and

ranking the software components or the parameters or the tasks by feasibility.

6. The method of claim 1, further comprising:

obtaining signals from the at least one user by the computing device; and

maintaining by the computing device, a user profile to supply default information or bindings; and

wherein the model of the user includes at least one perspective comprising;

expert, typical-user, clustered-user, personal-user, session, or focus; and

wherein defaults are maintained with respect to at least one perspective comprising expert, typical-user, clustered-user, personal-user, session, or focus.

7. The method of claim 1, further comprising:

maintaining by the computing device a history for establishing an association between the domain and user goals and between the domain and system tasks; and

predicting by the computing device a likely domain for association with a next user goal, using the associations in the history.

8. The method of claim 1, further comprising modeling domains, by the computing device, by combining models which represent sub-domains, including composing the linguistic or pragmatic items and the tasks associated with the domains by collecting those items and tasks from corresponding sub-domains.

9. The method of claim 1, further comprising inferring by the computing device, at least one task, one or more parameters, and the software component by inferring at least one item comprising user goals, intentions, beliefs, assumptions, preferences, or changes of state from a series of discourse transactions.

10. The method of claim 1, further comprising inferring by the computing device at least one user goal incrementally from elements of discourse including:

representing by the computing device, at least one terminological, ontological, lexical, grammatical, or idiomatic item of focusing information with respect to the at least one domain;

recording by the computing device, a history of transactions and the user goal associated with the transactions;

updating by the computing device, a representation of the user goals by increasing the likelihood of the user goals that are consistent with the transactions and by decreasing the likelihood of the user goals that are inconsistent with the transactions; and

generating by the computing device, a most likely meaning of subsequent user signals using the updated representations.

11. The method of claim 1, further comprising annotating by the computing device, a history of user inputs and system outputs of the computing device, with at least one annotation comprising:

a paraphrase of a user input,

an interpretation of a user input,

a set of working assumptions held by the computing device, or

changes of assumptions held by the computing device.

12. The method of claim 11, further comprising making the annotation available to the user by the computing device, and providing by the computing device a mechanism for the user to change the paraphrase, or the interpretation, or the working assumptions.

13. The method of claim 1, further comprising maintaining a history of user inputs or paraphrases of the inputs, by the computing device, and collecting by the computing device, a series of inputs or paraphrases for automatically forming at least one script.

14. The method of claim 1, further comprising:

maintaining by a computing device, a history of user inputs or paraphrases of the inputs;

collecting a series of inputs or paraphrases of the inputs by the computing device for automatically forming at least one script;

providing a mechanism to the user by the computing device for naming or designating the script; and

providing a mechanism by the computing device for invoking the script by name, or selection, or by at least one signal.

15. The method of claim 1, further comprising constructing by the computing device, a map of domain adjacency by:

maintaining by the computing device a history that reflects the domain associated with user inputs and with system outputs of the computing device;

counting by the computing device transitions from one domain to another; and

learning by the computing device, the likelihood of an interaction from one domain being followed by an interaction with the same domain or with a different domain.

16. The method of claim 1, further comprising providing a mechanism to the user, by the computing device, for choosing values of variables at a time of an invocation of at least one script or determining by the computing device values of variables from a user profile at the time of an invocation of the at least one script.

17. The method of claim 1, further comprising maintaining, by the computing device, a dynamically updated weight to reflect a change of transitions over time.

18. The method of claim 1, wherein the at least one user goal comprises intentions, beliefs, assumptions, preferences, or changes of state.

19. The method of claim 1, wherein the at least one linguistic or pragmatic item comprises vocabulary, grammar, idioms, parameters, goals, or assumptions.

20. A computer system, comprising:

means for representing at least one user goal with respect to a domain;

means for representing at least one task for a software component, which task can be accomplished in the domain independent of stated user goals;

means for establishing an association between at least one linguistic or pragmatic item and the user goal, the at least one linguistic or pragmatic item having an association with the at least one task;

means for inferring, using the associations, that the software component is appropriate for achieving the at least one user goal, using the associations; and

means for distributing the representation of the user goal, the representation of the task, the associations, an inference, or a performance of the task through execution of the software component among processing resources of the computer system.

21. The system of claim 20, further comprising means for inferring parameters for executing the software component to perform at least one task for the user, using the associations.

22. The system of claim 20, wherein the software component is configured to provide the at least one linguistic or pragmatic item corresponding with the task.

23. The system of claim 20, further comprising means for inferring the software component or one or more parameters for performing the task for the user including:

means for attempting to perform two or more tasks, using two or more sets of parameters;

means for rating feasibility of the tasks and the parameters; and

means for ranking the parameters or the tasks by feasibility to determine the parameters or task.

24. The system of claim 20, wherein the means for establishing an association between the at least one linguistic or pragmatic item and the at least one user goal comprises means for employing at least one model of the user with respect to the domain.

25. The system of claim 24, further comprising means for maintaining a user profile for supplying default information or bindings;

and

wherein;

the model of the user includes at least one perspective comprising expert, typical-user, clustered-user, personal-user, session, or focus; and

defaults are maintained with respect to at least one perspective comprising expert, typical-user, clustered-user, personal-user, session, or focus.

26. The system of claim 20, further comprising means for modeling domains by combining models which represent sub-domains, wherein the linguistic or pragmatic items and the tasks associated with the domains are composed by collecting items and tasks from corresponding sub-domains.

27. The system of claim 20, further comprising means for maintaining a history to associate the domain with user goals and with system tasks and means for using the associations in the history to predict a likely domain for association with a next user goal.

28. The system of claim 20, further comprising means for inferring at least one user goal incrementally from elements of discourse including:

means for representing at least one terminological, ontological, lexical, grammatical, or idiomatic item of focusing information with respect to the domain;

means for recording a history of transactions and the user goals associated with the transactions;

means for updating a representation of the user goals by increasing the likelihood of the user goals that are consistent with the transactions, and by decreasing the likelihood of the user goals that are inconsistent with the transactions; and

means for using the updated representation to generate the most likely meaning of subsequent user inputs.

29. The system of claim 20, further comprising means for annotating a history of the user inputs and system outputs with at least one annotation comprising:

a paraphrase of the user input,

an interpretation of the user input,

the set of working assumptions held by the system, or

changes of assumptions held by the system.

30. The system of claim 29, further comprising means for making the annotation available to the user, and means for providing a mechanism for the user to change the paraphrase, or the interpretation, or the working assumptions.

31. The system of claim 20, further comprising:

means for maintaining a history of user inputs or paraphrases of the inputs;

means for collecting a series of inputs or paraphrases of the inputs for automatically forming at least one script;

means for providing a mechanism to the user for naming or designating the script; and

means for providing a mechanism for invoking the script by name, or selection, or by at least one signal.

32. The system of claim 20, further comprising:

means for maintaining a history of user inputs or paraphrases;

means for collecting a series of inputs or paraphrases of the inputs for automatically forming at least one script; and

means for replacing one or more constants within the script with variables.

33. The system of claim 32 further comprising means for providing a mechanism to the user for choosing values of variables at the time of an invocation of the at least one script or means for determining values of variables from a user profile at the time of an invocation of the at least one script.

34. The system of claim 20, further comprising means for constructing a map of domain adjacency including:

means for maintaining a history that reflects the domain associated with user inputs and with system outputs;

means for counting the transitions from one domain to another; and

means for learning the likelihood of an interaction from one domain being followed by an interaction with the same domain or with a different domain.

35. The system of claim 20, further comprising means for maintaining a dynamically updated weight to reflect a change of transitions over time.

36. The computer system of claim 20, wherein:

the at least one user goal comprises intentions, beliefs, assumptions, preferences, and changes of state; and

the at least one linguistic or pragmatic item comprises vocabulary, grammar, idioms, parameters, goals, and assumptions.

37. An article of manufacture, comprising:

a storage medium; and

a plurality of programming instructions stored in the storage medium and configured to cause an apparatus, in response to execution of the instructions by an apparatus, to perform operations associated with inferring at least one meaning of a user'"'"'s actions with respect to a software component out of a plurality of software components for at least one user involving at least one domain, the operations comprising;

representing at least one user goal with respect to the domain, the goal represented independent of tasks which the plurality of software components can perform;

representing at least one task which can be accomplished by the software component in the domain independent of stated user goals;

establishing an association between at least one at least one linguistic or pragmatic item and a meaning of the user'"'"'s actions, the at least one linguistic or pragmatic item having an association with the task;

establishing an association between the at least one linguistic or pragmatic item and the user goal;

inferring, using the associations, that the software component which can accomplish the at least one task is appropriate out of the plurality of software components for achieving the at least one user goal; and

performing the at least one task by executing the software component using at least one association to infer the meaning of the user actions with respect to the software component.

38. The article of claim 37, wherein the software component provides the at least one linguistic or pragmatic item corresponding with the task.

39. The article of claim 37, wherein said establishing an association between the at least one linguistic or pragmatic item and the at least one user goal comprises employing at least one model of the user with respect to the domain.

40. The article of claim 39, wherein the operations further comprise conducting discourse transactions with the user, the conducting including:

receiving signals from the user through at least one communication channel; and

generating signals to the user through at least one communication channel;

wherein said establishing an association between the linguistic or pragmatic item and the user goal comprises employing a model of discourse transactions.

41. The article of claim 40, wherein the operations further comprise inferring at least one task, a parameter and the software component by inferring at least one item, the item comprising user goals, intentions, beliefs, assumptions, preferences, or changes of state, from a series of discourse transactions.

42. The article of claim 40, wherein the operations further comprise inferring at least one user goal incrementally from elements of discourse including:

representing at least one terminological, ontological, lexical, grammatical, or idiomatic item of focusing information with respect to the at least one domain;

recording a history of transactions and the user goals associated with the transactions;

updating a representation of the user goals by increasing the likelihood of the user goals that are consistent with the transactions and by decreasing the likelihood of the user goals that are inconsistent with the transactions; and

generating the most likely meaning of subsequent user actions using the updated representation.

43. The article of claim 39, wherein:

the operations further comprise maintaining a user profile to supply default information or bindings;

wherein;

the model of the user includes at least one perspective comprising expert, typical-user, clustered-user, personal-user, session, or focus; and

defaults are maintained with respect to at least one perspective comprising expert, typical-user, clustered-user, personal-user, session, or focus.

44. The article of claim 39, wherein the operations further comprise maintaining a history for reflecting the domain associated with the at least one user goal and the at least one task.

45. The article of claim 39, wherein the operations further comprise modeling domains by combining models which represent sub-domains, including composing the linguistic or pragmatic items and the tasks associated with the domains by collecting of those items and tasks from corresponding sub-domains.

46. The article of claim 39, wherein the operations further comprise constructing a map of domain adjacency by:

maintaining a history that reflects the domain associated with user actions and with system inferences;

counting the transitions from one domain to another; and

learning the likelihood of an interaction from one domain being followed by an interaction with the same domain or with a different domain.

47. The article of claim 46, wherein the operations further comprise inferring the software component or parameters to perform the task for the user by comparing the history of user actions and system inferences with a map of domain adjacency.

48. The article of claim 39, wherein operations further comprise obtaining a user action which has been accepted by some other software system, and is conveyed from the other software system as an information tuple, a file, as a stream of data, as a database entry, or as an electronic signal.

49. The article of claim 37, wherein the operations further comprise:

inferring the software component or one or more parameters for performing the task for the user by;

attempting to perform two or more tasks, using two or more software components, or using two or more sets of parameters;

rating the feasibility of the tasks and the software components and the parameters; and

ranking the software components or the parameters or the tasks by feasibility.

50. The article of manufacture of claim 37, wherein:

the user goal comprises intentions, beliefs, assumptions, preferences, or changes of state; and

the at least one linguistic or pragmatic item comprises vocabulary, grammar, idioms, parameters, goals, or assumptions.