Conversational computing via conversational virtual machine
First Claim
1. A method for providing conversational computing between a user and at least one application, the method comprising the steps of:
- engaging in dialog with the user;
processing the dialog, by a processor executing the at least one application, to one of complete a query, disambiguate a query, summarize a query, correct a query, correct a result of an executed task, communicate the result of such execution, determine a target application of an input/output event, and a combination thereof based on one of past dialog history, context, user preferences, meta information, and a combination thereof; and
presenting a unified and coordinated user interface across the plurality of applications;
wherein presenting a unified and coordinated user interface across a plurality of applications comprises the steps of;
registering, by each application, information comprising application state, application modes, arguments, context, modalities, engine resources, and a combination thereof; and
managing the dialog across the plurality of applications based on their registered information.
2 Assignments
0 Petitions
Accused Products
Abstract
A conversational computing system that provides a universal coordinated multi-modal conversational user interface (CUI) 10 across a plurality of conversationally aware applications (11) (i.e., applications that “speak” conversational protocols) and conventional applications (12). The conversationally aware applications (11) communicate with a conversational kernel (14) via conversational application APIs (13). The conversational kernel 14 controls the dialog across applications and devices (local and networked) on the basis of their registered conversational capabilities and requirements and provides a unified conversational user interface and conversational services and behaviors. The conversational computing system may be built on top of a conventional operating system and APIs (15) and conventional device hardware (16). The conversational kernel (14) handles all I/O processing and controls conversational engines (18). The conversational kernel (14) converts voice requests into queries and converts outputs and results into spoken messages using conversational engines (18) and conversational arguments (17). The conversational application API (13) conveys all the information for the conversational kernel (14) to transform queries into application calls and conversely convert output into speech, appropriately sorted before being provided to the user.
-
Citations
36 Claims
-
1. A method for providing conversational computing between a user and at least one application, the method comprising the steps of:
-
engaging in dialog with the user; processing the dialog, by a processor executing the at least one application, to one of complete a query, disambiguate a query, summarize a query, correct a query, correct a result of an executed task, communicate the result of such execution, determine a target application of an input/output event, and a combination thereof based on one of past dialog history, context, user preferences, meta information, and a combination thereof; and presenting a unified and coordinated user interface across the plurality of applications; wherein presenting a unified and coordinated user interface across a plurality of applications comprises the steps of; registering, by each application, information comprising application state, application modes, arguments, context, modalities, engine resources, and a combination thereof; and managing the dialog across the plurality of applications based on their registered information.
-
-
2. The method of claim 1, wherein the step of engaging in dialog comprises engaging in multi-modal dialog.
-
3. The method of claim 1, wherein the step of engaging in dialog comprises engaging in mixed-initiative dialog.
-
4. The method of claim 1, wherein the step of engaging in dialog comprises engaging in natural language dialog.
-
5. The method of claim 1, wherein the step of processing the dialog comprises deriving a user-specified action from the dialog and dispatching the user-specified action.
-
6. The method of claim 1, further comprising the step of adapting the rendering of an application to one of a registered dialog modality, device capability and channel capability.
-
7. The method of claim 1, further comprising the step of maintaining the context of each active task.
-
8. The method of claim 7, wherein the context comprises one of short term history, long term history, and both short-term and long-term history.
-
9. The method of claim 7, wherein the step of maintaining the context comprises maintaining one of query arguments, I/O events, application state, conversational arguments, application events, and a combination thereof, associated with the active task.
-
10. The method of claim 1, wherein the step of presenting a unified and coordinated user interface across the plurality of applications comprises coordinating a plurality of tasks spawned one of locally and remotely.
-
11. The method of claim 1, wherein the step of presenting a unified and coordinated user interface across the plurality of applications comprises dynamically negotiating a network topology among devices executing the applications based on registered information of the applications.
-
12. The method of claim 11, wherein the step of dynamically negotiating a topology comprises the step of repeatedly renegotiating the network topology among the devices executing the applications during a dialog session based on currently registered application states and resource requirements and capabilities.
-
13. The method of claim 1, further comprising the steps of arbitrating a response to an input query from each application to determine the target application.
-
14. The method of claim 13, further comprising the steps of one of prompting the user for additional input, checking for user confirmation, clarifying an interpretation of user intent, dispatching a user action that is derived from user input, and a combination thereof, after determining the target application.
-
15. The method of claim 1, wherein the step of managing the dialog comprises the step of coordinating the dialog through one of a master/slave, client/server, and peer-to-peer topology among the applications.
-
16. The method of claim 1, wherein the step of presenting a unified and coordinated user interface across the plurality of applications comprises managing the dialog through one of a centralized and distributed context.
-
17. The method of claim 1, wherein the step of presenting a unified and coordinated user interface across the plurality of applications comprises the step of discovering conversationally aware applications and systems.
-
18. The method of claim 1, further comprising the step of abstracting one of a user query, context, manipulated information, and a combination thereof, based on one of contexts, applications, history, user preferences, biometrics, and a combination thereof.
-
19. The method of claim 1, further comprising the steps of:
-
associating abstract meta-information with elements, wherein the elements comprises one of objects, files, applications, data stream handles, networks, peripherals, hardware, a local file system, a remote file system, dispatched tasks, links, macros, shortcuts, bookmarks, user preferences, user habits and behavior, biometrics, a-priori knowledge acquired, acquired user knowledge, learned user knowledge, and a combination thereof; and maintaining the abstract meta information of the elements.
-
-
20. The method of claim 19, wherein the step of associating abstract meta information comprises one of registering an abstract meta information category of an elements creating a new abstract meta information category of an element, and both.
-
21. The method of claim 19, further comprising the step of searching for a target element using the abstract meta information.
-
22. The method of claim 21, wherein the step of searching comprises searching the categories through multi-modal conversational dialog that comprises filling sets of attribute values out of set of possible types of queries and refining or modifying the result of current search queries based on similarities of the associated categories.
-
23. The method of claim 19, further comprising the step of selecting a desired element using the abstract meta information.
-
24. The method of claim 23, wherein the step of selecting comprises selecting the categories through multi-modal conversational dialog that comprises filling sets of attribute values out of set of possible types of queries and refining or modifying the result of current selection queries based on similarities of the associated categories.
-
25. The method of claim 19, wherein the meta-information comprises security meta-information, further comprising the step of automatically authenticating a user when a query to a restricted element is made, based on the security meta-information associated with the element.
-
26. The method of claim 1, further comprising the steps of coordinating and prioritizing output generation based on one of an active dialog, history, and a combination thereof.
-
27. The method of claim 1, further comprising the step of:
- memorizing an event, wherein the event comprises one of a user command, a user preference, an I/O event, and results of an executed task; and
subsequently activating the memorized event at a desired time during the dialog.
- memorizing an event, wherein the event comprises one of a user command, a user preference, an I/O event, and results of an executed task; and
-
28. The method of claim 27, wherein the step of activating comprises activating the memorized event for presentation to the user.
-
29. The method of claim 27, wherein the step of activating comprises activating the memorized event to resume a memorized dialog commencing from a state in which it was memorized.
-
30. The method of claim 27, wherein the memorized event is used to undo the effect and context impact of the memorized dialog.
-
31. The method of claim 27, wherein the memorized event is activated prior to dispatching an agent or executing a task.
-
32. The method of claim 27, wherein the step of memorizing comprises storing a current context, at the time of memorization of the event, as meta-information.
-
33. The method of claim 27, further comprising the steps of:
- summarizing the memorized event; and
activating the event summarization for presentation of the event summarization to the user.
- summarizing the memorized event; and
-
34. The method of claim 33, wherein the step of activating the event summarization is performed by one of a user query and a triggering event specified by an application.
-
35. The method of claim 1, further comprising the step of customizing the dialog based on one of user identity, usage history of the user, user preferences, active applications, context, and a combination.
-
36. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for providing conversational computing between a user and at least one application, the method steps comprising:
-
engaging in dialog with the user; processing the dialog to one of complete a query, disambiguate a query, summarize a query, correct a query, correct a result of an executed task, communicate the result of such execution, determine a target application of an input/output event, and a combination thereof, based on one of past dialog history, context, user preferences, meta information, and a combination thereof; and presenting a unified and coordinated user interface across the plurality of applications; wherein presenting a unified and coordinated user interface across a plurality of applications comprises the steps of; registering, by each application, information comprising application state, application modes, arguments, context, modalities, engine resources, and a combination thereof; and managing the dialog across the plurality of applications based on their registered information.
-
Specification