Interaction framework for executing user instructions with online services

• US 10,248,452 B2
• Filed: 05/20/2016
• Issued: 04/02/2019
• Est. Priority Date: 05/20/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Portnoy, William Lawrence, Monroy-Hernandez, Andres
• Primary Examiner(s): Tang, Kenneth
• Application Number:
  US15/160,399
• Publication Number:
  US 20170337079A1
• Time in Patent Office: 1,047 Days
• Field of Search: None
• CPC Class Codes: G06F 9/48 Program initiating; Program switching, e.g. by interrupt
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/562 Brokering proxy services
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

In order to complete user instructions beyond the scope of services that are provided on the user'"'"'s own computing device, a framework an interaction framework for executing user instructions with online services is provided. The framework includes an instruction processing agent that interacts with a user agent on a computer user'"'"'s computing device to execute instructions provided to the user agent. The instruction processing agent maintains a list of domain agents, each domain agent corresponding to a domain. Upon receiving a user instruction (from a user agent), the instruction processing agent identifies a suitable domain and corresponding domain agent for completing the user instruction and provides the instruction to the domain agent. The domain agent identifies an online service suitable for completing the user instruction and interfaces with the online service via a proxy for the online service to carry out the user instruction.

Claims

1. A computer implemented framework for processing one or more user instructions on behalf of a computer user, the framework comprising:

a first computing device comprising a processor and memory, hosting an instruction processing agent, wherein the instruction processing agent, in execution on the first computing device, is configured to receive a user instruction from a user agent executing on a user computing device, and maintain a list of domain agents; and

one or more domain agent computing devices, the one or more domain agent computing devices hosting a plurality of domain agents wherein each domain agent, in execution on a domain agent computing device of the one or more domain agent computing devices, corresponds to a domain and is configured to receive a domain instruction from the instruction processing agent that can be carried out within the domain, and carry out the domain instruction on behalf of the computer user;

wherein, in execution, the instruction processing agent;

receives the user instruction from the user agent;

identifies a domain suitable for carrying out the user instruction based on an identified intent of the user instruction;

maps the user instruction into at least one domain instruction according to a domain ontology of the identified domain;

selects a domain agent of the plurality of domain agents, the selected domain agent corresponding to the identified domain; and

submits the user instruction to the selected domain agent for execution; and

wherein, in execution, the selected domain agent;

maintains a plurality of proxies for interfacing with a plurality of online services, wherein each of the plurality of proxies interfaces with one each of the plurality of online services and maps the domain instruction to a respective one of the one each of the plurality of online services;

receives the domain instruction from the instruction processing agent;

identifies an online service from the plurality of online services for completing the domain instruction; and

executes the domain instruction with the online service via the respective one of the plurality of proxies for the identified online service.

2. The computer implemented framework of claim 1, wherein the instruction processing agent, in identifying a domain and selecting a domain agent:

determines an intent of the computer user with regard to the user instruction; and

selects the domain agent according to a catalogue of domains, the catalogue of domains identifying a plurality of domains with a corresponding one or more corresponding domain agents of a domain.

3. The computer implemented framework of claim 2, wherein determining an intent of the computer user with regard to the user instruction comprises determining the intent according to an external service to the instruction processing agent.

4. The computer implemented framework of claim 2, wherein the instruction processing agent, in execution, further maps the user instruction to domain executable instructions according to a domain ontology of the selected domain.

5. The computer implemented framework of claim 2, wherein the instruction processing agent, in execution, further obtains a user context of the computer user, and submits the user context with the domain instruction to the selected domain agent for execution.

6. The computer implemented framework of claim 5, wherein the selected domain agent:

identifies a second plurality of online services for completing the domain instruction from the plurality of online services; and

executes the domain instruction with the identified second plurality of online services via a corresponding plurality of proxies of the identified second plurality of online services.

7. The computer implemented framework of claim 5, wherein the selected domain agent:

determines an execution context for computer user for executing the domain instruction; and

executes the domain instruction with the online service according to the execution context via a respective one of the plurality of proxies proxy of the identified online service.

8. The computer implemented framework of claim 7, wherein the domain agent further:

receives result information regarding the execution of the domain instruction from the online service via the respective one of the plurality of proxies; and

returns the result information to the instruction processing agent.

9. The computer implemented framework of claim 8, wherein the domain agent further updates the execution context for the computer user according to the received result information regarding the execution of the domain instruction.

10. A computer system providing an instruction processing agent service for processing user instructions on behalf of a computer user, the computing system comprising a processor and a memory, wherein the processor executes computer executable instructions obtained from memory to provide the instruction processing agent service, the computer system further comprising:

an executable instruction processing agent, wherein in execution the instruction processing agent;

maintains a list of domain agents;

receives a user instruction from a user agent on behalf of the computer user;

identifies a domain suitable for carrying out the user instruction based on an identified intent of the user instruction;

maps the user instruction into at least one domain instruction according to a domain ontology of the identified domain;

selects a domain agent of the plurality of domain agents according to a domain selector, the selected domain agent corresponding to the identified domain, wherein the selected domain agent maintains a plurality of proxies for interfacing with a plurality of online services, wherein each of the plurality of proxies interfaces with one each of the plurality of online services and maps the domain instruction to a respective one of the one each of the plurality of online services; and

submits the domain instruction to the selected domain agent for execution.

11. The computing system of claim 10 further comprising:

an executable transcription module that, in execution on the computing system, transcribes an audio user instruction into a common format; and

wherein the instruction processing agent further;

determines that the user instruction comprises an audio user instruction;

transcribes the user instruction to a common format via the transcription module.

12. The computing system of claim 10 further comprising:

an intent identification module that, in execution on the computing system, determines the intent of the user instruction; and

wherein the instruction processing agent selects the domain agent according to a catalogue of domains, the catalogue of domains identifying a plurality of domains with a corresponding one or more corresponding domain agents of a domain.

13. The computing system of claim 12, wherein the intent identification module utilizes an external service for determining the intent of the user instruction.

14. The computer implemented framework of claim 12 further comprising:

an instruction generator that, in execution on the computing system, maps the user instruction to the domain instruction according to a domain ontology of the identified domain; and

wherein submitting the user instruction to the selected domain agent for execution comprises submitting the domain instruction to the domain agent for execution.

15. The computer implemented framework of claim 12, wherein the instruction processing agent further:

determines a user context of the computer user; and

submits the user context with the domain instruction to the selected domain agent for execution.

16. The computer implemented framework of claim 12, wherein the instruction processing agent further:

receives response information from the selected domain agent; and

communicates the response information to the computer user via the user agent.

17. The computer implemented framework of claim 12, wherein the instruction processing agent further maps the user instruction to one or more domain executable instructions according to a domain ontology of the selected domain.

18. The computer implemented framework of claim 12, wherein the instruction processing agent:

selects a second plurality of domain agents from the plurality of domain agents according to a domain selector, the selected second plurality of domain agents corresponding the identified domain; and

submits the domain instruction to the selected second plurality of domain agents for execution.

19. A computer system providing a domain agent service for processing user instructions on behalf of a computer user, the computer system comprising a processor and a memory, wherein the processor executes computer executable instructions obtained from memory to provide the domain agent service, the computer system further comprising:

a plurality of online service proxies, each online service proxy configured to interface with a corresponding online service for executing user instructions with the online service on behalf of the domain agent;

wherein, in execution, the domain agent;

receives a user instruction from the instruction processing agent for execution;

identifies an online service from a plurality of online services for completing the user instruction based on an identified intent of the user instruction;

maps the user instruction into at least one domain instruction according to a domain ontology of the identified domain;

executes the domain instruction with the online service via a corresponding online service proxy of the plurality of online service proxies; and

returns result information regarding execution of the domain instruction to the instruction processing agent.

20. The computer system of claim 19, wherein the domain agent further:

determines an execution context for computer user for executing the domain instruction; and

executes the domain instruction with the online service according to the execution context via the corresponding proxy of the identified online service.