Service Orchestration for Intelligent Automated Assistant

• US 20130111487A1
• Filed: 12/21/2012
• Published: 05/02/2013
• Est. Priority Date: 01/18/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cheyer, Adam John, Guzzoni, Didier Rene, Gruber, Thomas Robert, Brigham, Christopher Dean
• Granted Patent: US 8,892,446 B2
• Time in Patent Office: Days
• US Class Current: 718/102
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/041 Abduction
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 : Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 21/06 : Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04M 1/6091 : including a wireless interface
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72484 : wherein functions are triggered by incoming communication events

Abstract

Methods, systems, and computer readable storage medium related to operating an intelligent digital assistant are disclosed. A user request is received, the user request including at least a speech input received from the user. The user request is processed to obtain a representation of user intent, where the representation of user intent associates the user request with a task flow operationalizing a requested task, and the task flow is operable to invoke a plurality of services each supporting functions according to a respective plurality of service parameters. Based on the representation of user intent, one or more relevant task parameters are identified from a plurality of task parameters of the task flow. A subset of the plurality of services are selectively invoked during execution of the task flow, where the selectively invoked subset of the plurality of services support functions according to the identified one or more relevant task parameters.

Claims

1. A method for operating an intelligent automated assistant, comprising:

at an electronic device comprising a processor and memory storing instructions for execution by the processor;

receiving a user request, the user request including at least a speech input received from the user;

processing the user request to obtain a representation of user intent, wherein the representation of user intent associates the user request with a task flow operationalizing a requested task, and the task flow is operable to invoke a plurality of services each supporting functions according to a respective plurality of service parameters;

based on the representation of user intent, identifying one or more relevant task parameters from a plurality of task parameters of the task flow; and

selectively invoking a subset of the plurality of services during execution of the task flow, wherein the selectively invoked subset of the plurality of services support functions according to the identified one or more relevant task parameters.

2. The method of claim 1, further comprising:

transforming at least one task parameter according to respective APIs of multiple services of the selected subset of the plurality of services.

3. The method of claim 1, further comprising:

gathering service results from the selected subset of the plurality of services; and

merging the service results into a unified result in a service-independent format to fulfill the user request.

4. The method of claim 1, further comprising:

assessing relative ability of multiple subsets of services from the plurality of services to satisfy the user request, each of the multiple subsets operable to provide functions according to the identified one or more relevant task parameters; and

based on the assessment, selecting one of the multiple subsets of services for invocation during the execution of the task flow.

5. The method of claim 1, wherein each service is associated with a respective service capability model comprising declarative descriptions of respective capabilities and properties of the service.

6. The method of claim 5, wherein selectively invoking the subset of the plurality of services further comprises:

dynamically selecting the subset of the plurality of services for invocation based on the respective service capability models of the plurality of services.

7. The method of claim 1, further comprising:

generating at least one merged result based on results received from the selectively invoked subset of services; and

selectively invoking a second plurality of services to annotate the at least one merged result.

8. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the processors to perform operations comprising;

receiving a user request, the user request including at least a speech input received from the user;

processing the user request to obtain a representation of user intent, wherein the representation of user intent associates the user request with a task flow operationalizing a requested task, and the task flow is operable to invoke a plurality of services each supporting functions according to a respective plurality of service parameters;

based on the representation of user intent, identifying one or more relevant task parameters from a plurality of task parameters of the task flow; and

selectively invoking a subset of the plurality of services during execution of the task flow, wherein the selectively invoked subset of the plurality of services support functions according to the identified one or more relevant task parameters.

9. The system of claim 8, wherein the operations further comprise:

transforming at least one task parameter according to respective APIs of multiple services of the selected subset of the plurality of services.

10. The system of claim 8, wherein the operations further comprise:

gathering service results from the selected subset of the plurality of services; and

merging the service results into a unified result in a service-independent format to fulfill the user request.

11. The system of claim 8, wherein the operations further comprise:

assessing relative ability of multiple subsets of services from the plurality of services to satisfy the user request, each of the multiple subsets operable to provide functions according to the identified one or more relevant task parameters; and

based on the assessment, selecting one of the multiple subsets of services for invocation during the execution of the task flow.

12. The system of claim 8, wherein each service is associated with a respective service capability model comprising declarative descriptions of respective capabilities and proper-ties of the service.

13. The system of claim 12, wherein selectively invoking the subset of the plurality of services further comprises:

dynamically selecting the subset of the plurality of services for invocation based on the respective service capability models of the plurality of services.

14. The system of claim 8, wherein the operations further comprise:

generating at least one merged result based on results received from the selectively invoked subset of services; and

selectively invoking a second plurality of services to annotate the at least one merged result.

15. A non-transitory computer readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

receiving a user request, the user request including at least a speech input received from the user;

processing the user request to obtain a representation of user intent, wherein the representation of user intent associates the user request with a task flow operationalizing a requested task, and the task flow is operable to invoke a plurality of services each supporting functions according to a respective plurality of service parameters;

based on the representation of user intent, identifying one or more relevant task parameters from a plurality of task parameters of the task flow; and

selectively invoking a subset of the plurality of services during execution of the task flow, wherein the selectively invoked subset of the plurality of services support functions according to the identified one or more relevant task parameters.

16. The computer readable storage medium of claim 15, wherein the operations further comprise:

transforming at least one task parameter according to respective APIs of multiple services of the selected subset of the plurality of services.

17. The computer readable storage medium of claim 15, wherein the operations further comprise:

gathering service results from the selected subset of the plurality of services; and

merging the service results into a unified result in a service-independent format to fulfill the user request.

18. The computer readable storage medium of claim 15, wherein the operations further comprise:

assessing relative ability of multiple subsets of services from the plurality of services to satisfy the user request, each of the multiple subsets operable to provide functions according to the identified one or more relevant task parameters; and

based on the assessment, selecting one of the multiple subsets of services for invocation during the execution of the task flow.

19. The computer readable storage medium of claim 15, wherein each service is associated with a respective service capability model comprising declarative descriptions of respective capabilities and properties of the service.

20. The computer readable storage medium of claim 19, wherein selectively invoking the subset of the plurality of services further comprises:

dynamically selecting the subset of the plurality of services for invocation based on the respective service capability models of the plurality of services.

21. The computer readable storage medium of claim 15, wherein the operations further comprise:

generating at least one merged result based on results received from the selectively invoked subset of services; and

selectively invoking a second plurality of services to annotate the at least one merged result.