Intelligent Automated Assistant

• US 20120245944A1
• Filed: 06/09/2012
• Published: 09/27/2012
• Est. Priority Date: 01/18/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gruber, Thomas Robert, Cheyer, Adam John, Kittlaus, Dag, Guzzoni, Didier Rene, Brigham, Christopher Dean, Giuli, Richard Donald, Bastea-Forte, Marcello, Saddler, Harry Joseph
• Granted Patent: US 9,548,050 B2
• Time in Patent Office: Days
• US Class Current: 704/270.1
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06F 9/54 Interprogram communication
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/041 Abduction
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/08 Text analysis or generation of parameters for speech synthesis out of text, e.g. grapheme to phoneme translation, prosody generation or stress or intonation determination
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 : Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 21/06 : Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence
• CPC Class Codes: H04M 1/6091 : including a wireless interface
• CPC Class Codes: H04M 1/7243 : with interactive means for internal management of messages
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 1/72484 : wherein functions are triggered by incoming communication events

Abstract

The intelligent automated assistant system engages with the user in an integrated, conversational manner using natural language dialog, and invokes external services when appropriate to obtain information or perform various actions. The system can be implemented using any of a number of different platforms, such as the web, email, smartphone, and the like, or any combination thereof. In one embodiment, the system is based on sets of interrelated domains and tasks, and employs additional functionally powered by external services with which the system can interact.

Claims

1. A method for launching an application on a user device using a digital assistant, comprising:

at an electronic device comprising a processor and memory storing instructions for execution by the processor;

providing, at the user device, an at least partially speech-based conversational interface for interacting with the user;

obtaining context information associated with an interaction between the user and the user device;

receiving a speech input from the user through the conversational interface;

processing the speech input to determine a user intent associated with the speech input; and

upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device;

invoking the software application on the user device outside of the conversational interface; and

providing the context information to the software application.

2. The method of claim 1, wherein obtaining the context information further comprises:

receiving an additional speech input through the conversational interface prior to receiving the speech input; and

processing the additional speech input to obtain the context information.

3. The method of claim 1, wherein processing the speech input to determine the user intent associated with the speech input further comprises:

disambiguating the speech input using the context information.

4. The method of claim 1, further comprising:

receiving an additional speech input from the user through the conversational interface;

processing the additional speech input to determine an additional user intent associated with the additional speech input; and

based on the additional user intent that has been determined;

obtaining another input related to the additional user intent from the user through the conversational interface; and

executing a task for fulfilling the additional user intent within the conversational interface.

5. The method of claim 1, further comprising:

receiving an additional speech input from the user through the conversational interface;

processing the additional speech input to determine an additional user intent associated with the additional speech input;

upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device;

invoking the additional software application on the user device outside of the conversational interface.

6. The method of claim 1, wherein obtaining the context information further comprises:

receiving non-speech input from the user prior to the speech input; and

processing the non-speech input to obtain the context information.

7. A system, comprising:

one or more processors;

memory storing instructions, the instructions configured to be executed by the one or more processors and cause the one or more processors to perform operations comprising;

at an electronic device comprising a processor and memory storing instructions for execution by the processor;

providing, at the user device, an at least partially speech-based conversational interface for interacting with the user;

obtaining context information associated with an interaction between the user and the user device;

receiving a speech input from the user through the conversational interface;

processing the speech input to determine a user intent associated with the speech input; and

upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device;

invoking the software application on the user device outside of the conversational interface; and

providing the context information to the software application.

8. The system of claim 7, wherein obtaining the context information further comprises:

receiving an additional speech input through the conversational interface prior to receiving the speech input; and

processing the additional speech input to obtain the context information.

9. The system of claim 7, wherein processing the speech input to determine the user intent associated with the speech input further comprises:

disambiguating the speech input using the context information.

10. The system of claim 7, wherein the operations further comprise:

receiving an additional speech input from the user through the conversational interface;

processing the additional speech input to determine an additional user intent associated with the additional speech input; and

based on the additional user intent that has been determined;

obtaining another input related to the additional user intent from the user through the conversational interface; and

executing a task for fulfilling the additional user intent within the conversational interface.

11. The system of claim 7, wherein the operations further comprise:

receiving an additional speech input from the user through the conversational interface;

processing the additional speech input to determine an additional user intent associated with the additional speech input;

upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device;

invoking the additional software application on the user device outside of the conversational interface.

12. The system of claim 7, wherein obtaining the context information further comprises:

receiving non-speech input from the user prior to the speech input; and

processing the non-speech input to obtain the context information.

13. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device, cause the device to:

provide, at the user device, an at least partially speech-based conversational interface for interacting with the user;

obtain context information associated with an interaction between the user and the user device;

receive a speech input from the user through the conversational interface;

process the speech input to determine a user intent associated with the speech input; and

upon determination that the user intent associated with the speech input is for invoking a software application installed on the user device;

invoke the software application on the user device outside of the conversational interface; and

provide the context information to the software application.

14. The computer readable storage medium of claim 13, wherein the instructions further cause the device to:

receive an additional speech input through the conversational interface prior to receiving the speech input; and

process the additional speech input to obtain the context information.

15. The computer readable storage medium of claim 13, wherein processing the speech input to determine the user intent associated with the speech input further comprises:

disambiguating the speech input using the context information.

16. The computer readable storage medium of claim 13, wherein the instructions further cause the device to:

receive an additional speech input from the user through the conversational interface;

process the additional speech input to determine an additional user intent associated with the additional speech input; and

based on the additional user intent that has been determined;

obtain another input related to the additional user intent from the user through the conversational interface; and

execute a task for fulfilling the additional user intent within the conversational interface.

17. The computer readable storage medium of claim 13, wherein the instructions further cause the device to:

receive an additional speech input from the user through the conversational interface;

process the additional speech input to determine an additional user intent associated with the additional speech input;

upon determination that the additional user intent associated with the additional speech input is for invoking an additional software application installed on the user device;

invoke the additional software application on the user device outside of the conversational interface.

18. The computer readable storage medium of claim 13, wherein obtaining the context information further comprises:

receiving non-speech input from the user prior to the speech input; and

processing the non-speech input to obtain the context information.