APPLICATION INTEGRATION WITH A DIGITAL ASSISTANT

• US 20170358303A1
• Filed: 09/19/2016
• Published: 12/14/2017
• Est. Priority Date: 06/11/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: WALKER, II, Robert A., NEWENDORP, Brandon J., DASARI, Rohit, GIULI, Richard D., GRUBER, Thomas R., RADEBAUGH, Carey E., GARG, Ashish, KHOSLA, Vineet, RUSSELL, Jonathan H., PETERSON, Corey
• Granted Patent: US 10,297,253 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Systems and processes for application integration with a digital assistant are provided. In accordance with one example, a method includes, at an electronic device having one or more processors and memory, receiving a natural-language user input; identifying, with the one or more processors, an intent object of a set of intent objects and a parameter associated with the intent, where the intent object and the parameter are derived from the natural-language user input. The method further includes identifying a software application associated with the intent object of the set of intent objects; and providing the intent object and the parameter to the software application.

Claims

1. An electronic device, comprising:

one or more processors;

a memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

receiving a natural-language user input;

identifying, with the one or more processors, an intent object of a set of intent objects and a parameter associated with the intent object, wherein the intent object and the parameter are derived from the natural-language user input;

identifying a software application associated with the intent object of the set of intent objects; and

providing the intent object and the parameter to the software application.

2. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

receiving a response from the software application, wherein the response is indicative of whether the parameter is valid.

3. The electronic device according to claim 2, wherein receiving a response from the software application comprises:

receiving, from the software application, a disambiguation list associated with the parameter, wherein the disambiguation list includes a plurality of candidate parameters.

4. The electronic device according to claim 3, wherein the one or more programs further include instructions for:

outputting the disambiguation list;

receiving a user input indicative of a selection of a candidate parameter value of the disambiguation list; and

providing the selected candidate parameter value of the disambiguation list to the software application.

5. The electronic device according to claim 1, wherein the natural-language user input is a first natural-language input and the parameter is a first parameter, wherein the one or more programs further include instructions for:

receiving, from the software application, a request for a second parameter associated with the intent object;

providing a natural-language query based on the request;

receiving a second natural-language user input;

identifying the second parameter, wherein the second parameter is derived from the second natural-language user input; and

providing the second parameter to the software application.

6. The electronic device according to claim 5, wherein the electronic device further comprises an audio output component, and wherein providing the natural-language query comprises speaking, via speech synthesis, the natural-language query through the audio output component.

7. The electronic device according to claim 5, wherein providing the natural-language query comprises providing the natural-language query based on a type of the electronic device, a state of the electronic device, or a combination thereof.

8. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

after providing the intent object and the parameter to the software application, receiving, from the software application, a result response associated with the intent object.

9. The electronic device according to claim 8, wherein receiving a result response comprises:

receiving, from the software application, a set of response items associated with the intent object; and

outputting the set of response items.

10. The electronic device according to claim 9, wherein the one or more programs further include instructions for:

providing, to the software application, a set of view controller parameters; and

receiving, from the software application, a set of view controller parameter values corresponding to the set of view controller parameters,

wherein outputting the set of response items comprises displaying the set of response items in a user interface based on the received view controller parameter values.

11. The electronic device according to claim 1, wherein each intent object of the set of intent objects is associated with the software application.

12. The electronic device according to claim 1, wherein the intent object is a first intent object and wherein the parameter is associated with the first intent object and not associated with a second intent object.

13. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

determining whether the electronic device is in a lock state; and

in accordance with a determination that the device is in the lock state, determining whether the intent object is allowed to be provided to the software application while the electronic device is in the lock state,

wherein the intent object and the parameter value are provided to the software application only in accordance with a determination that the intent object is allowed to be provided to the software application while the electronic device is in the lock state.

14. The electronic device according to claim 1, wherein the parameter is derived from the natural-language user input by analyzing the natural-language user input using each of a plurality of detectors.

15. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

determining a user context of the electronic device;

wherein the parameter is based at least in part on the user context.

16. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

requesting a confirmation of the parameter; and

receiving a user input corresponding to the confirmation of the parameter, wherein the parameter is provided to the software application in response to receiving the user input.

17. The electronic device according to claim 1, wherein identifying the software application comprises:

determining whether the software application resides on the electronic device;

in accordance with a determination that the software application does not reside on the electronic device, determining whether the software application resides on an external device that is in communication with the electronic device; and

wherein providing the intent object and the parameter to the software application comprises;

in accordance with a determination that the software application resides on an external device that is in communication with the electronic device;

providing the intent object and the parameter to the software application by providing the intent object and the parameter to the external device.

18. The electronic device according to claim 1, wherein identifying the software application comprises:

determining whether the software application resides on the electronic device;

in accordance with a determination that the software application does not reside on the electronic device;

identifying a set of software applications associated with the intent object;

displaying, in a user interface, the set of software applications associated with the intent object;

receiving a user input including a selection of one or more software applications of the set of software applications associated with the intent object; and

accessing the selected one or more software applications of the set of software applications associated with the intent object.

19. The electronic device according to claim 1, wherein identifying a software application associated with the intent object of the set of intent objects comprises identifying the software application based on the parameter.

20. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

identifying, based on the natural-language user input, a third intent object from the set of intent objects;

identifying, based on the identified third intent object, a second software application;

providing the third intent object and at least one response item to the second software application; and

receiving, from the second software application, a second response item associated with the third intent object.

21. The electronic device according to claim 1, wherein the software application is a first software application, and wherein providing the intent object and the parameter to the software application comprises:

causing a third software application to provide the intent object and the parameter to the first software application.

22. The electronic device according to claim 1, wherein receiving a natural-language user input comprises:

receiving a natural-language user input including an application-specific term; and

wherein identifying a software application associated with the intent object of the set of intent objects comprises;

identifying the software application based on the application-specific term.

23. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

receiving a command including an identification of the software application; and

in response to the command, determining whether the software application is permitted to access data associated with the intent object.

24. The electronic device according to claim 1, wherein the one or more programs further include instructions for:

causing the software application to perform a task associated with the intent object based on the parameter.

25. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to:

receive a natural-language user input;

identify, with the one or more processors, an intent object of a set of intent objects and a parameter associated with the intent object, wherein the intent and the parameter are derived from the natural-language user input;

identify a software application associated with the intent object of the set of intent objects; and

provide the intent object and the parameter to the software application.

26. A method, comprising:

at an electronic device having one or more processors;

receiving a natural-language user input;

identifying, with the one or more processors, an intent object of a set of intent objects and a parameter associated with the intent object, wherein the intent object and the parameter are derived from the natural-language user input;

identifying a software application associated with the intent object of the set of intents objects; and

providing the intent object and the parameter to the software application.