Using context information to facilitate processing of commands in a virtual assistant

• US 10,475,446 B2
• Filed: 06/12/2014
• Issued: 11/12/2019
• Est. Priority Date: 06/05/2009
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Gruber, Thomas R., Brigham, Christopher D., Keen, Daniel S., Novick, Gregory, Phipps, Benjamin S.
• Primary Examiner(s): Le, Thuykhanh
• Application Number:
  US14/303,413
• Publication Number:
  US 20140297284A1
• Time in Patent Office: 1,979 Days
• Field of Search: None
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models

Abstract

A virtual assistant uses context information to supplement natural language or gestural input from a user. Context helps to clarify the user'"'"'s intent and to reduce the number of candidate interpretations of the user'"'"'s input, and reduces the need for the user to provide excessive clarification input. Context can include any available information that is usable by the assistant to supplement explicit user input to constrain an information-processing problem and/or to personalize results. Context can be used to constrain solutions during various phases of processing, including, for example, speech recognition, natural language processing, task flow processing, and dialog generation.

Claims

1. A computer-implemented method for processing user input to perform a task on an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors, the method comprising:

receiving, at the electronic device, user input, the user input including a named relationship;

in response to receiving the user input;

displaying a first message corresponding to the user input that includes the named relationship;

identifying, at the electronic device, contact information based on an association between the named relationship and a contact associated with the user, wherein the association between the named relationship and the contact is locally available on the electronic device as a field of the contact; and

interpreting, using the one or more processors, the received user input to derive a representation of user intent; and

in response to identifying the contact information and deriving the representation of user intent;

displaying a second message indicative of at least one task to be performed by the electronic device, wherein the at least one task is based at least in part on the derived representation of user intent and the identified contact information; and

causing, using the one or more processors, to be performed the at least one task.

2. The computer-implemented method of claim 1, wherein the association between a named relationship and a contact is created by the user.

3. The computer-implemented method of claim 1, wherein the association between a named relationship and a contact is based on prior user input.

4. The computer-implemented method of claim 3, wherein the prior user input comprises user dialog history.

5. The computer-implemented method of claim 3, wherein the prior user input comprises a stated relationship.

6. The computer-implemented method of claim 1, wherein the contact information comprises a phone number and the at least one task includes calling the contact associated with the named relationship.

7. The computer-implemented method of claim 1, wherein the contact information comprises a phone number and the at least one task includes texting the contact associated with the named relationship.

8. The computer-implemented method of claim 1, wherein the contact information comprises an email address and the at least one task includes drafting an email to the contact associated with the named relationship.

9. The computer-implemented method of claim 1, wherein the named relationship comprises a family relationship.

10. The computer-implemented method of claim 1, wherein the named relationship comprises one of mother, father, sister, brother, or boss.

11. The computer-implemented method of claim 1, wherein the received user input is received speech data.

12. The computer-implemented method of claim 1, wherein the received user input is received text data.

13. The computer-implemented method of claim 1, wherein the received user input is received speech data and received text data.

14. A computer-implemented method for processing user input to perform a task on an electronic device with one or more processors and memory storing one or more programs for execution by the one or more processors, the method comprising:

receiving user input,

receiving, at the electronic device, a named relationship;

in response to receiving the user input and the named relationship;

displaying a first message corresponding to the user input that includes the named relationship;

identifying, at the electronic device, contact information based on an association between the named relationship and a contact associated with the user, wherein the association between the named relationship and the contact is locally available on the electronic device as a field of the contact; and

interpreting, using the one or more processors, the received user input to derive a user intent; and

in response to identifying the contact information and deriving the representation of user intent;

displaying a second message indicative of at least one action to be performed by the electronic device, wherein the at least one action is based at least in part on the derived representation of user intent and the identified contact information; and

causing, using the one or more processors, to be performed the at least one action.

15. The computer-implemented method of claim 14, wherein the contact information comprises a phone number and the at least one task includes calling the contact associated with the named relationship.

16. The computer-implemented method of claim 14, wherein the contact information comprises a phone number and the at least one task includes texting the contact associated with the named relationship.

17. The computer-implemented method of claim 14, wherein the contact information comprises an email address and the at least one task includes drafting an email to the contact associated with the named relationship.

18. The computer-implemented method of claim 14, wherein the named relationship comprises a family relationship.

19. An electronic device controllable by user inputs, comprising a processor, an input interface, and an output interface;

the processor operative to;

receive user input, the user input including a named relationship;

in response to receiving the user input;

display a first message corresponding to the user input that includes the named relationship;

identify, at the electronic device, contact information based on an association between the named relationship and a contact associated with the user, wherein the association between the named relationship and the contact is locally available on the electronic device as a field of the contact; and

interpret the received user input to derive a representation of user intent; and

in response to identifying the contact information and deriving the representation of user intent;

display a second message indicative of at least one task to be performed by the electronic device, wherein the at least one task is based at least in part on the derived representation of user intent and the identified contact information; and

cause the at least one task to be performed.

20. The electronic device of claim 19, wherein the contact information comprises a phone number and the at least one task includes calling the contact associated with the named relationship.

21. The electronic device of claim 19, wherein the contact information comprises a phone number and the at least one task includes texting the contact associated with the named relationship.

22. The electronic device of claim 19, wherein the contact information comprises an email address and the at least one task includes drafting an email to the contact associated with the named relationship.

23. The electronic device of claim 19, wherein the named relationship comprises a family relationship.

24. A non-transitory computer readable medium for processing user input, the computer readable media comprising computer program logic for:

receiving user input, the user input including a named relationship;

in response to receiving the user input;

displaying a first message corresponding to the user input that includes the named relationship;

identifying, at the electronic device, contact information based on an association between the named relationship and a contact associated with the user;

wherein the association between the named relationship and the contact is locally available on the electronic device as a field of the contact; and

interpreting the received user input to derive a representation of user intent; and

in response to identifying the contact information and deriving the representation of user intent;

displaying a second message indicative of at least one task to be performed by the electronic device, wherein the at least one task is based at least in part on the derived representation of user intent and the identified contact information; and

causing to be performed the at least one task.

25. The non-transitory computer readable medium of claim 24, wherein the contact information comprises a phone number and the at least one task includes calling the contact associated with the named relationship.

26. The non-transitory computer readable medium of claim 24, wherein the contact information comprises a phone number and the at least one task includes texting the contact associated with the named relationship.

27. A computer-implemented method for processing user input to perform a task on an electronic device with one or more processors, a microphone, and memory storing one or more programs for execution by the one or more processors, the method comprising:

receiving user input with the microphone, the user input including a named relationship and associated with at least one task;

in response to receiving the user input;

displaying a first message corresponding to the user input that includes the named relationship;

identifying, at the electronic device;

contact information based on an association between the named relationship and a contact associated with the user, wherein the association between the named relationship and the contact is locally available on the electronic device as a field of the contact; and

interpreting, using the one or more processors, the received user input to derive a representation of user intent; and

in response to identifying the contact information and deriving the representation of user intent;

displaying a second message indicative of the at least one task, wherein the at least one task is based at least in part on the derived representation of user intent and the identified contact information; and

causing, using the one or more processors, to be performed the at least one task.