Providing virtual personal assistance with multiple VPA applications

• US 10,204,627 B2
• Filed: 08/14/2013
• Issued: 02/12/2019
• Est. Priority Date: 04/16/2013
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Nitz, Kenneth C., Lincoln, Patrick D.
• Primary Examiner(s): Roberts, Shaun
• Application Number:
  US13/966,665
• Publication Number:
  US 20140310002A1
• Time in Patent Office: 2,008 Days
• Field of Search: 704 9, 704231, 704270, 7042701, 704257
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems

Abstract

The activities of multiple virtual personal assistant (VPA) applications are coordinated. For example, different portions of a conversational natural language dialog involving a user and a computing device may be handled by different VPAs.

Claims

1. A computer-implemented method, comprising:

using an input signal received by an input device, determining an input intent,

wherein the input signal comprises a dialog request,

wherein the input intent is determined at least in part by a first virtual assistant application of a first device;

based on the input intent, identifying a second virtual assistant application;

obtaining pairing information;

using the pairing information, establishing a communication connection between the first virtual assistant application and the second virtual assistant application,

wherein the second virtual assistant application is to fulfill the dialog request by generating an output intent or communicating the output intent to an output generator to cause a second device to present output that corresponds to the output intent,

wherein the output intent comprises a structured representation of a response to the input intent.

2. The computer-implemented method of claim 1, wherein the first and second virtual assistant applications determine input intents differently using different models.

3. The computer-implemented method of claim 1, wherein the second virtual assistant application modifies the input intent to produce a modified input intent, and the output intent comprises a structured representation of a response to the modified input intent.

4. The computer-implemented method of claim 1, wherein one of the first and second virtual assistant applications is a single user, multiple application virtual assistant and the other of the first and second virtual assistant applications is a multiple user, single application virtual assistant.

5. The computer-implemented method of claim 1, wherein using the pairing information enables the first virtual assistant application to communicate directly with the second virtual assistant application.

6. The computer-implemented method of claim 1, wherein the output intent is determined based on a model that is coupled to the second virtual assistant application, wherein the model is not accessible to the first virtual assistant application.

7. The computer-implemented method of claim 1, further comprising:

determining, based on the pairing information, that the first virtual assistant application has permission to communicate with the second virtual assistant application.

8. The computer-implemented method of claim 1, further comprising:

receiving, by the first virtual assistant application, a notification that the second virtual assistant has generated the output intent.

9. The computer-implemented method of claim 1, wherein establishing a communication connection between the first virtual assistant application and the second virtual assistant application protects communications sent over the communication connection from being accessible to other virtual assistant applications.

10. The computer-implemented method of claim 1, comprising, before establishing the communication connection, determining that both the first and second virtual assistant applications allow the communication connection to be established.

11. The computer-implemented method of claim 1, comprising identifying the second virtual assistant application based on a preference or a usage history of a user of the first virtual assistant application.

12. The computer-implemented method of claim 1, wherein the first and second virtual assistant applications are embodied in different devices.

13. A computing device, comprising:

one or more processors; and

a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including;

using an input signal received by an input device, determining an input intent,

wherein the input signal comprises a dialog request,

wherein the input intent is determined at least in part by a first virtual assistant application of a first device;

based on the input intent, identifying a second virtual assistant application;

obtaining pairing information;

using the pairing information, establishing a communication connection between the first virtual assistant application and the second virtual assistant application,

wherein the second virtual assistant application is to fulfill the dialog request by generating an output intent or communicating the output intent to an output generator to cause a second device to present output that corresponds to the output intent,

wherein the output intent comprises a structured representation of a response to the input intent.

14. The computing device of claim 13, wherein the first and second virtual assistant applications determine input intents differently using different models.

15. The computing device of claim 13, wherein the second virtual assistant application modifies the input intent to produce a modified input intent, and the output intent comprises a structured representation of a response to the modified input intent.

16. The computing device of claim 13, wherein one of the first and second virtual assistant applications is a single user, multiple application virtual assistant and the other of the first and second virtual assistant applications is a multiple user, single application virtual assistant.

17. The computing device of claim 13, wherein using the pairing information enables the first virtual assistant application to communicate directly with the second virtual assistant application.

18. The computing device of claim 13, wherein the output intent is determined based on a model that is coupled to the second virtual assistant application, wherein the model is not accessible to the first virtual assistant application.

19. The computing device of claim 13, further comprising:

determining, based on the pairing information, that the first virtual assistant application has permission to communicate with the second virtual assistant application.

20. The computing device of claim 13, further comprising:

receiving, by the first virtual assistant application, a notification that the second virtual assistant has generated the output intent.

21. The computing device of claim 13, wherein establishing a communication connection between the first virtual assistant application and the second virtual assistant application protects communications sent over the communication connection from being accessible to other virtual assistant applications.

22. The computing device of claim 13, comprising, before establishing the communication connection, determining that both the first and second virtual assistant applications allow the communication connection to be established.

23. The computing device of claim 13, comprising identifying the second virtual assistant application based on a user preference or a usage history of a user of the first virtual assistant application.

24. The computing device of claim 13, wherein the first and second virtual assistant applications are embodied in different devices.

25. A computer-program product embodied in a non-transitory machine-readable storage medium, including instructions that, when executed by one or more processors, cause the one or more processors to:

using an input signal received by an input device, determining an input intent,

wherein the input signal comprises a dialog request,

wherein the input intent is determined at least in part by a first virtual assistant application of a first device;

based on the input intent, identifying a second virtual assistant application;

obtaining pairing information;

using the pairing information, establishing a communication connection between the first virtual assistant application and the second virtual assistant application,

wherein the second virtual assistant application is to fulfill the dialog request by generating an output intent or communicating the output intent to an output generator to cause a second device to present output that corresponds to the output intent,

wherein the output intent comprises a structured representation of a response to the input intent.

26. The computer-program product of claim 25, wherein the first and second virtual assistant applications determine input intents differently using different models.

27. The computer-program product of claim 25, wherein the second virtual assistant application modifies the input intent to produce a modified input intent, and the output intent comprises a structured representation of a response to the modified input intent.

28. The computer-program product of claim 25, wherein one of the first and second virtual assistant applications is a single user, multiple application virtual assistant and the other of the first and second virtual assistant applications is a multiple user, single application virtual assistant.

29. The computer-program product of claim 25, wherein using the pairing information enables the first virtual assistant application to communicate directly with the second virtual assistant application.

30. The computer-program product of claim 25, wherein the output intent is determined based on a model that is coupled to the second virtual assistant application, wherein the model is not accessible to the first virtual assistant application.

31. The computer-program product of claim 25, further comprising:

determining, based on the pairing information, that the first virtual assistant application has permission to communicate with the second virtual assistant application.

32. The computer-program product of claim 25, further comprising:

receiving, by the first virtual assistant application, a notification that the second virtual assistant has generated the output intent.

33. The computer-program product of claim 25, wherein establishing a communication connection between the first virtual assistant application and the second virtual assistant application protects communications sent over the communication connection from being accessible to other virtual assistant applications.

34. The computer-program product of claim 25, comprising, before establishing the communication connection, determining that both the first and second virtual assistant applications allow the communication connection to be established.

35. The computer-program product of claim 25, comprising identifying the second virtual assistant application based on a user preference or a usage history of a user of the first virtual assistant application.

36. The computer-program product of claim 25, wherein the first and second virtual assistant applications are embodied in different devices.