Generic virtual personal assistant platform

• US 9,575,964 B2
• Filed: 06/30/2015
• Issued: 02/21/2017
• Est. Priority Date: 12/08/2011
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Yadgar, Osher, Yorke-Smith, Neil, Peintner, Bart, Tur, Gokhan, Ayan, Necip Fazil, Wolverton, Michael J., Acharya, Girish, Parimi, Venkatarama Satyanarayana, Mark, William S., Wang, Wen, Kathol, Andreas, Vincent, Regis, Franco, Horacio E.
• Primary Examiner(s): AZAD, ABUL K
• Application Number:
  US14/754,937
• Publication Number:
  US 20150302003A1
• Time in Patent Office: 602 Days
• Field of Search: 704/9, 704/257
• US Class Current: 1/1
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

A method for assisting a user with one or more desired tasks is disclosed. For example, an executable, generic language understanding module and an executable, generic task reasoning module are provided for execution in the computer processing system. A set of run-time specifications is provided to the generic language understanding module and the generic task reasoning module, comprising one or more models specific to a domain. A language input is then received from a user, an intention of the user is determined with respect to one or more desired tasks, and the user is assisted with the one or more desired tasks, in accordance with the intention of the user.

Claims

1. A method, using a computer processing system, for assisting a user with one or more desired tasks within a domain, the method comprising:

receiving, by a computing system comprising one or more computing devices, a language input from the user and at least personalization information input of the user;

determining, by the computing system, from the language input and the personalization information input, an intention of the user with respect to the one or more desired tasks, by an executable generic language understanding module and at least one run-time specification comprising a model configured to a specific field of use; and

performing, by the computing system, a domain-specific task in accordance with the intention of the user, by an executable generic task reasoning module and a run-time specification comprising a task flow configured to the specific field of use.

2. The method of claim 1, wherein the language input of the user comprises a portion of a dialog with the computer processing system having multiple rounds, and wherein the determining the intention of the user includes considering, during a given round of the dialog, personalization information comprising a previous intention of the user determined during one or more previous rounds of the dialog.

3. The method of claim 1, wherein the personalization information input comprises a second run time specification.

4. The method of claim 1, wherein the personalization information input comprises at least one of current or prior context information, records of frequently executed tasks in the task flow, times that the user is interacting with the system, and current or prior location information of the user.

5. The method of claim 1, wherein the personalization information input comprises a prioritization of common ones of the desired tasks invoked by the user.

6. The method of claim 1, wherein the user is unprompted for previously required secondary information based on the personalization information input.

7. The method of claim 6, wherein the personalization information input comprises at least contact information known to the system.

8. The method of claim 1, wherein the personalization information comprises at least one of personalized grammar of the user, and known geographic patterns of the user.

9. The method of claim 1, wherein the model configured to a specific field of use utilized by the generic language understanding module includes a language model.

10. The method of claim 9, wherein the language model comprises a grammar and an ontology.

11. The method of claim 1, wherein determining the intention of the user includes considering feedback from the generic task-reasoning module.

12. The method of claim 1, wherein the generic language understanding module comprises at least one of an utterance-level understanding component, and a context-level understanding component.

13. The method of claim 1, further comprising providing an executable speech recognition engine for execution on the computer processing system;

and wherein receiving the language input from the user includes receiving voice input from the user, and converting the voice input to text using the speech recognition engine.

14. The method of claim 1, wherein the language input comprises a natural language input.

15. The method of claim 1, wherein the personalization information input comprises a stored input.

16. The method of claim 1, further comprising receiving an additional input including at least one of:

a current geographical location of the user, an interaction context of the user within a web page, and an image input or a non-natural language command; and

wherein determining the intention of the user further includes considering the additional input.

17. The method of claim 1, wherein the desired tasks comprise at least one of:

content search;

content retrieval; and

transaction completion.

18. The method of claim 1, further comprising:

generating the model configured to a specific field of use via a computer-assisted development tool to convert a graphical representation of at least one task flow into the set of run-time specifications.

19. A virtual personal assistant platform embodied in one or more non-transitory computer-readable media, the virtual personal assistant platform configured to cause a computing system comprising one or more computing devices to:

receive, by a computing system comprising one or more computing devices, a language input from the user and at least personalization information input of the user;

determine, by the computing system, from the language input and the personalization information input, an intention of the user with respect to the one or more desired tasks, by an executable generic language understanding module and a run-time specification comprising a model configured to a specific field of use; and

perform, by the computing system, a domain-specific task in accordance with the intention of the user, by an executable generic task reasoning module and a run-time specification comprising a task flow configured to the specific field of use.

20. The platform of claim 19, comprising a sentence level understanding module configured to derive an understanding of a fragment of the language input.

21. The platform of claim 20, wherein the sentence level understanding module comprises a rule-based grammar module configured to parse the language input, and a statistical parser module configured to parse the language input if the rule-based grammar module fails to confidently parse the input.

22. The platform of claim 21, comprising an interpreter module configured to analyze a user intent argument output by the sentence level understanding module in a context of the personalization information input and output an interpreter intent argument.

23. The platform of claim 22, comprising a reasoner module configured to use the task flow configured to the specific field of use and the interpreter intent argument to determine a task to be performed by the computing system.

24. The platform of claim 23, wherein (i) the reasoner module is configured to output, to the interpreter module, a reasoner intent argument indicative of the determined task to be performed, and (ii) the interpreter module is configured to use the reasoner intent argument to analyze the user intent argument output by the sentence level understanding module.

25. The platform of claim 23, wherein the interpreter module is configured to compute a confidence level indicative of a level of certainty associated with the interpreter intent argument and output the confidence level to the reasoner module.

26. The platform of claim 23, wherein the reasoner module is configured to learn a preference of the user over time, and wherein the preference is stored to at least one database as a subsequent one of the personalization information input.

27. The platform of claim 26, wherein the reasoner module is configured to output a reasoner intent argument indicative of the determined task to be performed to a service module, and wherein the service module is configured to interface with a server to retrieve the subsequent one of the personalization information input from the at least one database.

28. The platform of claim 23, wherein the reasoner module is configured to:

(i) determine that a parameter needed to perform the task is missing from the language input and the personalization information input; and

(ii) cause the user to be prompted for the missing parameter.

29. A computing system for assisting a user with one or more desired tasks within a domain, the computing system comprising one or more computing devices configured to:

receive, by a computing system comprising one or more computing devices, a language input from the user and at least personalization information input of the user;

determine, by the computing system, from the language input and the personalization information input, an intention of the user with respect to the one or more desired tasks, by an executable generic language understanding module and a run-time specification comprising a domain-specific plug-in model configured to a specific field of use; and

perform, by the computing system, a domain-specific task in accordance with the intention of the user, by an executable generic task reasoning module and a run-time specification comprising a domain-specific plug-in task flow configured to the specific field of use.