Adaptive Virtual Intelligent Agent

• US 20160117593A1
• Filed: 10/27/2015
• Published: 04/28/2016
• Est. Priority Date: 11/20/2013
• Status: Active Grant

• Current Assignee: Justin London
• Original Assignee: Justin London
• Inventors: London, Justin
• Granted Patent: US 10,565,509 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/042 Knowledge-based neural networks; Logical representations of neural networks
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/016 After-sales
• CPC Class Codes: G06Q 30/0613 Third-party assisted
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Embodiments of an adaptive virtual intelligent agent (“AVIA”) service are disclosed. It may include the functions of a human administrative assistant for an enterprise including customer support, customer relationship management, and fielding incoming caller inquiries. It also has multi-modal applications for the home through interaction with AVIA implemented in the home. It may engage in free-form natural language dialogs. During a dialog, embodiments maintain the context and meaning of the ongoing dialog and provides information and services as needed by the domain of the application. Over time, the service automatically extends its knowledge of the domain (as represented in the Knowledge Tree Graphs) through interaction with external resources. Embodiments can intelligently understand and converse with users using free-form speech without pre-programmed deterministic sequences of questions and answers, can dynamically determine what it needs to know to converse meaningfully with users, and knows how to obtain information it needs.

Claims

1. A conversational system comprising at least one server coupled to a telephone system wherein:

the server comprises a context-and-meaning engine and a reasoning-and-inference engine;

the context-and-meaning engine is designed to configure a specific knowledge-tree graph instantiation derived from a knowledge-tree graph template using either at least one input received from an speech recognition engine in a caller session, or a type of entity associated with a call session;

said context-and-meaning engine is configured to define a type of relationship existing between said at least one input and one or more nodes in a specific knowledge-tree graph instantiation;

said context-and-meaning engine is configured to facilitate and transform caller inquiries and queries into structured meaning; and

said reasoning-and-inference engine is configured to trigger a method in said specific knowledge-tree graph instantiation in response to said at least one input.

2. A conversational system as claimed in claim 1 further comprising at least one data repository chosen from the group consisting of a customer-relationship-management (CRM) data repository, a business-specific data repository, and a remotely connected third-party data repository, wherein:

said CRM data repository stores a set of user preferences and a set of user data, and

said business-specific data repository includes one or more business-knowledge-items chosen from the group consisting of an employee directory, a product list, an inventory list, a pricing list, and a company'"'"'s profile further comprising an hours of operation list and a locations list; and

said context-and-meaning engine is further configured to instantiate said specific knowledge-tree graph instantiation by querying said at least one data repository.

3. A conversational system as claimed in claim 2 wherein said knowledge-tree graph template is chosen from the group consisting of a goods template, a services template or a combined template;

and wherein;

said goods template further includes a store template;

said services template further includes a restaurant template, a medical template, a legal template, and an office template;

said combined template further includes a hybrid of said goods template and said services template;

said store template includes a set of states for an inquiry and a method to check an item from an inventory stored in said business-specific data repository;

said restaurant template includes a set of states for an order placement and a method to order a menu item from a menu stored in said business-specific data repository;

said medical template includes a set of states about a patient and a method to schedule a medical procedure;

said legal template includes a set of states about a client and a method to determine a type of legal services required using said reasoning-and-inference engine and a second method to prompt said client for information regarding said type of legal services required;

said office template includes a set of states about a caller and a method to schedule an appointment in an electronic appointment book; and

said service template includes a set of states about service offerings offered by the business stored in a said business-specific data repository.

4. A conversational system as claimed in claim 1 further comprising an adaptive learning engine wherein, if a relationship between said at least one input and said knowledge-tree graph template cannot be determined by said context-and-meaning engine, said context-and-meaning engine is further configured to:

process said at least one input using a remotely connected third party data repository to determine at least one synonym for said at least one input;

send said at least one synonym to said context-and-meaning engine to determine if said relationship between said at least one input and said knowledge-tree graph template can be established; and

wherein said adaptive-learning-engine is configured to update lexicons and add said at least one input to a list of acceptable responses associated with a specific node in said knowledge-tree graph template.

5. A conversational system as claimed in claim 1 wherein said reasoning-and-inference engine is further configured to determine whether any additional information is required to complete said specific-knowledge-tree graph instantiation and, if so, said method iteratively generates a series of prompts to request each piece of missing information from a caller.

6. A conversational system as claimed in claim 1 further comprising one or more feature components chosen from the group consisting of a foreign language engine (FLE), a speech-to-text (STT) engine, a communication engine, and a voice analysis engine,

wherein said FLE is configured to:

receive said at least one input;

determine a language associated with said at least one input; and

translate each response by said conversational system to said language before sending it back to said caller;

wherein said STT is configured to;

produce a written transcript of said caller session; and

send said written transcript to said caller;

wherein said communication engine is configured to;

send one or more computer machine outputs chosen from the group consisting of confirmations, alerts, uniform resource locator (URL) links, menus, advertisements, and promotions to said caller; and

update a CRM data repository associated with said caller with each message sent to said caller; and

wherein said voice analysis engine is configured to

receive an original input to said conversational system;

match a caller'"'"'s voice to a voice data repository; and

retrieve a previously created specific knowledge-tree graph instantiation associated with said caller'"'"'s voice from a CRM data repository.

7. A conversational system as claimed in claim 6 wherein:

said reasoning-and-inference engine is further configured to automatically trigger a method associated with a state stored in said previously created specific knowledge-tree graph instantiation; and

said conversational system is an interactive voice response system.

8. A conversational system as claimed in claim 1 wherein said context-and-meaning engine is further configured to assign a weight to a relationship between said at least one input and each node to which it is related and wherein a higher weight is assigned to said at least one input if it has either previously occurred or has a high confidence match for contextual relevance to the node with which it is associated.

9. A conversational system as claimed in claim 8 further comprising an adaptive learning engine and

wherein said conversational system is further configured to analyze a plurality of specific knowledge-tree graph instantiations to determine if any of said at least one input are more common than others, and if a first input, from said at least one input, is more common, determine if said conversational system responded appropriately;

and

wherein said adaptive learning engine is configured to;

if said conversational system responded appropriately, increase the weight of said relationship between said at least one input and its node; and

if said conversational system did not respond appropriately, decrease the weight of said relationship between said at least one input and its node.

10. A conversational system comprising a server coupled to a telephone system wherein said server further comprises:

a speech processing system further comprising a speech biometric engine configured with a speaker identification routine;

a language identification routine, and an audio recording mechanism;

a reasoning-and-inference engine further comprising one or more agents chosen from the group consisting of an artificial intelligence engine, a Bayesian inference engine and a decision-making engine;

an adaptive learning engine further comprising a deep neural network learning engine; and

a plurality of data repositories including a data repository of phrase constructs, a customer-relationship management (CRM) data repository, and a business knowledge data repository, wherein;

said speech biometric engine identifies a caller in a call session;

if said caller is recognized, said reasoning-and-inference engine selects a knowledge-tree graph instantiation associated with said caller from said CRM data repository, and said reasoning-and-inference engine selects a prompt from said data repository of phrase constructs which has the highest likelihood, according to said agent, of being relevant to said call session based on a set of states in said caller'"'"'s knowledge-tree graph instantiation; and

if said caller is not recognized, said reasoning-and-inference engine selects a knowledge-tree graph template from said business knowledge data repository associated with a type of business called in said call session; and

said reasoning-and-inference engine selects a prompt, from said data repository of phrase constructs, which has the highest likelihood, using said agent, of being relevant to said call session based on a set of states in a plurality of knowledge tree graph instantiations stored in said CRM data repository.

11. A conversational system as claimed in claim 10 further comprising a natural language processing (NLP) engine;

a foreign language translation engine;

a language identification engine; and

a context-and-meaning engine (CME), wherein

said NLP engine is configured to receive and recognize an input;

said language identification engine is configured to determine a language to be associated with said recognized input; and

if said CME determines that said recognized input is not associated with a default language associated with said conversational system, said conversational system is configured to;

process any input through said foreign language translation engine to translate said input into said first language; and

process any output through said foreign language translation engine to translate said output into a second language before sending said output that has been translated to said caller.

12. A conversational system as claimed in claim 11 wherein said reasoning-and-inference engine is further configured to determine if said recognized input is a data item or a command and:

if said caller was not recognized, said CME is further configured to instantiate a knowledge-tree graph based on said knowledge-tree graph template;

if said recognized input is first data item, said CME is further configured to fill in said first data item in said knowledge-tree graph instantiation;

if said recognized input is a first command, said CME is further configured to execute a method, stored in said knowledge-tree graph instantiation, associated with said first command;

said reasoning-and-inference engine is further configured to determine if said method associated with said first command has a predetermined, sufficient number of data items filled in on said knowledge-tree graph instantiation to execute said method;

if said method can be executed, said conversational system executes said method; and

if said method cannot be executed, said reasoning-and-inference engine selects and delivers a prompt from said data repository of phrase constructs to elicit one or more missing data items prior to execution of said method.

13. A conversational system as claimed in claim 11 wherein said CME is further configured to perform historical context analysis to calculate if other callers are calling with regard to a specific reason by analyzing a plurality of knowledge-tree graph instantiations in said CRM data repository, across a plurality of callers, for a defined period of time, and if a predetermined threshold is reached increase a weight, associated with a node corresponding to that specific reason in said knowledge-tree graph instantiation.

14. A method of interactively responding to voice comprising:

receiving a voice-based input;

sending said voice-based input to a speech biometric engine to identify a speaker associated with said voice-based input;

if said speaker is recognized, selecting a knowledge-tree graph instantiation associated with said speaker from a customer-relationship management data repository;

if said speaker is not recognized, selecting a knowledge-tree graph template from a business knowledge data repository associated with a type of activity and creating a knowledge-tree graph instantiation which is associated with said speaker; and

prompting said speaker with a phrase construct with the highest likelihood of being relevant to said activity based on an analysis of one or more contexts chosen from the group consisting of said knowledge-tree graph instantiation associated with said speaker, and a plurality of knowledge-tree graph instantiations stored in said customer-relationship management data repository associated with said activity.

15. A method of interactively responding to voice as claimed in claim 14 further comprising, if said knowledge-tree graph instantiation is new, pre-populating a set of nodes in said knowledge-tree graph instantiation with, at least, a first data item from a business-specific data repository and generating a prompt to elicit a second data item to fill in an empty node in said knowledge-tree graph instantiation.

16. A method of interactively responding to voice as claimed in claim 14 wherein if said speaker responds to said prompting step, determining if a first response is a data item or a command or query and

if said first response is a data item:

filling in a node in said knowledge-tree graph instantiation corresponding to said data item; and

generating a second prompt to elicit an additional data item to fill in an empty node in said knowledge-tree graph instantiation;

if said first response is a command;

checking said knowledge-tree graph instantiation to determine if a predetermined number of nodes have been filled in to execute said command; and

if an insufficient number of nodes have been filled in, iteratively generating additional prompts to complete said knowledge-tree graph instantiation sufficiently to execute said command; and

if said first response is a query;

checking whether the query is related in context to a dialog;

if the query is context-related, checking said knowledge-tree graph instantiation to determine if a node contains a responsive answer; and

if there is no responsive answer, generating a responsive prompt.

17. A method of interactively responding to voice as claimed in claim 16 wherein, if any of the responses received are not recognized:

processing an unrecognized response with an ontological semantics engine to determine a context and meaning for said unrecognized response;

adding a node to said knowledge-tree graph instantiation and populating said node with said unrecognized response;

evaluating a plurality of knowledge-tree graph instantiations and increasing a weight associated with an added node each time it appears in a knowledge-tree graph instantiation; and

if a given added node reaches a predetermined threshold, adding a prompt to a data repository of phrase constructs.

18. A method of interactively responding to voice as claimed in claim 17 wherein processing an unrecognized response further comprises performing one or more actions chosen from the group consisting of:

accessing a third party data repository to determine at least one property chosen from the group consisting of a category and/or a synonym for said unrecognized response and attempting to find a node associated with said at least one synonym; and

searching one or more knowledge-tree graph instantiations stored in a customer-relationship management data repository for a relationship between said unrecognized response and a concept.

19. A method of interactively responding to voice as claimed in claim 16 wherein, if any of the responses received are recognized:

filling in at least one node in said knowledge-tree graph instantiation corresponding to said response;

defining a relationship between said response and at least one matched node in said knowledge-tree graph instantiation; and

calculating and assigning a weight for each relationship, indicative how strong said relationship between said response and each of the matched nodes is.

20. A method of interactively responding to voice as claimed in claim 14 further comprising:

receiving an input;

associating a set of inputs to one or more nodes in said knowledge-tree graph instantiation wherein said knowledge-tree graph instantiation is based upon a home automation template comprising a set of states relating to a set of appliances and a set of commands for controlling said set of appliances;

wherein said set of commands comprises temperature adjustment commands, lighting commands, security commands, and appliance commands;

iteratively performing said receiving and associating steps until either a command input is received or said knowledge-tree graph instantiation is complete; and

upon completion of said activity, increasing a weight associated with a setting for an appliance according to a command received for said activity.