Adaptive virtual intelligent agent

• US 10,565,509 B2
• Filed: 10/27/2015
• Issued: 02/18/2020
• Est. Priority Date: 11/20/2013
• Status: Active Grant

• Current Assignee: Justin London
• Original Assignee: Justin London
• Inventors: London, Justin
• Primary Examiner(s): Zhen, Li B.
• Assistant Examiner(s): Chubb, Mikayla
• Application Number:
  US14/923,592
• Publication Number:
  US 20160117593A1
• Time in Patent Office: 1,575 Days
• Field of Search: None
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/042 Knowledge-based neural networks; Logical representations of neural networks
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/016 After-sales
• CPC Class Codes: G06Q 30/0613 Third-party assisted
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Embodiments of an adaptive virtual intelligent agent (“AVIA”) service are disclosed. It may include the functions of a human administrative assistant for an enterprise including customer support, customer relationship management, and fielding incoming caller inquiries. It also has multi-modal applications for the home through interaction with AVIA implemented in the home. It may engage in free-form natural language dialogs. During a dialog, embodiments maintain the context and meaning of the ongoing dialog and provides information and services as needed by the domain of the application. Over time, the service automatically extends its knowledge of the domain (as represented in the Knowledge Tree Graphs) through interaction with external resources. Embodiments can intelligently understand and converse with users using free-form speech without pre-programmed deterministic sequences of questions and answers, can dynamically determine what it needs to know to converse meaningfully with users, and knows how to obtain information it needs.

Claims

1. A conversational system comprising:

a microphone; and

a processor coupled to memory that includes;

a data structure object that includes a graph data structure comprising structure nodes, wherein the structure nodes include structure edges that define links between the structure nodes; and

a program that instructs the processor to perform;

receiving a first voice-based input via the microphone;

generating, in response to the first voice-based input, a knowledge-tree graph instantiation based on the data structure object, wherein the knowledge tree graph instantiation includes;

instantiation nodes based on the structure nodes, wherein the instantiation nodes each include an initial node timestamp; and

instantiation edges based on the structure edges, wherein the instantiation edges define links between the instantiation nodes, wherein the instantiation edges each include an initial edge timestamp;

receiving a second voice-based input via the microphone;

modifying the instantiation nodes based on the second voice-based input and including a modified node timestamp corresponding to the modification;

modifying the instantiation edges based on the second voice-based input and including a modified edge timestamp of the modification; and

tracing the knowledge-tree graph instantiation backwards based on;

the received second voice-based input,

the initial node timestamp,

the initial edge timestamp,

the modified node timestamp, and

the modified edge timestamp.

2. The conversational system of claim 1, wherein the program instructs the processor to further perform:

communicating with sensors placed remotely from the system to receive sensor data; and

controlling a home device based on the sensor data and the knowledge tree graph instantiation nodes.

3. The conversational system of claim 2, wherein:

the home device includes a security camera and an alarm; and

the data structure template object includes a security alarm ontology.

4. The conversational system of claim 3, wherein the sensors include the security camera.

5. The conversational system of claim 1 further including a communications interface to a display, wherein the communications interface is controlled based the second voice-based input.

6. The conversational system of claim 1 further including a panel display, wherein the panel display is controlled based the second voice-based input.

7. The conversational system of claim 1, wherein the program instructs the processor to further perform:

determining a prompt based on the knowledge tree graph instantiation nodes;

issuing the prompt; and

receiving the second voice-based input via the microphone further includes receiving the second voice-based input via the microphone after the prompt is issued.

8. The conversational system of claim 1, wherein:

the memory further includes additional data structure objects that each include a graph data structure comprising structure nodes and template edges; and

generating, in response to the first voice-based input, a knowledge-tree graph instantiation based on the data structure object includes;

selecting a select data structure template object from among the data structure template object and the additional data structure template objects based on the first voice-based input; and

generating the knowledge-tree graph instantiation based on the select data structure template object and the first voice-based input.

9. The conversational system of claim 8, wherein selecting a select data structure object from among the data structure object and the additional data structure objects based on the first voice-based input includes:

determining an identity of a speaker of the first voice-based input; and

selecting a select data structure object from among the data structure object and the additional data structure objects based on the identity of the speaker.

10. The conversational system of claim 1, further including:

determining an identity of a speaker of the first voice-based input;

wherein;

generating, in response to the first voice-based input, a knowledge-tree graph instantiation based on the data structure object further includes;

generating a knowledge-tree graph instantiation based on the data structure object if the speaker is unknown; and

utilizing a previously generated knowledge-tree graph instantiation as the knowledge-tree graph instantiation if the user is known.

11. The conversational system of claim 1, wherein:

the data structure template object includes a car ontology; and

the program instructs the processor to further perform communicating with sensors within a vehicle to receive sensor data.

12. The conversational system of claim 11, wherein the program instructs the processor to further perform:

determining whether the second voice-based input indicates that a speaker of the second voice-based input wants information about the sensor; and

relaying the information about the sensor via an audio speaker of the car.

13. The conversational system of claim 12 with a housing including a communications interface to a display.

14. The conversational system of claim 12 further including a panel display, wherein the panel display is controlled based on the second voice-based input.

15. The conversational system of claim 11 further including a camera sensor.

16. The conversational system of claim 15, wherein the program further instructs the processor to communicate with sensors of a vehicle based on the camera sensor.

17. The conversational system of claim 15, wherein the program further instructs the processor to communicate with a panel display of a vehicle based on the camera sensor.

18. A conversational system comprising:

a processor coupled to memory that includes;

a data structure object that includes a graph data structure comprising structure nodes, wherein the structure nodes include structure edges that define links between the structure nodes; and

a program that instructs the processor to perform;

receiving a first voice-based input;

generating, in response to the first voice-based input, a knowledge-tree graph instantiation based on the data structure object, wherein the knowledge tree graph instantiation includes;

instantiation nodes based on the structure nodes, wherein the instantiation nodes each include an initial node timestamp; and

instantiation edges based on the structure edges, wherein the instantiation edges define links between the instantiation nodes, wherein the instantiation edges each include an initial edge timestamp;

receiving a second voice-based input;

modifying the instantiation nodes based on the second voice-based input and including a modified node timestamp corresponding to the modification;

modifying the instantiation edges based on the second voice-based input and including a modified edge timestamp of the modification; and

tracing the knowledge-tree graph instantiation backwards based on;

the received second voice-based input,

the initial node timestamp,

the initial edge timestamp,

the modified node timestamp, and

the modified edge timestamp.