Architecture and processes for computer learning and understanding

• US 10,496,754 B1
• Filed: 03/20/2017
• Issued: 12/03/2019
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Mind Lakes, LLC
• Original Assignee: Elemental Cognition LLC
• Inventors: Ferrucci, David, Barborak, Mike, Buchanan, David, Burnham, Greg, Chu-Carroll, Jennifer, Kalyanpur, Aditya, Lally, Adam, Pacifico, Stefano, Wang, Chang
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/464,111
• Time in Patent Office: 988 Days
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/027 Frames
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An architecture and processes enable computer learning and developing an understanding of arbitrary natural language text through collaboration with humans in the context of joint problem solving. The architecture ingests the text and then syntactically and semantically processes the text to infer an initial understanding of the text. The initial understanding is captured in a story model of semantic and frame structures. The story model is then tested through computer generated questions that are posed to humans through interactive dialog sessions. The knowledge gleaned from the humans is used to update the story model as well as the computing system'"'"'s current world model of understanding. The process is repeated for multiple stories over time, enabling the computing system to grow in knowledge and thereby understand stories of increasingly higher reading comprehension levels.

Claims

1. A computer-implemented method, comprising:

forming, by a computing system, a semantic representation of a natural language story, the semantic representation using knowledge stored in a current world model that expresses beliefs about how natural language is understood, wherein the semantic representation includes multiple generative semantic primitive (GSP) structures, the multiple GSP structures including one or more predicates and one or more roles that indicate one or more beliefs regarding an understanding of natural language, wherein the current world model is developed over time through processing of multiple natural language stories;

generating, by the computing system, multiple questions to evaluate the semantic representation;

maintaining a dependency structure for linear dialogs to identify questions that are independent from one another;

submitting, from the computing system to multiple user computing devices, different ones of the multiple questions for presentation to multiple different human users;

receiving, by the computing system from the multiple user computing devices, multiple responses indicative of input from the multiple different human users when answering the multiple questions;

iterating through the dependency structure based at least in part on the multiple responses to identify a set of next questions and branch conditions indicating that at least two next questions of the set of next questions are independent of one another;

learning, by the computing system, at least one new GSP structure based at least in part on aggregating the multiple responses received from the multiple user computing devices, wherein the at least one new GSP structure includes a new predicate that indicates a new belief regarding the understanding of the natural language, the new belief being based at least in part on the aggregating of the multiple responses; and

revising, by the computing system, the semantic representation of the natural language story based at least in part on the multiple responses received from the multiple user computing devices and the at least one new GSP structure.

2. The computer-implemented method of claim 1, wherein submitting the multiple questions comprises:

determining from the dependency structure that one or more first questions are independent of one or more second questions; and

sending the first questions to one or more first user computing devices and sending the second questions to one or more second user computing devices.

3. The computer-implemented method of claim 2, wherein the first questions are sent in parallel with sending the second questions.

4. The computer-implemented method of claim 1, wherein generating the multiple questions comprises forming the multiple questions in a natural language.

5. The computer-implemented method of claim 1, further comprising aggregating the multiple responses from the multiple user computing devices in association with the multiple questions to which the responses apply.

19. The computer-implemented method of claim 1, wherein the dependency structure provides a dialog context for interpreting and answering the questions.

20. The computer-implemented method of claim 1, wherein submitting the different ones of the multiples questions includes submitting the different ones of the multiples questions for presentation via a computer user interface to the multiple different human users.

6. A computer-implemented method, comprising:

receiving, at a computing system, a natural language story composed of multiple sentences;

performing, by the computing system, a syntactic analysis of the natural language story to produce one or more linguistic analysis results;

inferring, by the computing system and from the linguistic analysis results, one or more generative semantic primitive (GSP) structures corresponding to a linguistic structure of a sentence in the natural language story, wherein a GSP structure of the one or more GSP structures includes a predicate and one or more roles to express a first understanding of the linguistic structure, and wherein the first understanding of the linguistic structure is based at least in part on the linguistic analysis results;

forming, by the computing system, one or more frame structures that comprise the one or more GSP structures as inferred from the linguistic analysis results, wherein a frame structure of the one or more frame structures is associated with a particular context of a setting or a background of the natural language story, wherein at least one GSP structure of the one or more GSP structures has at least a threshold probability of being included in the frame structure that is associated with the particular context of the setting or the background of the natural language story;

aligning, by the computing system, the one or more frame structures with entities in the natural language story to represent an understanding of the natural language story;

generating, by the computing system, one or more questions to evaluate at least one of the one or more GSP structures or the one or more frame structures;

determining a number of multiple different human users and to whom the one or more questions are to be submitted using heuristics and profiles of the multiple different human users;

submitting, from the computing system to multiple user computing devices, the one or more questions for presentation to the multiple different human users;

receiving, by the computing system from the multiple user computing devices, multiple responses indicative of input from the multiple different human users when answering the one or more questions;

aggregating, by the computing system, the multiple responses from the multiple user computing devices in association with the one or more questions to which the multiple responses apply;

using, by the computing system, the multiple responses to evaluate an extent to which the one or more frame structures represent an understanding of the natural language story; and

revising, by the computing system, at least the a GSP structure by redefining the one or more roles for the predicate to express a second understanding of the linguistic structure, and wherein the second understanding of the linguistic structure is based at least in part on aggregating the multiple responses.

7. The computer-implemented method of claim 6, wherein generating the one or more questions comprises building a dependency structure for linear dialogs to identify questions are independent of one another.

8. The computer-implemented method of claim 7, wherein submitting the one or more questions comprises:

determining from the dependency structure that one or more first questions are independent of one or more second questions; and

sending the first questions to one or more first user computing devices and sending the second questions to one or more second user computing devices.

9. The computer-implemented method of claim 8, wherein the first questions are sent in parallel with sending the second questions.

10. The computer-implemented method of claim 6, further comprising revising the one or more GSP structures or the one or more frame structures based on information received from the multiple responses.

11. The computer-implemented method of claim 6, further comprising:

repeating the method of claim 9 for multiple natural language stories to create multiple sets of questions, wherein each set of question is associated with a corresponding natural language story; and

submitting, by the computing system, the sets of questions to different sets of multiple human users.

12. The computer-implemented method of claim 11, further comprising storing, by the computing system in a datastore, a current world model that expresses beliefs about how natural language is understood, the current world model comprising the GSP structures and frame structures derived from the multiple language stories as evaluated by the multiple different human users.

13. The computer-implemented method of claim 6, further comprising selecting a number of the multiple different human users as a function of load balancing and scalability to process a plurality of natural language stories in parallel.

14. A computing system, comprising:

one or more processors; and

memory coupled to the one or more processors, the memory storing computer-executable modules comprising;

a knowledge integration engine to form semantic structures that provide a semantic representation of a natural language story, wherein to form the semantic structures includes creating multiple generative semantic primitive (GSP) structures by defining a predicate and one or more roles for a GSP structure of the multiple GSP structures;

a dialog engine to facilitate multiple dialog sessions with multiple human users in parallel and to aggregate responses from the human users to evaluate an extent to which the semantic structures represent the natural language story, wherein the dialog engine includes a distributed dialog dispatcher to break up the dialog sessions into sub-dialog sessions, and the distributed dialog dispatcher includes a user selector to choose the human users to answer independent questions in parallel, wherein the user selector employs heuristics and profiles of the human users to determine a quantity and to whom the independent questions are to be submitted; and

wherein the knowledge integration engine updates the semantic structures based in part on the responses aggregated from the multiple human users and learns a new GSP structure including a new predicate discovered by the responses aggregated.

15. The computing system of claim 14, wherein the dialog engine distributes the sub-dialog sessions to different human users in parallel.

16. The computing system of claim 15, wherein the distributed dialog dispatcher uses a dependency structure to break up the dialog sessions, the dependency structure providing a dialog context for interpreting and answering questions pertaining to the natural language story, traversing the dependency structure to identify the questions that are independent of one another.

17. The computing system of claim 16, wherein the distributed dialog dispatcher further includes a structure crawler to iterate through the dependency structure based on the human user responses to identify one or more next questions and branch conditions indicating that at least two next questions are independent from one another.

18. The computing system of claim 14, wherein the dialog engine comprises a natural language generation engine to generate questions in natural language for a user to understand.

21. The computing system of claim 14, wherein the user selector selects a number of the human users as a function of load balancing and scalability to answer the independent questions in parallel.