Architecture and processes for computer learning and understanding

• US 10,657,205 B2
• Filed: 03/20/2017
• Issued: 05/19/2020
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Mind Lakes, LLC
• Original Assignee: Elemental Cognition LLC
• Inventors: Ferrucci, David, Barborak, Mike, Buchanan, David, Burnham, Greg, Chu-Carroll, Jennifer, Kalyanpur, Aditya, Lally, Adam, Pacifico, Stefano, Wang, Chang
• Primary Examiner(s): He, Jialong
• Application Number:
  US15/463,936
• Publication Number:
  US 20200034421A1
• Time in Patent Office: 1,156 Days
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/027 Frames
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

An architecture and processes enable computer learning and developing an understanding of arbitrary natural language text through collaboration with humans in the context of joint problem solving. The architecture ingests the text and then syntactically and semantically processes the text to infer an initial understanding of the text. The initial understanding is captured in a story model of semantic and frame structures. The story model is then tested through computer generated questions that are posed to humans through interactive dialog sessions. The knowledge gleaned from the humans is used to update the story model as well as the computing system'"'"'s current world model of understanding. The process is repeated for multiple stories over time, enabling the computing system to grow in knowledge and thereby understand stories of increasingly higher reading comprehension levels.

Claims

1. A computer-implemented method, comprising:

maintaining a collection of frame structures within memory of a computing system, individual frame structures of the collection of frame structures including multiple generative semantic primitive (GSP) structures defined with roles that are commonly associated with a theme or a context of a setting of natural language stories;

receiving, by the computing system, a natural language story;

retrieving, by the computing system and from the collection of frame structures in the memory, one or more frame structures that includes GSP structures defined with the roles that are relevant to a current understanding of the natural language story, wherein a frame structure of the one or more frame structures defines the theme represented within the natural language story, and the frame structure comprises a set of GSP structures that define the roles associated with the theme, an individual GSP structure of the GSP structures having at least a threshold probability of being included in the one or more frame structures that is associated with a particular context of a particular setting of the natural language story;

aligning, by the computing system, entities in the natural language story to the roles defined in the GSP structures of the one or more frame structures; and

evaluating, by the computing system, an extent to which the one or more frame structures, when aligned with the entities in the natural language story, represent the current understanding of the natural language story by computing a confidence score, the confidence score being based at least in part on the individual GSP structure of the GSP structures having a probability of being true for the one or more frame structures, wherein the confidence score is determined according to a function that produces a higher score based on a comparison of a lower number of frame structures with respect to a higher number of GSP structures included in the frame structures that match GSP structures extracted from the natural language story.

2. The computer-implemented method of claim 1, wherein the confidence score is computed according to a function that produces a score decrease when the one or more frame structures are contradictory to beliefs extracted from the natural language story.

3. The computer-implemented method of claim 1, further comprising:

creating, by the computing system, a new frame structure based at least in part on information derived from the natural language story; and

storing the new frame structure as part of the collection of frame structures in the memory of the computing system.

4. The computer-implemented method of claim 1, further comprising engaging in a dialog session with a human user to receive input from the human user for use by the computing system as part of evaluating the extent to which the one or more frame structures represent the current understanding of the natural language story.

5. The computer-implemented method of claim 1, further comprising iterating the retrieving, the aligning, and the evaluating to produce a hierarchy of the one or more frame structures, wherein at least one frame structure of the one or more frame structures references at least another frame structure.

23. The computer-implemented method of claim 1, further comprising engaging in multiple dialog sessions with computing devices of multiple human users in parallel to receive responses from the multiple human users for use by the computing system to evaluate an extent to which the one or more frame structures represent the current understanding of the natural language story.

24. The computer-implemented method of claim 1, further comprising determining, by the computer system, co-reference chains in one or more roles of the natural language story.

27. The computer-implemented method of claim 1, wherein at least one frame structure of the one or more frame structures is nested within, and referred to by, at least another semantic structure.

6. A computer-implemented method, comprising:

receiving, at a computing system, a natural language input;

performing, by the computing system, a syntactic analysis of the natural language input to produce one or more linguistic analysis results;

inferring, by the computing system, one or more frame structures from the one or more linguistic analysis results, the one or more frame structures expressing an understanding of the natural language input, a frame structure of the one or more frame structures including beliefs having relevance to a context of the natural language input, wherein at least one frame structure of the one or more frame structures is nested within, and referred to by, at least another semantic structure; and

calculating, by the computing system, a confidence score indicative of an extent to which the one or more frame structures express the understanding of the natural language input based at least in part on whether the confidence score satisfies a predetermined threshold and on the relevance of the beliefs to the context of the natural language input, wherein the confidence score is computed according to a function that produces a low score, when the one or more frame structures are contradictory to the beliefs extracted from the natural language input, relative to higher number of frame structures that match the beliefs extracted from the natural language input.

7. The computer-implemented method of claim 6, wherein the confidence score is computed according to a function that produces a higher score based on a ratio of a lower number of frame structures with respect to a higher number of beliefs associated with the frame structures match the beliefs extracted from the natural language input.

8. The computer-implemented method of claim 6, further comprising engaging in a dialog session with a human user to receive input from the human user for use by the computing system as part of evaluating the extent to which the one or more frame structures represent the understanding of the natural language input.

25. The computer-implemented method of claim 6, further comprising engaging in multiple dialog sessions with computing devices of multiple human users in parallel to receive responses from the multiple human users for use by the computing system to evaluate an extent to which the one or more frame structures represent the natural language input.

26. The computer-implemented method of claim 6, wherein performing the syntactic analysis of the natural language input comprises:

performing analysis on the natural language input to produce a syntactic representation of the natural language input;

performing predicate argument structure (PAS) analysis on the syntactic representation of the natural language input;

assigning one or more entity types to one or more words in the natural language input; and

determining co-reference chains in the one or more words in the natural language input.

9. A computer-implemented method, comprising:

receiving, at a computing system, a natural language story composed of multiple sentences;

performing, by the computing system, a syntactic analysis of the natural language story to produce one or more linguistic analysis results;

inferring, by the computing system and from the one or more linguistic analysis results, one or more generative semantic primitive (GSP) structures corresponding to a linguistic structure of a sentence in the natural language story, wherein individual GSP structures of the one or more GSP structures include a predicate and one or more roles;

forming one or more frame structures that comprise the one or more GSP structures as inferred from the one or more linguistic analysis results, wherein forming the one or more frame structures includes associating the one or more GSP structures with a particular context based at least in part on a setting or a background of the natural language story and defining a relationship between the one or more GSP structures, a GSP structure of the one or more GSP structures having at least a threshold probability of being included with the one or more frame structures that is associated with the particular context of the setting or the background of the natural language story, wherein a frame structure of the one or more frame structures defines a context represented within the natural language story, and the frame structure comprises a set of GSP structures that define the roles associated with the context; and

aligning the one or more frame structures with entities in the natural language story to represent an understanding of the natural language story, wherein aligning the one or more frame structures includes mapping at least one entity of the entities in the natural language story to at least one role of the roles, wherein the one or more frame structures form an episode within the natural language story.

10. The computer-implemented method of claim 9, further comprising iterating the inferring, the forming, and the aligning to produce a hierarchy of frame structures wherein a first frame structure references one or more second frame structures.

11. The computer-implemented method of claim 9, further comprising:

storing the one or more GSP structures and the one or more frame structures in memory as part of a current world model that expresses beliefs about how natural language is understood; and

repeating the receiving, the performing, the inferring, the forming, and the aligning for multiple additional natural language stories to produce additional GSP structures and additional frame structures; and

storing the additional GSP structures and the additional frame structures in the memory as part of the current world model.

12. The computer-implemented method of claim 9, further comprising engaging in a dialog session with a human user to receive input from the human user for use by the computing system as part of evaluating an extent to which the one or more frame structures represent an understanding of the natural language story.

13. A computer-implemented method, comprising:

receiving, at a computing system, a natural language input;

forming, by the computing system, one or more frame structures to represent an understanding of the natural language input, the one or more frame structures including one or more sets of generative semantic primitive (GSP) structures that are interrelated, the one or more sets of GSP structures being interrelated based at least in part on individual GSP structures of the one or more sets of GSP having an above threshold probability of being included with the one or more frame structures that is associated with a context of the natural language input, wherein at least one frame structure of the one or more frame structures is nested within, and referred to by, at least another semantic structure;

engaging, via communication between the computing system and a computing device used by a human user, in a dialog session with the human user to present the understanding of the natural language input for the human user to evaluate and to receive input from the human user for use by the computing system to evaluate an extent to which the one or more frame structures represent the understanding of the natural language input; and

using, by the computing system, the input from the human user to infer an additional frame structure to improve the understanding of the natural language input, wherein the additional frame structure includes a new set of GSP structures.

14. The computer-implemented method of claim 13, further comprising:

performing, by the computing system, a syntactic analysis of the natural language input to produce one or more linguistic analysis results; and

wherein forming the one or more frame structures comprises inferring, by the computing system, the one or more frame structures at least in part from the one or more linguistic analysis results.

15. The computer-implemented method of claim 13, further comprising engaging, via communication between the computing system and multiple computing devices used by multiple human users, in multiple dialog sessions with the multiple human users.

16. The computer-implemented method of claim 13, wherein forming the one or more frame structures further comprises inferring, by the computing system, one or more GSP structures corresponding to a linguistic structure of the natural language input.

17. A computing system, comprising:

a datastore containing a current world model that expresses beliefs about how natural language is understood;

one or more processors; and

memory coupled to the one or more processors, the memory storing computer-executable modules comprising;

a story parsing engine to syntactically analyze a natural language story to produce linguistic analysis results, wherein producing the linguistic analysis results includes;

parsing the natural language story to produce a syntactic representation of the natural language story;

performing a predicate argument structure (PAS) analysis on the syntactic representation of the natural language story;

assigning one or more entity types to one or more words in the natural language story; and

determining co-reference chains in the one or more words in the natural language story;

a knowledge induction engine to induce information from language corpora to form induced information; and

a knowledge integration engine to form frame structures that provide a semantic representation of the natural language story, the knowledge integration engine using the linguistic analysis results, information from the current world model, and the induced information to form the frame structures, wherein to form individual frame structures includes interrelating multiple generative semantic primitive (GSP) structures by associating the multiple GSP structures with a particular context of the natural language story based at least in part on a GSP structure of the multiple GSP structures having at least a threshold probability of being included with the individual frame structures that is associated with the particular context of the natural language story, wherein at least one semantic structure of the frame structures is nested within, and referred to by, at least another semantic structure.

18. The computing system of claim 17, wherein the frame structures are composed of the multiple GSP structures that are defined with one or more roles that are common to the particular context.

19. The computing system of claim 17, wherein the linguistic analysis results comprise a syntactic parse, a predicate argument structure, entity type, and co-references of the natural language story.

20. The computing system of claim 17, wherein the knowledge integration engine maps the linguistic analysis results to GSP structures by selecting uninstantiated GSP structures and instantiating the GSP structures with information from the linguistic analysis results.

21. The computing system of claim 17, wherein the memory further stores additional computer-executable modules comprising:

a dialog engine to facilitate a dialog session with a human user by generating and submitting one or more questions to a computing device used by the human user and collecting one or more responses from the computing device indicative of input from the human user; and

the knowledge integration engine revising the frame structures based on the one or more responses from the human user and updating the current world model in the datastore, wherein revising the frame structures includes adding at least one new GSP structure.

22. The computing system of claim 21, wherein the story parsing engine, the knowledge integration engine, and the dialog engine process multiple natural language stories over time to build the information in the current world model.