Extracting facts from unstructured text

• US 9,424,524 B2
• Filed: 12/02/2014
• Issued: 08/23/2016
• Est. Priority Date: 12/02/2013
• Status: Active Grant

• Current Assignee: Finch Computing LLC (Qbase, LLC)
• Original Assignee: Qbase, LLC
• Inventors: Dave, Rakesh, Boddhu, Sanjay
• Primary Examiner(s): Waldron, Scott A
• Application Number:
  US14/557,802
• Publication Number:
  US 20150154193A1
• Time in Patent Office: 630 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method for extracting facts from unstructured text files are disclosed. Embodiments of the disclosed system and method may receive a text file as input and perform extraction and disambiguation of entities, as well as extract topics and facts. The facts are extracted by comparing against a fact template store and associating facts with events or topics. The extracted facts are stored in a data store.

Claims

1. A method comprising:

receiving, by an entity extraction computer, an electronic document having unstructured text, wherein the electronic document is a text file;

extracting, by the entity extraction computer, an entity identifier from the unstructured text in the electronic document;

extracting, by a topic extraction computer, a topic identifier from the unstructured text in the electronic document;

extracting, by a fact extraction computer, a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, wherein the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and

associating, by a fact relatedness estimator computer, the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.

2. The method of claim 1, wherein the distance in the unstructured text is calculated using tokenization.

3. The method of claim 1, wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.

4. The method of claim 1 wherein the fact template model includes metadata.

5. The method of claim 4 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.

6. The method of claim 4 wherein the confidence score is stored in the metadata.

7. A system comprising:

one or more server computers having one or more processors executing computer readable instructions for a plurality of computer modules including;

an entity extraction module which receives an electronic document having unstructured text and extracts an entity identifier from the unstructured text in the electronic document, wherein the electronic document is a text file;

a topic extraction module which extracts a topic identifier from the unstructured text in the electronic document;

a fact extraction module which extracts a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, wherein the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and

a fact relatedness estimator module which associates the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.

8. The system of claim 7, wherein the distance in the unstructured text is calculated using tokenization.

9. The system of claim 7, wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.

10. The system of claim 7 wherein the fact template model includes metadata.

11. The system of claim 10 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.

12. The system of claim 10 wherein the confidence score is stored in the metadata.

13. A non-transitory computer readable medium having stored thereon computer executable instructions instructive of a method comprising:

receiving, by an entity extraction computer, an electronic document having unstructured text, wherein the electronic document is a text file;

extracting, by the entity extraction computer, an entity identifier from the unstructured text in the electronic document;

extracting, by a topic extraction computer, a topic identifier from the unstructured text in the electronic document;

extracting, by a fact extraction computer, a fact identifier from the unstructured text in the electronic document by comparing text string structures in the unstructured text to a fact template database, the fact template database having stored therein a fact template model identifying keywords pertaining to specific fact identifiers and corresponding keyword weights; and

associating, by a fact relatedness estimator computer, the entity identifier with the topic identifier and the fact identifier to determine a confidence score indicative of a degree of accuracy of extraction of the fact identifier, wherein the confidence score is based at least in part on a spatial distance between a part of the unstructured text in the electronic document from where the fact identifier was extracted and a part of the unstructured text from where at least one of the topic identifier or the entity identifier was extracted.

14. The non-transitory computer readable medium of claim 13, wherein the distance in the unstructured text is calculated using tokenization.

15. The non-transitory computer readable medium of claim 13, wherein the confidence score is further based at least in part on comparing co-occurring entity identifiers in the electronic document.

16. The non-transitory computer readable medium of claim 13 wherein the fact template model includes metadata.

17. The non-transitory computer readable medium of claim 16 wherein the metadata includes a count of a number of times a sentence structure corresponding to the fact template model is repeated across a plurality of electronic documents.