Optimization of fact extraction using a multi-stage approach

• US 20080027888A1
• Filed: 07/31/2006
• Published: 01/31/2008
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Azzam, Saliha, Humphreys, Kevin William
• Granted Patent: US 7,668,791 B2
• Time in Patent Office: Days
• US Class Current: 706/45
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

Facts are extracted from electronic documents by recognizing factual descriptions using a fact-word table to match to words of the electronic documents. The words of those factual descriptions may be tagged with the appropriate part of speech. More detailed analysis is then performed on those factual descriptions, rather than on the entire electronic document, and particularly to the text in the neighborhood of the fact-word matches. The analysis may involve identifying the linguistic constituents of each phrase and determining the role as either subject or object. Exclusion rules may be applied to eliminate those phrases unlikely to be part of facts, the exclusion rules being based in part on the linguistic constituents. Scoring rules may be applied to remaining phrases, and for those phrases having a score in excess of a threshold, the corresponding sentence part, whole sentence, paragraph, or other document portion may be presented as representing one or more facts.

Claims

1. A method of finding facts within electronic resources, comprising:

scanning an electronic resource to discover factual descriptions of sentences that comprise words matching words of a fact-word table;

examining the discovered factual descriptions to identify the linguistic constituents of the factual descriptions; and

determining whether to present a factual description as a fact based on the identified linguistic constituents.

2. The method of claim 1, wherein determining whether to present a factual description as fact based on the identified linguistic constituent comprises:

applying excluding rules in relation to the linguistic constituents of the factual descriptions to eliminate certain factual descriptions from consideration;

scoring the factual descriptions;

comparing the score of each factual description remaining for consideration to a threshold; and

for each factual description having a score that exceeds the threshold, presenting at least a portion of the sentence containing the factual description as a fact.

3. The method of claim 2, further comprising tagging words of the factual descriptions with their parts of speech.

4. The method of claim 3, wherein tagging words of the factual descriptions with their parts of speech comprises applying a noun tag when a word may be either a verb or a noun.

5. The method of claim 4, wherein applying the excluding rules comprises applying a first set of rules for syntactic phrases that have a role of subjects and applying a second set of rules for syntactic phrases that have a role of objects.

6. The method of claim 5, wherein applying the first set of rules comprises excluding noun phrases having an opinion or biased modifier of subjects or objects.

7. The method of claim 5, wherein applying the second set of rules comprises excluding subject noun phrases which non-proper name definite descriptions, excluding noun phrases which conatain pronouns, and excluding subject noun phrases which do not appear at the beginning of text.

8. The method of claim 5, further comprising applying a third set of rules without regard to the role of the noun phrase.

9. The method of claim 8, wherein applying the third set of rules comprises excluding factual descriptions where the punctuation of the sentence is a question mark, and excluding sentences with phrases that include a stop word.

10. The method of claim 2, wherein scoring the factual descriptions comprises scoring only those factual descriptions remaining for consideration either after or during application of the excluding rules.

11. A computer readable medium containing instructions that perform acts comprising:

receiving a search term;

parsing a plurality of electronic documents to discover factual descriptions of sentences that comprise words matching words of a fact-word table;

examining the discovered factual descriptions to identify the linguistic constituents of the factual descriptions; and

determining whether to present a factual description as a fact relevant to the search term based on the identified linguistic constituent.

12. The computer readable medium of claim 11, wherein the acts further comprise obtaining the plurality of documents by searching an collection of electronic documents to find those documents containing the search term, wherein the collection is searched to find those documents containing the search term prior to parsing the plurality of electronic documents.

13. The computer readable medium of claim 11, wherein the acts further comprise obtaining the electronic documents and presenting factual descriptions prior to receiving the search term and searching the electronic documents and factual descriptions to find those electronic documents and corresponding factual descriptions that are relevant to the search term.

14. The computer readable medium of claim 11, wherein determining whether to present a factual description as a fact relevant to the search term based on the identified linguistic constituent comprises:

applying excluding rules in relation to the linguistic constituents of the factual descriptions to eliminate a portion of the factual descriptions from consideration;

scoring the factual descriptions;

comparing the score of each factual description remaining for consideration to a threshold; and

for each factual description that is taken from an electronic document that contains the search term and that has a score that exceeds the threshold, presenting at least a portion of the sentence containing the factual description as a fact relevant to the search term.

15. The computer readable medium of claim 14, wherein scoring the factual descriptions comprises scoring only those factual descriptions remaining for consideration after applying the excluding rules.

16. A computer system, comprising:

storage containing a plurality of electronic resources that comprise textual information;

a processor that receives a request to present facts that are related to the search term from a set of electronic documents, wherein the processor parses the plurality of electronic documents to discover factual descriptions of sentences that comprise words matching words of a fact-word table, examines the discovered factual descriptions to identify the linguistic constituents of the factual descriptions, determines whether to present a factual description as a fact based on the identified linguistic constituent, and presents at least a portion of sentences that contain the factual descriptions that are determined to be presented as a fact and that are related to the search term.

17. The computer system of claim 16, further comprising a display device and wherein the processor presents at least the portion of the sentences by displaying at least the portions of the sentences on the display device.

18. The computer system of claim 16, further comprising a network interface and wherein the processor presents at least the portion of the sentences by outputting those portions to another computer via the network interface.

19. The computer system of claim 16, further comprising a network interface and wherein the storage is accessible by the processor via the network interface.

20. The computer system of claim 16, wherein the processor determines whether to present a factual description as fact by:

applying excluding rules in relation to the linguistic constituents of the factual descriptions to eliminate a portion of the factual descriptions from consideration;

scoring the factual descriptions;

comparing the score of each factual description remaining for consideration to a threshold; and

for each factual description that contains the search term and that has a score that exceeds the threshold, presenting at least the portion of the sentence containing the factual description as a fact relevant to the search term.