System and method for automating the generation of an ontology from unstructured documents

• US 20080021700A1
• Filed: 07/24/2006
• Published: 01/24/2008
• Est. Priority Date: 07/24/2006
• Status: Active Grant

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Inventors: Moitra, Abha, Crapo, Andrew Walter, Bodkin, Michael Allen
• Granted Patent: US 7,987,088 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/367 Ontology

Abstract

Systems and methods for the substantially automatic creation of ontologies from unstructured documents identify phrases and core noun phrases from the respective documents. Links can be extracted from the documents. Concepts can be identified from the documents. Ontologies can be automatically created for the documents. The processing is domain independent.

Claims

1. A domain independent method of creating an ontology comprising:

providing a corpus of documents;

extracting phrases from the documents;

extracting core noun phrases from the documents;

extracting links from the documents; and

generating an ontology in accordance with at least the extracted phrases.

2. A method as in claim 1 where generating the ontology includes adding core noun phrases.

3. A method as in claim 1 where generating the ontology includes adding link concepts.

4. A method as in claim 2 where generating the ontology includes adding link concepts.

5. A method as in claim 1 where extracting phrases includes splitting sentences in the documents into fragments.

6. A method as in claim 2 which includes processing fragments into phrases.

7. A method as in claim 6 which includes limiting phrases to a predetermined number of words.

8. A method as in claim 7 which includes creating a frequency indicator for each phrase.

9. A method as in claim 8 which includes removing phrases that fail to exceed a predetermined frequency parameter.

10. A method as in claim 7 which includes providing an adjective word list, a barrier word list and a verb list.

11. A method as in claim 10 which includes establishing a core noun phrase file the members of which include first and second sections where each word in each first section is in the adjective word list.

12. A method as in claim 11 and where the words in the respective second sections are not in any of the lists.

13. A method as in claim 12 which includes establishing a frequency indicator for the members of the file.

14. A method as in claim 13 where those members of the file whose respective frequency indicator fails to exceed a predetermined threshold are deleted from the file.

15. A method as in claim 12 where extracting links includes automatically generating subject and object lists from members of the corpus of documents.

16. A method as in claim 15 which includes forming multi-element links that include a subject element, a predicate element and an object element.

17. A method as in claim 16 where the multi-element links are sorted in accordance with a predetermined criterion.

18. A method as in claim 17 where generating includes evaluating each phrase and, in accordance with a first criterion, determining if each respective phrase should be added to the ontology.

19. A method as in claim 18 where generating includes evaluation of each core noun phrase, and in accordance with a second criterion determining if each respective core noun phrase should be added to the ontology.

20. A method as in claim 19 where generating includes evaluating each link, and in accordance with a third criterion determining if at least portions of each link should be added to the ontology.

21. An apparatus comprising:

a document file;

first software, executable by a processor that analyzes documents in the file and forms an extracted phrases file;

second software, executable by a processor that analyzes documents in the file and forms a core noun phrases file;

third software, executable by a processor, that analyzes documents in the file and forms a link file; and

fourth software, executable by a processor, that forms an ontology in accordance with selected phrases in the extracted phrases file.

22. An apparatus as in claim 21 which includes fifth software, executable by a processor, that adds core noun phrase concepts from the core noun phrases file to the ontology.

23. An apparatus as in claim 22 which includes sixth software, executable by a processor, that adds to the ontology link concepts from the links file.

24. An apparatus as in claim 23 which includes a display for reviewing the ontology.

25. An ontology generating system comprising:

first software, recorded on a computer readable medium, that extracts and stores phrases from at least one text source;

second software, recorded on a computer readable medium, that extracts and stores core noun phrases from the extracted phrases;

third software, recorded on a computer readable medium, that extracts and stores links from the extracted phrases; and

fourth software, recorded on a computer readable medium, that generates an ontology for the at least one text source.

26. A system as in claim 25 where the fourth software generates the ontology substantially automatically irrespective of the subject matter of the at least one text source.

27. A system as in claim 25 which includes additional software for combining textual material from a plurality of sources to form the at least one text source.

28. A system as in claim 27 which includes at least one processor which executes at least some of the first, second, third or fourth software.