Form-based ontology creation and information harvesting

• US 8,103,962 B2
• Filed: 11/04/2009
• Issued: 01/24/2012
• Est. Priority Date: 11/04/2008
• Status: Expired due to Fees

• Current Assignee: Brigham Young University (The Church of Jesus Christ of Latter-Day Saints)
• Original Assignee: Brigham Young University (The Church of Jesus Christ of Latter-Day Saints)
• Inventors: Embley, David W., Tao, Cui, Liddle, Stephen W.
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): PHANTANA ANGKOOL, DAVID
• Application Number:
  US12/612,570
• Publication Number:
  US 20100115436A1
• Time in Patent Office: 811 Days
• Field of Search: 715/763, 715/780, 715/811
• US Class Current: 715/763
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/174 Form filling; Merging

Abstract

Extracting data from web pages. User input is received defining a tabular form. User input is received correlating portions of the form with user selected data items contained in one or more first web pages. The user input is correlated to create an ontology defining relationships between the user selected data items based on the definition of the tabular form. One or more other web pages are accessed, and based on a context of the one or more data items in the first web page being similar to a context of the selected data items in the one or more first web pages, one or more similar data items are extracted from the one or more other web pages. The extracted data items are correlated to each other in accordance with the ontology defining relationships between the user selected data items and are output as a user searchable data structure.

Claims

1. In a computing environment, a method of extracting data from web pages and organizing the extracted data in a user searchable format, the method comprising:

at a graphical user interface, receiving user input defining a tabular form;

at the graphical user interface, receiving user input correlating one or more portions of the form with one or more user selected data items contained in one or more first web pages;

a computer module correlating the user input to create an ontology defining relationships between the user selected data items based on the definition of the tabular form;

a computer module accessing one or more other web pages, and based on a context of the one or more data items in the first web page being similar to a context of the selected data items in the one or more first web pages, extracting one or more similar data items from the one or more other web pages;

a computer module correlating the extracted data items to each other in accordance with the ontology defining relationships between the user selected data items;

a computer module outputting the correlated extracted data items as a user searchable data structure.

2. The method of claim 1, wherein the searchable data structure is at least one of an OWL file, an RDF file, a spreadsheet file, a csv file, or a database file.

3. The method of claim 1, wherein extracting comprises comparing an Xpath pointing to the user selected data items with an Xpath in the one or more other web pages from which the similar data items are extracted.

4. The method of claim 3, further comprising determining context of a node or set of nodes identified by the Xpath pointing to the user selected data items and wherein extracting comprises using the context to determine the extracted data items.

5. The method of claim 4, wherein the context of a node or set of nodes comprises at least one of a symbol or measurement unit.

6. The method of claim 4, wherein the data items are a list or are included in a list, and wherein the context comprises at least one of paragraphs, tabs, or punctuation.

7. The method of claim 1, wherein extracting is performed in a refined fashion by using a plurality of user input data selections.

8. The method of claim 1, further comprising determining that the act of extracting may not be accurate and thereafter prompting the user for input to facilitate the act of correlating extracted data items.

9. The method of claim 8, wherein determining that the act of extracting may not be accurate comprises determining that data is missing from an expected location based on the context.

10. The method of claim 8, wherein determining that the act of extracting may not be accurate comprises determining that a found node does not include an expected context.

11. The method of claim 8, wherein determining that the act of extracting may not be accurate comprises determining that a data correlation fails validation against another ontology.

12. The method of claim 8, wherein determining that the act of extracting may not be accurate comprises determining that a data value violates a range constraint.

13. The method of claim 1, wherein extracting is performed in an automatic mode that does not allow for user interaction during the extraction.

14. The method of claim 1, wherein extracting is performed in a user interactive mode where a user provides user input facilitating extraction during the extraction.

15. The method of claim 14, wherein the user input facilitating extraction during the extraction is provided as a result of a prompt to the user for each correlation.

16. The method of claim 14, wherein the user input facilitating extraction during the extraction is provided as a result of a prompt to the user for each data item.

17. The method of claim 14, wherein the user input facilitating extraction during the extraction is provided as a result of a prompt to the user when an error is detected.

18. The method of claim 1, wherein extracting is performed such that after a first iteration of extracting data, a list of suspected errors is provided to the user whereafter the user provides additional correlation, which is then used by subsequent iterations of the extraction.

19. In a computing environment, a system for extracting data from web pages and organizing the extracted data in a user searchable format, the system comprising:

one or more processors;

a graphical user interface, wherein the graphical user interface;

receives user input defining a tabular form; and

receives user input correlating one or more portions of the form with one or more user selected data items contained in one or more first web pages;

a first computer module implemented using computer executable instructions executed by one or more processors, wherein the first computer module correlates the user input received at the graphical user interface to create an ontology defining relationships between the user selected data items based on the definition of the tabular form;

a second computer module implemented using computer executable instructions executed by one or more processors, wherein the second computer module accesses one or more other web pages, and based on a context of the one or more data items in the first web page being similar to a context of the selected data items in the one or more first web pages, extracts one or more similar data items from the one or more other web pages;

a third computer module implemented using computer executable instructions executed by one or more processors, wherein the second computer module correlates the extracted data items to each other in accordance with the ontology defining relationships between the user selected data items; and

a fourth computer module implemented using computer executable instructions executed by one or more processors, wherein the fourth computer module outputs the correlated extracted data items as a user searchable data structure.