Interactive System for Extracting Data from a Website

• US 20110191381A1
• Filed: 01/29/2010
• Published: 08/04/2011
• Est. Priority Date: 01/29/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zheng, Shuyi, Wen, Ji-Rong, Song, Ruihua, Scott, Matthew Robert
• Application Number:
  US12/696,061
• Publication Number:
  US 20110191381A1
• Time in Patent Office: Days
• US Class Current: 707/793
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking

Abstract

Described is a technology for efficiently labeling a webpage. A wrapper tool labels records of a webpage at the record level. If an existing wrapper exists that is appropriate for labeling a record, the wrapper tool automatically labels that record. For unlabeled records, the tool provides a user interface to label those records, and updates the set of existing wrappers with a new wrapper that is generated based upon the labeling operation; the new wrapper is then applied to any unlabeled records if appropriate for those records. As a result, a user typically needs only to label a relatively few records, with the wrappers generated for those records automatically used to label the other unlabeled records of the webpage.

Claims

1. In a computing environment, a method comprising, generating a wrapper for a record of a webpage based on a label assigned to that record, and adding the wrapper to a set of one or more existing wrappers.

2. The method of claim 1 further comprising, using the wrapper to label another record of the webpage.

3. The method of claim 2 wherein using the wrapper to label another record of the webpage comprises matching a sub-DOM tree corresponding to the other record of the webpage.

4. The method of claim 2 further comprising, selecting a closer matching wrapper to extract data from a record when two or more existing wrappers match the other record.

5. The method of claim 1 further comprising, using the wrapper to label another record of a different webpage.

6. The method of claim 1 further comprising, using a different wrapper in the set of one or more existing wrappers to label at least one other record of the webpage.

7. The method of claim 1 further comprising, revising an incomplete record based upon detection of interaction with a representation of a webpage element corresponding to the incomplete record.

8. The method of claim 1 further comprising, revising an incorrect record based upon detection of interaction with a representation of a webpage element corresponding to the incorrect record.

9. In a computing environment, a system comprising, a record-level wrapper tool, including logic that uses existing wrappers to automatically assign a label to a record of a webpage when an existing wrapper corresponds to that record, and a user interface for interaction with a display of a webpage to label a record of the webpage, the logic configured to update the existing wrappers with a new wrapper that is generated based upon the user interaction to label the record.

10. The system of claim 9 wherein the logic automatically assigns a label to another record of the webpage based upon the new wrapper.

11. The system of claim 9 wherein the logic automatically assigns the label to the record by matching a sub-DOM tree corresponding to that record.

12. The system of claim 9 wherein the user interface provides a menu for labeling a record based upon detection of interaction with a displayed element of the webpage that corresponds to the record.

13. The system of claim 9 wherein the user interface provides a mechanism to command an update the existing wrappers, the logic responding to the command by updating the existing wrappers with the new wrapper that is generated based upon the user interaction to label the one or more records, and with any other new wrapper that is generated based upon other user interaction to label another record.

14. The system of claim 9 wherein the user interface includes a mechanism for selecting an extraction schema.

15. The system of claim 9 wherein the user interface includes a mechanism for invoking an extraction schema editor.

16. The system of claim 9 wherein the user interface displays a set of labeled records to export.

17. One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:

(a) for each record of a webpage, determining whether any existing wrapper applies to that record, and if so, using an applicable wrapper to label that record;

(b) detecting interaction with an element of the webpage to label a record corresponding to that element;

(c) generating a new wrapper based upon the interaction to label the record;

(d) applying the new wrapper to any unlabeled record of the webpage; and

(e) determining whether the webpage contains any unlabeled record, and if so, returning to step (b).

18. The one or more computer-readable media of claim 17 having further computer-executable instructions, comprising, detecting interaction with an element of the webpage to revise a label corresponding to that element.

19. The one or more computer-readable media of claim 17 having further computer-executable instructions, comprising, determining at step (e) that the webpage does not contain any unlabeled record, and in response, selecting a new webpage, and returning to step (a).

20. The one or more computer-readable media of claim 17 having further computer-executable instructions, comprising, exporting the labeled records to another entity.