EXTRACTING DATA CONTENT ITEMS USING TEMPLATE MATCHING

• US 20090063500A1
• Filed: 08/31/2007
• Published: 03/05/2009
• Est. Priority Date: 08/31/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Yi, Gao, Hong, Tan, Lei, Qian, Richard, Zhai, Yanhong
• Granted Patent: US 7,765,236 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/81 Indexing, e.g. XML tags; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions

Abstract

Systems and methods for extracting data content items from a web page are provided. A template is created by labeling data content items of interest associated with a web page and generating a template Document Object Model (DOM) tree based on the labeled web page. DOM trees are also generated for additional web pages that contain data content items for which extraction may be desired. These DOM trees are compared to the template DOM tree to determine alignment there between. The aligned data content items may then be extracted from the additional web pages and indexed, as desired. Labeling the data content items of interest prior to generating a template DOM tree allows for the desired data content items to be specified and more accurately extracted from related and/or similarly structured web pages.

Claims

1. One or more computer storage media having computer-executable instructions embodied thereon for performing a method for extracting data content items from web pages, the method comprising:

receiving a first web page having one or more data content items associated therewith;

receiving an indication to label at least one of the data content items associated with the first web page;

generating a Document Object Model (DOM) tree associated with the first web page, the DOM tree having a node associated with each data content item;

labeling the node of the DOM tree associated with the at least one indicated data content item to generate a template DOM tree;

comparing the template DOM tree with a DOM tree associated with a second web page to determine alignment there between; and

if it is determined that a node of the DOM tree associated with the second web page aligns with the labeled node associated with the template DOM tree, extracting a data content item from the second web page that is associated with the aligned node of the DOM tree.

2. The computer storage media of claim 1, wherein the one or more data content items associated with the first web page comprise items associated with an individual or organization data listing.

3. The computer storage media of claim 1, wherein one or more data content items associated with the first web page comprise one or audio content items, video content items, images, text, or any combination thereof.

4. The computer storage media of claim 1, wherein at least two of the data content items associated with the template DOM tree have a repeated pattern.

5. The computer storage media of claim 4, wherein labeling the node of the DOM tree associated with the at least one indicated data content item to create a template DOM tree further comprises labeling a node of the template DOM tree associated with one of the at least two data content items having a repeated pattern with a repeat label.

6. The computer storage media of claim 1, wherein comparing the template DOM tree with a DOM tree associated with a second web page to determine alignment there between comprises measuring the top-down distance between one node of the template DOM tree and a corresponding node of the DOM tree associated with a second web page.

7. The computer storage media of claim 1, wherein comparing the template DOM tree with a DOM tree associated with a second web page to determine alignment there between comprises employing partial tree alignment to insert one or more nodes into the template DOM tree.

8. The computer storage media of claim 1, wherein the method further comprises indexing the extracted data content item.

9. A computer system embodied on at least one computer storage media having computer-executable instructions embodied thereon for performing a method for extracting data content items from web pages, the system comprising:

a receiving component configured for receiving a plurality of web pages, each web page having at least one data content item associated therewith;

a Document Object Model (DOM) tree generating component configured for generating a DOM tree associated with one or more of the received web pages;

a labeling component configured for labeling at least one node associated with a generated DOM tree in accordance with a received labeling indication;

a comparing component configured for comparing a first DOM tree having at least one labeled node associated therewith with a second DOM tree; and

an extracting component configured for extracting at least one data content item associated with the second DOM tree in accordance with the at least one labeled note associated with the first DOM tree.

10. The computer system of claim 9, wherein the at least one data content item comprises at least one item associated with an individual or organization listing, an audio content item, a video content item, an image, text, or any combination thereof.

11. The computer system of claim 9, further comprising a search request receiving component configured for receiving a search request for at least one satisfying data content item.

12. The computer system of claim 11, further comprising a presenting component configured for presenting the at least one satisfying data content item.

13. The computer system of claim 9, further comprising an aligning component configured for aligning the matching nodes of the first DOM tree and the second DOM tree.

14. The computer system of claim 9, wherein the comparing component is further configured for measuring the top-down distance between the first DOM tree and the second DOM tree.

15. The computer system of claim 9, wherein the comparing component is further configured to utilize partial tree alignment algorithm to insert at least one node into the first DOM tree.

16. A method in a computing environment for extracting data content items from a web page, at least two of the data content items having a repeated pattern, the method comprising:

receiving a first web page having a plurality of data content items associated therewith;

receiving an indication to label at least two of the plurality of data content items, wherein the at least two of the plurality of data content items have a repeated pattern;

generating a Document Object Model (DOM) tree associated with the first web page, the DOM tree having a node associated with each of the plurality of data content items;

labeling the nodes of the DOM tree to create a template DOM tree, wherein a node associated with one of the at least two data content items having a repeated pattern is labeled as a repeat node;

comparing the template DOM tree with a DOM tree associated with a second web page to determine alignment there between; and

if it is determined that a node of the DOM tree associated with the second web page aligns with the labeled node associated with the template DOM tree, extracting one or more data content items from the second web page that are associated with the aligned repeat node of the DOM tree.

17. The method of claim 16, wherein the one or more data content items associated with the first web page comprise items associated with an individual or organization listing.

18. The method of claim 16, wherein the one or more data content items associated with the first web page comprise one or audio content items, video content items, images, text, or any combination thereof.

19. The method of claim 16, wherein comparing the template DOM tree with a DOM tree associated with a second web page comprises measuring the top-down distance between one node of the template DOM tree and a corresponding node of the DOM tree associated with a second web page.

20. The method of claim 16, wherein comparing the template DOM tree with a DOM tree associated with a second web page comprises using partial tree alignment to insert one or more nodes into the template DOM tree.