Method and system for identifying targeted data on a web page

• US 20070073758A1
• Filed: 09/23/2005
• Published: 03/29/2007
• Est. Priority Date: 09/23/2005
• Status: Active Grant

• Current Assignee: Pronto, Inc. (Match Group, Inc.)
• Original Assignee: Redcarpet Incorporated
• Inventors: Perry, Bradley, Marriott, Daniel, Perry, Nancy
• Granted Patent: US 7,912,755 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing

Abstract

A method and system is provided that in a fully automated manner crawls web sites and identifies specific types of web pages, then extracts targeted data from those web pages. One or more text nodes containing product-related information on a first web page are first identified, and the locations of those text nodes are described using one or more vectors. The vectors are then analyzed to identify one or more patterns and to generate a model from those patterns that discriminates between text nodes that contain product-related information and text nodes that do not contain product-related information on a second web page. The model can then be used to crawl web sites to identify and extract targeted data, or the model can be installed on a user'"'"'s computer to identify and extract targeted information from web sites as the user is browsing.

Claims

1. A method for identifying product-related information on a web page, the method comprising:

a. identifying one or more text nodes containing product-related information on a first web page;

b. using one or more vectors to describe the locations of the text nodes containing potential product-related information on the first web page;

c. analyzing one or more of the vectors to identify one or more patterns; and

d. generating a model that discriminates between text nodes that contain product-related information and text nodes that do not contain product-related information on a second web page.

2. The method of claim 1, wherein the first and second web pages are written using HTML programming language.

3. The method of claim 1, wherein the vectors are 4-place vectors.

4. The method of claim 3, wherein one field of the vectors represents the text of the text node.

5. The method of claim 3, wherein one field of the vectors represents the anonymous HTML tag path leading to the text node.

6. The method of claim 3, wherein one field of the vectors represents the indexed HTML tag path leading to the text node.

7. The method of claim 3, wherein one field of the vectors represents the attribute-annotated HTML tag path leading to the text node.

8. The method of claim 1, wherein the model includes one or more symbolic expressions that represent the pattern of text node locations.

9. The method of claim 1, further comprising using the model to crawl a plurality of web pages to identify and extract product-related information.

10. The method of claim 1, further comprising providing the model to a client application on a user'"'"'s computer.

11. The method of claim 10, further comprising using the model to identify and extract product-related information from web pages as they are displayed to the user.

12. A system comprising:

a first computer having a first computer-readable medium containing

a copy of source code for a first web page,

one or more first computer programs configured to parse the copy of the source code to identify all text nodes and analyze the text nodes to identify any text nodes that contain product-related information;

one or more second computer programs configured to generate vectors describing the location of the text nodes containing product-related information, analyze one or more of the vectors to identify one or more patterns and generate one or more models that discriminate between text nodes that contain product-related information and text nodes that do not contain product-related information on a second web page; and

a second computer coupled to the first computer having a second computer-readable medium,

wherein the one or more models are transmitted to the second computer, stored in the second computer-readable medium, and used to identify and extract information about one or more products available for sale on one or more merchant web pages.

13. The system of claim 12, wherein the web pages are written using HTML programming language.

14. The system of claim 12, wherein the vectors are 4-place vectors.

15. The system of claim 14, wherein one field of the vectors represents the text of the text node.

16. The system of claim 14, wherein one field of the vectors represents the anonymous HTML tag path leading to the text node.

17. The system of claim 14, wherein one field of the vectors represents the indexed HTML tag path leading to the text node.

18. The system of claim 14, wherein one field of the vectors represents the attribute-annotated HTML tag path leading to the text node.

19. The system of claim 12, wherein the model includes one or more symbolic expressions that represent the pattern of text node locations.

20. The system of claim 12, wherein the model is used to crawl web pages to identify and extract product-related information.

21. A method for identifying and extracting product-related information from a web page, the method comprising:

a. locating potential product-related text nodes on a first web page;

b. creating a representation space that describes the potential product-related text nodes on the first web page;

c. analyzing the representation space to identify one or more patterns;

d. using the patterns to generate one or more models that discriminate between product-related text nodes and non product-related text nodes on a second web page.