Automatic classification of segmented portions of web pages

• US 8,849,725 B2
• Filed: 08/10/2009
• Issued: 09/30/2014
• Est. Priority Date: 08/10/2009
• Status: Expired due to Fees

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Duan, Lei, Li, Fan, Vadrevu, Srinivas, Velipasaoglu, Emre, Hajela, Swapnil, Chakrabarti, Deepayan
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Traktovenko, Ilya
• Application Number:
  US12/538,776
• Publication Number:
  US 20110035345A1
• Time in Patent Office: 1,877 Days
• Field of Search: 706/12, 706/62
• US Class Current: 706/12
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Exemplary methods and apparatuses are provided which may be used for classifying and indexing segmented portions of web pages and providing related information for use in information extraction and/or information retrieval systems.

Claims

1. A method comprising:

with one or more special purpose computing devices;

for at least one of a plurality of segmented portions obtained from at least one of A plurality of displayable web pages as represented by one or more digital signals of one or more data files, using one or more machine learned models to;

identify one or more feature properties of said segmented portion, wherein at least one of said one or more feature properties affects a presentation of said segmented portion within a rendered version of at least one displayable web page and corresponds to one or more query dependent properties based, at least in part, on one or more historical queries for said at least one of a plurality of displayable web pages;

classify said segmented portion as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

generating one or more digital signals representative of at least part of an index for said plurality of segmented portions, said index being based, at least in part, on said segment type.

2. The method as recited in claim 1, further comprising:

with said one or more special purpose computing devices, training at least one of said one or more machine learned models based, at least in part, on editorial input from a sample set of segmented portions.

3. The method as recited in claim 2, wherein said one or more machine learned models operating in an unsupervised mode identifies one or more digital signals representing a vector space representation as one of said feature properties.

4. The method as recited in claim 1, wherein at least one of said one or more machine learned models operates in an unsupervised mode.

5. The method a recited in claim 1, wherein at least one of said plurality of segmented portions comprises one or more digital signals representing at least one document object model (DOM) node.

6. The method as recited in claim 1, further comprising:

with said one or more special purpose computing devices;

accessing said at least one data file of said at least one of said plurality of displayable web pages; and

identifying one or more digital signals representing said plurality of segmented portions based, at least in part, on an initial set of properties identifiable in one or more digital signals representing said at least one data file.

7. The method as recited in claim 1, wherein at least one other feature property corresponds to a likelihood of a particular user interaction via said segmented portion within said rendered version, said likelihood of user interaction being based, at least in part, on previously obtained web traffic data.

8. The method as recited in claim 1, wherein at least one other feature property corresponds to a likelihood of a particular user viewing response to said segmented portion within said rendered version, said likelihood of user viewing response being based, at least in part, on previously obtained user viewing response studies.

9. The method as recited in claim 1, wherein at least one other feature property corresponds to a distribution of link properties within said rendered version.

10. The method as recited in claim 1, wherein at least one other feature property corresponds to a distribution of user interface features of said rendered version.

11. The method as recited in claim 1, wherein said one or more historical queries comprises one or more top N historical queries retrieving said at least one of a plurality of displayable web pages, and said one or more query dependent properties is based, at least in part, on one or more of a position of match, and/or a quality of match.

12. An apparatus comprising:

memory having stored therein one or more digital signals representing at least one data file of at least one displayable web page;

at least one processing unit coupled to said memory and programmed with instructions to;

for at least one of a plurality of segmented portions obtained from said displayable web page, use one or more machine learned models to;

identify one or more feature properties of said segmented portion, wherein at least one of said one or more feature properties affects a presentation of said segmented portion within a rendered version of at least one displayable web page and corresponds to one or more query dependent properties based, at least in part, on one or more historical queries for said at least one of a plurality of displayable web pages;

classify said segmented portion as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

establish an index for said plurality of segmented portions that is based, at least in part, on said segment type.

13. The apparatus as recited in claim 12, wherein at least one of said one or more machine learned models operates in an unsupervised mode and identifies one or more digital signals representing a vector space representation as one of said feature properties.

14. The apparatus as recited in claim 12, wherein said at least one processing unit is programmed with instructions to identify said plurality of segmented portions based, at least in part, on an initial set of properties identifiable in said at least one data file.

15. The apparatus as recited in claim 12, wherein at least one other feature property corresponds to:

a likelihood of a particular user interaction via said segmented portion within said rendered version, said likelihood of user interaction being based, at least in part, on previously obtained web traffic data;

a likelihood of a particular user viewing response to said segmented portion within said rendered version, said likelihood of user viewing response being based, at least in part, on previously obtained user viewing response studies;

a distribution of link properties within said rendered version;

a distribution of user interface features of said rendered version;

or

some combination thereof.

16. The apparatus as recited in claim 12, wherein said one or more historical queries comprises one or more top N historical queries retrieving said at least one of a plurality of displayable web pages, and said one or more query dependent properties is based, at least in part, on one or more of a position of match or a quality of match.

17. An article comprising:

a non-transitory computer readable medium having computer implementable instructions stored thereon that are executable by one or more processing units in a computing device to;

for at least one of a plurality of segmented portions obtained from at least one of a plurality of displayable web pages as represented by one or more digital signals of one or more data files, use one or more machine learned models to;

identify one or more feature properties of said segmented portion, wherein at least one of said one or more feature properties affects a presentation of said segmented portion within a rendered version of at least one displayable web page and corresponds to one or more query dependent properties based, at least in part, on one or more historical queries for said at least one of a plurality of displayable web pages;

classify said segmented portion as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

maintain an index for said plurality of segmented portions that is based, at least in part, on said segment type.

18. The article as recited in claim 17, wherein at least one of said one or more machine learned models operates in an unsupervised mode and identifies one or more digital signals representing a vector space representation as one of said feature properties.

19. The article as recited in claim 17, wherein at least one other feature property corresponds to:

a likelihood of a particular user interaction via said segmented portion within said rendered version, said likelihood of user interaction being based, at least in part, on previously obtained web traffic data;

a likelihood of a particular user viewing response to said segmented portion within said rendered version, said likelihood of user viewing response being based, at least in part, on previously obtained user viewing response studies;

a distribution of link properties within said rendered version;

a distribution of user interface features of said rendered version;

or

some combination thereof.

20. The article as recited in claim 17, wherein said one or more historical queries comprises one or more top N historical queries retrieving said at least one of a plurality of displayable web pages, and said one or more query dependent properties is based, at least in part, on one or more of a position of match or a quality of match.