AUTOMATIC CLASSIFICATION OF SEGMENTED PORTIONS OF WEB PAGES

• US 20110035345A1
• Filed: 08/10/2009
• Published: 02/10/2011
• Est. Priority Date: 08/10/2009
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Duan, Lei, Li, Fan, Chakrabarti, Deepayan, Hajela, Swapnil, Vadrevu, Srinivas, Velipasaoglu, Emre
• Granted Patent: US 8,849,725 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

Exemplary methods and apparatuses are provided which may be used for classifying and indexing segmented portions of web pages and providing related information for use in information extraction and/or information retrieval systems.

Claims

1. A method comprising:

with one or more special purpose computing devices coupled to a memory;

for at least one of a plurality of segmented portions associated with at least one of a plurality of displayable web pages as represented by one or more digital signals of one or more data files stored in said memory, using one or more machine learned models to;

identify one or more feature properties associated with said segmented portion within said one or more data files, and/or otherwise inferable from said one or more data files, and

classify said at least one of said plurality of segmented portions as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

storing one or more digital signals in said memory as part of an index associated with said plurality of segmented portions, said index being based, at least in part, on said segment type.

2. The method as recited in claim 1, wherein classifying said at least one of said plurality of segmented portions further comprises:

establishing segmented portion key-value information associated with at least one of said one or more machine learned models.

3. The method as recited in claim 2, wherein said segmented portion key-value information comprises a segment portion score.

4. The method as recited in claim 2, further comprising:

with said one or more special purpose computing devices, indexing said at least one of said plurality of segmented portions based, at least in part, on at least a portion of said segmented portion key-value information.

5. The method as recited in claim 1, further comprising:

with said one or more special purpose computing devices, combining two or more of said plurality of segmented portions to form a single segmented portion based, at least in part, on at least one feature associated with said two or more of said plurality of segmented portions.

6. The method as recited in claim 1, further comprising:

with said one or more special purpose computing devices, training at least one of said one or more machine learned models based, at least in part, on editorial input associated with a sample set of segmented portions.

7. The method as recited in claim 1, wherein at least one of said one or more machine learned models operates in an unsupervised mode.

8. The method as recited in claim 6, wherein said one or more machine learned models operating in said unsupervised mode identifies one or more digital signals representing a vector space representation as one of said feature properties.

9. The method a recited in claim 1, wherein at least one of said plurality of segmented portions comprises one or more digital signals representing at least one document object model (DOM) node.

10. The method as recited in claim 1, further comprising:

with said one or more special purpose computing devices;

accessing said at least one data file associated with said at least one of said plurality of displayable web pages from said memory; and

identifying one or more digital signals representing said plurality of segmented portions based, at least in part, on an initial set of properties identifiable in one or more digital signals representing said at least one data file.

11. An apparatus comprising:

memory having stored therein one or more digital signals representing at least one data file associated with at least one displayable web page;

at least one processing unit coupled to said memory and programmed with instructions to;

for at least one of a plurality of segmented portions associated with said displayable web page, use one or more machine learned models to;

identify one or more feature properties associated with said segmented portion within said one or more data files, and/or otherwise inferable from said one or more data files, and

classify said at least one of said plurality of segmented portions as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

establish an index in said memory, said index associated with said plurality of segmented portions and being based, at least in part, on said segment type.

12. The apparatus as recited in claim 11, wherein said at least one processing unit is programmed with instructions to establish segmented portion key-value information associated with at least one of said one or more machine learned models, for said at least one of said plurality of segmented portions, and index said at least one of said plurality of segmented portions based, at least in part, on at least a portion of said segmented portion key-value information.

13. The apparatus as recited in claim 11, wherein said at least one processing unit is programmed with instructions to combine two or more of said plurality of segmented portions to form a single segmented portion based, at least in part, on at least one feature associated with said two or more of said plurality of segmented portions.

14. The apparatus as recited in claim 11, wherein at least one of said one or more machine learned models operates in an unsupervised mode and identifies one or more digital signals representing a vector space representation as one of said feature properties.

15. The apparatus as recited in claim 11, wherein said at least one processing unit is programmed with instructions to identify said plurality of segmented portions based, at least in part, on an initial set of properties identifiable in said at least one data file.

16. An article comprising:

a computer readable medium having computer implementable instructions stored thereon which if implemented by one or more processing units in a computing device operatively transform the computing device into a special purpose device to;

for at least one of a plurality of segmented portions associated with at least one of a plurality of displayable web pages as represented by one or more digital signals of one or more data files stored in a memory, use one or more machine learned models to;

identify one or more feature properties associated with said segmented portion within said one or more data files, and/or otherwise inferable from said one or more data files, and

classify said at least one of said plurality of segmented portions as being at least one of a plurality of segment types based, at least in part, on said one or more identified feature properties; and

establish one or more digital signals representing an index within a memory coupled to said one or more processing units, said index associated with said plurality of segmented portions and being based, at least in part, on said segment type.

17. The article as recited in claim 16, wherein said computer implementable instructions, if implemented by said one or more processing units in the computing device operatively transform the computing device into said special purpose device to establish one or more digital signals representing segmented portion key-value information associated with at least one of said one or more machine learned models, for said at least one of said plurality of segmented portions, and index at least one of said plurality of segmented portions based, at least in part, on at least a portion of said segmented portion key-value information.

18. The article as recited in claim 16, wherein said computer implementable instructions, if implemented by said one or more processing units in the computing device operatively transform the computing device into said special purpose device to combine two or more of said plurality of segmented portions to form a single segmented portion based, at least in part, on at least one feature associated with said two or more of said plurality of segmented portions.

19. The article as recited in claim 16, wherein at least one of said one or more machine learned models operates in an unsupervised mode and identifies one or more digital signals representing a vector space representation as one of said feature properties.

20. The article as recited in claim 16, wherein said computer implementable instructions, if implemented by said one or more processing units in the computing device operatively transform the computing device into said special purpose device to identify said plurality of segmented portions based, at least in part, on one or more digital signals representing an initial set of properties identifiable in said at least one data file.