CLASSIFYING FUNCTIONS OF WEB BLOCKS BASED ON LINGUISTIC FEATURES

• US 20080270334A1
• Filed: 04/30/2007
• Published: 10/30/2008
• Est. Priority Date: 04/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Ma, Wei-Ying, Xie, Xing, Xiao, Xiangye
• Granted Patent: US 7,895,148 B2
• Time in Patent Office: Days
• US Class Current: 706/20
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A classification system trains a classifier to classify blocks of the web page into various classifications of the function of the block. The classification system trains a classifier using training web pages. To train a classifier, the classification system identifies the blocks of the training web pages, generates feature vectors for the blocks that include a linguistic feature, and inputs classification labels for each block. The classification system learns the coefficients of the classifier using any of a variety of machine learning techniques. The classification system can then use the classifier to classify blocks of web pages.

Claims

1. A method in a computing device for classifying a block of a document based on its function, the method comprising:

identifying (203) blocks of training documents;

for each identified block,

receiving (207) a classification label for the identified block indicating its function; and

generating (206) a feature vector for the identified block, the feature vector including a linguistic feature;

training a classifier using the feature vectors and classification labels; and

classifying a block of a document based on its function by applying the trained classifier to a feature vector for the block.

2. The method of claim 1 wherein the linguistic feature is based on parts of speech.

3. The method of claim 2 wherein the linguistic feature includes a count of words within the text of a block for various parts of speech.

4. The method of claim 1 wherein the linguistic feature is selected from a group consisting of parts of speech, capitalization, symbols, and named entities.

5. The method of claim 1 wherein the feature vector further includes a term feature.

6. The method of claim 1 wherein the feature vector further includes a layout feature.

7. The method of claim 1 wherein the classifications include information and non-information.

8. The method of claim 1 wherein the classifications include information, interaction, navigation, advertisement, and other.

9. The method of claim 1 including displaying a document by giving preference to displaying blocks of a certain classification.

10. The method of claim 1 including displaying search results in an order that is based at least in part on analysis of blocks with a certain classification.

11. The method of claim 1 including when a document will not fit on a device, displaying blocks of the document giving preference to blocks with a certain classification.

12. A computing device generating a classifier for classifying blocks of web pages into functional classifications, comprising:

a training data store that includes training web pages, the web pages having blocks;

a block identification component that identifies blocks within a web document;

a feature generation component that generates a feature vector for a block of a web page, the feature vector including layout features and linguistic features;

a labeler component that inputs a classification label for each block of each training web page; and

a component that learns coefficients of a classifier using the feature vectors of the training web page and the label classifications and stores the coefficients in a classifier coefficients store.

13. The computing device of claim 12 including a classifier component that inputs a block of a web page, generates a feature vector for the block, and classifies the block by applying a classifier with the learned coefficients to the feature vector.

14. The computing device of claim 12 wherein the linguistic feature is based on parts of speech.

15. The computing device of claim 14 wherein the linguistic feature includes a count of the parts of speech of the words within the text of a block.

16. The computing device of claim 12 wherein the linguistic feature is selected from a group consisting of parts of speech, capitalization, symbols, and named entities.

17. The computing device of claim 12 wherein the feature vector further includes a layout feature.

18. The computing device of claim 12 wherein the classifications include information and non-information.

19. A computer-readable storage medium encoded with instructions for controlling a computing device to classify blocks of web pages based on their function, by a method comprising:

identifying blocks of training web pages;

for each identified block,

receiving a classification label for the identified block, the classifications including information and non-information; and

generating a feature vector for the identified block, the feature vector including a linguistic feature and a layout feature, the linguistic feature based on parts of speech of words within the text of the block;

training a classifier using the feature vectors and classification labels; and

classifying a block of a web page as information or non-information by applying the trained classifier to a feature vector for the block.

20. The computer-readable medium of claim 19 wherein an additional linguistic feature is selected from a group consisting of capitalization, symbols, and named entities.