System and method for indexing web content using click-through features

• US 20070255689A1
• Filed: 04/28/2006
• Published: 11/01/2007
• Est. Priority Date: 04/28/2006
• Status: Active Grant

• Current Assignee: Pinterest, Inc.
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Sun, Gordon, Zheng, Zhaohui
• Granted Patent: US 7,647,314 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The system and method of the present invention allows for the determination of the relevance of a content item to a query through the use of a machine learned relevance function that incorporates click-through features of the content items. A method for selecting a relevance function to determine a relevance of a query-content item pair comprises generating training set having one or more query-URL pairs labeled for relevance based on their click-through features. The labeled query-URL pairs are used to determine the relevance function by minimizing a loss function that accounts for click-through features of the content item. The computed relevance function is then applied to the click-though features of unlabeled content items to assign relevance scores thereto. An inverted click-through index of query-score pairs is formed and combined with the content index to improve relevance of search results.

Claims

1. A method for indexing content items based on click-through features, the method comprising:

generating a training set comprising one or more query-content item pairs, wherein a given query-content item pair has one or more click-through features associated therewith;

labeling one or more query-content item pairs in the training set by assigning click score thereto based on the one or more click-through features thereof;

determining a click score function based on the click scores of the labeled query-content item pairs and the click-through features thereof;

applying the click score function to a plurality of unlabeled query-content item pairs to determine click scores thereof based on the one or more click-through features of the unlabeled query-content item pairs.

2. The method of claim 1 further comprising the step of identifying a content item using a Uniform Resource Locator (“

URL”

).

3. The method of claim 1 wherein labeling a given content item in the training set comprises providing a given query-content item pair to a human judge to assign a click score.

4. The method of claim 1 wherein determining the click score function comprises minimizing a difference between a click score for the given query-content item pair and an output of the click score function.

5. The method of claim 1 wherein determining the click score function comprises using a loss function selected from a set of loss functions including least mean square loss function, a query pairwise loss function and a loss function that compares a desired ranking order with an output ranking order.

6. The method of claim 1 comprising storing the determined click score function for application to an unlabeled query-content item pair.

7. The method of claim 1 comprising generating an inverted click-through index of the unlabeled content items and the associated query-score pairs, wherein a key to the index is a URL of the content item.

8. The method of claim 7 comprising the step of combing the inverted click-through index with a content index by associating the query-score pairs with the content items identified in the content index.

9. The method of claim 8 comprising the steps of:

retrieving one or more content items in a result set in response to receipt of the query from the user;

for a given content item in the result set, determining a click-through feature vector for the given content item;

applying the click score function to the click-through feature vector for the given content item to generate a click score for the given content item;

generating one or more <

query, content-item, click-score>

data structures for the given content item;

indexing one or more <

query, click-score>

pairs for the content item from the <

query, content-item, click-score>

data structures.

10. A system for indexing and searching content items based on its click-through features, the system comprising:

an index component operative to determine a click score function based on a training set of labeled query-content item pairs and the click-through features thereof, assign click scores to a plurality of unlabeled query-content item pairs through application of the click score function to the one or more click-through features and generate an inverted click-through index of the unlabeled content items and the associated query-score pairs;

a relevance engine operative to receive one or more query scores for one or more content items and generate one or more relevance scores therefore; and

a search engine operative to retrieve one or more content items in a result set in response to receipt of the query from the user and order the content items in the result set according to the relevance scores from the relevance engine.

11. The system of claim 10 wherein a Uniform Resource Locator (URL) of a content item is used as a key to locate the given content item the inverted click-through index.

12. The system of claim 10 wherein the training set of labeled query-content items comprises a plurality of query-content items having click scores assigned thereto by a human judge.

13. The system of claim 10 wherein the click score function is determined by minimizing difference between click score for the given query-content item pair in the training set and the output of the click score function.

14. The system of claim 10 wherein the click score function is determined using a loss function selected from a set of loss functions including a least mean square loss function, a query pairwise loss function and a loss function that compares a desired ranking order with an output ranking order.

15. The system of claim 10 wherein the index component is operative to store the determined click score function for application to an unlabeled query-content item pair.

16. The system of claim 10 wherein the index component is operative to combine the inverted click-through index with a content index by associating the query-score pairs with the content items in the content index.