Related links recommendation

• US 8,412,726 B2
• Filed: 06/03/2010
• Issued: 04/02/2013
• Est. Priority Date: 06/03/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Yan, Jun, Liu, Ning, Chen, Zheng, Ji, Lei, Wang, Jiulong, Liang, Xiao
• Primary Examiner(s): NGUYEN, CAM LINH T
• Application Number:
  US12/793,047
• Publication Number:
  US 20110302155A1
• Time in Patent Office: 1,034 Days
• Field of Search: 707/767, 707/754, 707/758, 707/780
• US Class Current: 707/767
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

The related links recommendation technique described herein employs combined collaborative filtering to recommend related web pages to users. The technique creates multiple collaborative filters which are combined in order to create a combined collaborative filter to recommend web pages similar to a given web page to a user. One query-based collaborative filter is created by using query search clicks (e.g., user input device selection actions on search results returned in response to a search query). Another user-behavior-based collaborative filter is created by using query search clicks and user clicks while browsing websites (e.g., user input device selection actions while a user is browsing websites). Lastly, another content-based collaborative filter based on similar content of web pages is created by finding web pages with similar content.

Claims

1. A computer-implemented process for recommending web pages to a user based on query data and user behavior data, comprising:

using a computing device for;

inputting query log data and user click data;

computing a query-based collaborative filter based on the query data, comprising pairs of compared websites and a confidence value, wherein the confidence value is obtained by determining a similarity value for each pair of websites that was clicked by a user for a same query and normalizing the similarity value;

computing a user-based collaborative filter based on the user behavior data, comprising pairs of compared websites with the most frequent number of co-clicks by users and a confidence score for each pair, wherein the confidence value is obtained by determining a similarity value of each pair of websites that were co-clicked by users during searching and browsing during the same user sessions and normalizing the similarity value;

combining the query-based collaborative filter and user-based collaborative filter to create a combined collaborative filter, that computes the similarity between pairs of websites, for recommending web pages related to a given web page returned in response to a search query.

2. The computer-implemented process of claim 1, further comprising using the combined collaborative filter to recommend to a user one or more web pages related to a given web page.

3. The computer-implemented process of claim 1, wherein computing the query-based collaborative filter further comprises:

identifying websites in the query log which have the most frequent co-clicks for each query;

creating a query index of the identified websites that have the most frequent co-clicks for each query;

creating a query feature matrix comprising the query that lead to a given website and the number of times the website was clicked;

comparing pairs of websites for each query in the query feature matrix to assess similarity between the websites clicked for a given query and assigning each pair of the compared websites a confidence value based on the similarity between them;

creating a collaborative query filter comprising the pairs of compared websites and the confidence value based on their similarity.

4. The computer-implemented process of claim 3, wherein a cosine similarity function is used to determine similarity between pairs of websites.

5. The computer-implemented process of claim 4 further comprising removing noise from the query feature matrix by deleting websites that do not have a sufficient number of user clicks for a given query prior to determining similarity.

6. The computer-implemented process of claim 3 further comprising computing features of the query feature matrix by employing a min-hash function.

7. The computer-implemented process of claim 1, wherein computing the user-based collaborative filter further comprises:

creating a user-based index comprising websites which have the most frequent co-clicks by users based on the user click data; and

creating a user-based feature matrix comprising pairs of websites with the most frequent number of co-clicks by users;

computing a similarity between the pairs of websites with the most frequent number of co-clicks by users and creating a confidence score based on the similarity; and

creating the user-based collaborative filter comprising the pairs of websites with the most frequent number of co-clicks by users and the confidence score for each pair.

8. The computer-implemented process of claim 7, wherein a cosine similarity function is used to determine similarity between websites.

9. The computer-implemented process of claim 7, further comprising removing websites pairs that do not have a sufficient number of co-clicks prior to determining similarity.

10. The computer-implemented process of claim 1 wherein the user behavior data further comprises user session data comprising, user ID, queries submitted by each user ID and clicked URLs by user ID over a given time session.

11. The computer-implemented process of claim 1 wherein the query log data further comprises a set of search queries, web pages returned in response to each search query in the set of search queries and a score as to how many times each of the web pages was clicked by a user for each search query.

12. A computer-implemented process for recommending related web pages to users when they are browsing websites, comprising:

using a computing device for;

for each of a set of web pages, associated queries and user behavior data, searching for web pages with the most frequent clicks for the same query to create a feature matrix for query-based collaborative filtering comprising pairs of compared websites and a confidence value, wherein the confidence value is obtained by determining a similarity value for each pair of websites that was clicked by a user for a same query and normalizing the similarity value;

for each of a set of web pages, associated queries and user behavior data, searching for web pages which have the most frequent co-clicks by users to create a feature matrix for user-based collaborative filtering, comprising pairs of compared websites with the most frequent number of co-clicks by users and a confidence score for each pair, wherein the confidence score is obtained by determining a similarity value of each pair of web pages that were co-clicked by users during searching and browsing during the same user sessions and normalizing the similarity value;

creating a combined feature matrix by combining the feature matrix for query-based collaborative filtering and user-based collaborative filtering by combining the feature matrix for query-based collaborative filtering and the feature matrix of the user-based collaborative filtering and adding the confidence scores of each pair of web pages to create a final confidence score for each pair of web pages; and

using the combined feature matrix and the final confidence score that computes the similarity between pairs of websites, to recommend a web page related to another web page.

13. The computer-implemented process of claim 12, wherein creating a feature matrix for query-based collaborative filtering further comprises employing a minhash function to create features for the query-based feature matrix.

14. The computer-implemented process of claim 12, wherein the query-based collaborative filter is based on input query log data, comprising submitted queries, and websites clicked by users for each submitted query.

15. The computer-implemented process of claim 12, wherein the user-based collaborative filter is based on user session data comprising:

user ID and websites clicked by user ID over a given time period.

16. A system for recommending one or more related websites to a user based on a website returned in response to a search query or while browsing, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to,

compute a query-based collaborative filter based on the number of co-clicks on a websites by users for a given query;

compute a user-based collaborative filter based on the number of co-clicks by users on a given website during searching and browsing during the same user sessions;

compute a content-based collaborative filter based on the similarity of content of website pairs;

combine the query-based collaborative filter, user-based collaborative filter and content-based filter to create a combined collaborative filter, that computes the similarity between pairs of websites, for recommending web pages related to a web page returned in response to a search query.

17. The system of claim 16, further comprising computing the query-based collaborative filter, the user-based collaborative filter and the content-based collaborative filter by:

for each of a set of web pages,

for a given web page, searching for web pages with the most similar contents to create a feature matrix for the content-based collaborative filter;

for a given web page, searching for the web pages which have the most frequent co-clicks by users during searching and browsing during the same user sessions to create a feature matrix for the user-based collaborative filter;

for a given web page, searching for web pages with the most frequent co-clicks for the same query to create a feature matrix for the query-based collaborative filter.

18. The system of claim 17 further comprising a module for:

computing a first confidence score for the feature matrix for the content-based collaborative filter using similarity scores of each pair of web pages,

computing a second a confidence score for the feature matrix for user-based collaborative filter using similarity scores of each pair of web pages; and

computing a third confidence score for the feature matrix for the query-based collaborative filter using similarity scores of each pair of web pages.

19. The system of claim 18, further comprising using the first, second and third confidence scores to merge the content-based collaborative filter, user-based collaborative filter, query-based collaborative filter to create a combined filter for use in recommending one or more web pages related to a web page retrieved in response to a search query.

20. The system of claim 18 wherein the content-based collaborative filter, user-based collaborative filter, query-based collaborative filter are weighted when merging to create the combined collaborative filter to created a weighted combined collaborative filter.