Framework for evaluating web search scoring functions
First Claim
1. A computer-implemented method comprising:
- receiving a first query;
selecting, at a data processing apparatus, a first and a second candidate scoring function from a plurality of candidate scoring functions, the selecting comprising;
receiving a plurality of first search results responsive to the first query;
applying each of the candidate scoring functions to the plurality of first search results to determine respective scores for the first search results for the candidate scoring function;
identifying one or more pairs of candidate scoring functions from the candidate scoring functions, each pair comprising a respective alpha candidate scoring function and a respective beta candidate scoring function;
calculating a respective diversity score for each of the pairs of candidate scoring functions, where the diversity score measures diversity between a number of highest ranked first search results ordered according to the respective scores for the alpha candidate scoring function and a number of highest ranked first search results ordered according to the respective scores for the beta candidate scoring function; and
selecting the respective alpha candidate scoring function for a first pair of candidate scoring functions as the first scoring function and the respective beta candidate scoring function for the first pair as the second scoring function, where the first pair of candidate scoring functions is chosen from the one or more pairs of candidate scoring functions based on the respective diversity scores;
presenting the plurality of first search results ordered according to scores from the first scoring function; and
presenting the plurality of first search results ordered according to scores from the second scoring function.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods, systems, and apparatus, including computer program products, for testing web search scoring functions. A query is received. A first and a second scoring function are selected by receiving search results responsive to the query; applying candidate scoring functions to the search results to determine scores for the search results for each candidate scoring function; identifying pairs of the candidate scoring functions, and calculating a diversity score for each of the pairs. A pair of candidate scoring functions is chosen from the one or more pairs of candidate scoring functions based on the diversity scores, and the alpha function is selected as the first scoring function and the beta function is selected as the second scoring function. The plurality of search results are presented in an order according to scores from the first scoring function and are presented in an order according to scores from the second scoring function.
66 Citations
20 Claims
-
1. A computer-implemented method comprising:
-
receiving a first query; selecting, at a data processing apparatus, a first and a second candidate scoring function from a plurality of candidate scoring functions, the selecting comprising; receiving a plurality of first search results responsive to the first query; applying each of the candidate scoring functions to the plurality of first search results to determine respective scores for the first search results for the candidate scoring function; identifying one or more pairs of candidate scoring functions from the candidate scoring functions, each pair comprising a respective alpha candidate scoring function and a respective beta candidate scoring function; calculating a respective diversity score for each of the pairs of candidate scoring functions, where the diversity score measures diversity between a number of highest ranked first search results ordered according to the respective scores for the alpha candidate scoring function and a number of highest ranked first search results ordered according to the respective scores for the beta candidate scoring function; and selecting the respective alpha candidate scoring function for a first pair of candidate scoring functions as the first scoring function and the respective beta candidate scoring function for the first pair as the second scoring function, where the first pair of candidate scoring functions is chosen from the one or more pairs of candidate scoring functions based on the respective diversity scores; presenting the plurality of first search results ordered according to scores from the first scoring function; and presenting the plurality of first search results ordered according to scores from the second scoring function. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system comprising:
-
one or more computers comprising one or more processors and one or more memory devices; the one or more computers programmed to perform operations comprising; receiving a first query; selecting a first and a second candidate scoring function from a plurality of candidate scoring functions, the selecting comprising; receiving a plurality of first search results responsive to the first query; applying each of the candidate scoring functions to the plurality of first search results to determine respective scores for the first search results for the candidate scoring function; identifying one or more pairs of candidate scoring functions from the candidate scoring functions, each pair comprising a respective alpha candidate scoring function and a respective beta candidate scoring function; calculating a respective diversity score for each of the pairs of candidate scoring functions, where the diversity score measures diversity between a number of highest ranked first search results ordered according to the respective scores for the alpha candidate scoring function and a number of highest ranked first search results ordered according to the respective scores for the beta candidate scoring function; and selecting the respective alpha candidate scoring function for a first pair of candidate scoring functions as the first scoring function and the respective beta candidate scoring function for the first pair as the second scoring function, where the first pair of candidate scoring functions is chosen from the one or more pairs of candidate scoring functions based on the respective diversity scores; presenting the plurality of first search results ordered according to scores from the first scoring function; and presenting the plurality of first search results ordered according to scores from the second scoring function. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification