System and method for automated optimization of search result relevance
First Claim
Patent Images
1. A method for automating the optimization of search results displayed in a search Web page, the method comprising:
- receiving search results provided from a search engine to a user according to a search term, wherein the search results are selected and ranked by a relevance schema;
collecting data that represents a performance of each of the received search results, the collected data quantifying interactions of various users with the search results, the collected data originating from at least one of a plurality of various sources having various types of valuations, the search results and collected data being initially incomparable against each other absent normalization, wherein the various sources reflect data of quantified interactions of the various users with the search results;
combining the collected data by a normalization procedure that generates performance data for each of the search results, wherein the performance data reflects behavior of the various users, and wherein the normalization procedure comprises;
(1) aggregating the collected data from the various sources;
(2) compiling various types of valuations, associated with the aggregated data, into common measurements;
(3) weighting the common measurements based on relative importance of the various sources, wherein the relative importance is a reflection of a value of each of the various sources as a predictor of relevance of the search results; and
(4) normalizing the weighted common measurements to determine performance data associated with each of the search results such that the search results are comparable against each other and such that the normalized performance data is comparable against expected performance data of the search results;
determining whether the expected performance data is below the normalized performance data by comparing the normalized performance data to the expected performance data for each of the search results;
when the normalized performance data of a search result of the search results is below the expected performance data associated therewith, identifying the search result as underperforming and diagnosing the underperforming search result based on results of the comparison between the normalized performance data and the expected performance data; and
updating the relevance schema based on the diagnosis such that operation of a search engine that provided the search results is adjusted to improve relevance of subsequent search results.
2 Assignments
0 Petitions
Accused Products
Abstract
A method, system, and computer-accessible medium are provided for automating the optimization of search result relevance in a search engine. The system and method continually collect data that represent various aspects of how a search result is performing and compare that performance data to the expected performance for the search result. The system and method further diagnose the possible causes of under performing results and automatically adjust the search engine operation to optimize the search result relevance.
54 Citations
66 Claims
-
1. A method for automating the optimization of search results displayed in a search Web page, the method comprising:
-
receiving search results provided from a search engine to a user according to a search term, wherein the search results are selected and ranked by a relevance schema; collecting data that represents a performance of each of the received search results, the collected data quantifying interactions of various users with the search results, the collected data originating from at least one of a plurality of various sources having various types of valuations, the search results and collected data being initially incomparable against each other absent normalization, wherein the various sources reflect data of quantified interactions of the various users with the search results; combining the collected data by a normalization procedure that generates performance data for each of the search results, wherein the performance data reflects behavior of the various users, and wherein the normalization procedure comprises; (1) aggregating the collected data from the various sources; (2) compiling various types of valuations, associated with the aggregated data, into common measurements; (3) weighting the common measurements based on relative importance of the various sources, wherein the relative importance is a reflection of a value of each of the various sources as a predictor of relevance of the search results; and (4) normalizing the weighted common measurements to determine performance data associated with each of the search results such that the search results are comparable against each other and such that the normalized performance data is comparable against expected performance data of the search results; determining whether the expected performance data is below the normalized performance data by comparing the normalized performance data to the expected performance data for each of the search results; when the normalized performance data of a search result of the search results is below the expected performance data associated therewith, identifying the search result as underperforming and diagnosing the underperforming search result based on results of the comparison between the normalized performance data and the expected performance data; and updating the relevance schema based on the diagnosis such that operation of a search engine that provided the search results is adjusted to improve relevance of subsequent search results. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
23. An automated search result optimization system implemented in a computer that provides search results to a user, comprising:
-
an input component to receive input data representing a performance of a search result generated by a search engine that is executed on the computer for a search term, the input data originating from a plurality of various sources; a data collection processor to collect the input performance data from the plurality of various sources wherein the input performance data collected includes implicit performance data that describes an aggregation of various users interactions during various sessions with the search result, wherein the input performance data from one of the plurality of various sources is initially incomparable against the input performance data from another of the plurality of various sources and the search results are initially incomparable against each other absent normalization, wherein the plurality of various sources reflecting collected input performance data; a comparison processor to compare the collected input performance data of the search result to an expected performance data of the search result, wherein comparing comprises normalizing input performance data by the following steps; (1) aggregating the collected input performance data from the plurality of various sources; (2) compiling various types of valuations, associated with the aggregated collected input performance data, into common measurements; and (3) normalizing the common measurements to determine common input performance data associated with each of the search results such that search results are comparable against each other and such that normalized input performance data of search results is comparable against expected performance data of the search results; a diagnostic processor to determine, based on the comparison, whether the collected input performance data diverges from the expected performance data by a quantified threshold, and if so, identify the search result as underperforming and diagnose the underperforming search result utilizing results of the comparison to select from a set of predefined corrective actions, wherein one of the set of corrective actions is implementing an adjustment processor; and the adjustment processor to automatically adjust operation of the search engine, wherein the adjusted operation of the search engine improves the performance of the search result by tailoring a ranking of subsequent search results in accordance with the collected input performance data, and wherein an increased weight is attached to the collected input performance data from one or more of the various sources based on the implicit performance data thereby influencing the automatic adjustment of the operation of the search engine. - View Dependent Claims (24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44)
-
-
45. One or more computer-accessible media having instructions stored on the media for facilitating the automated optimization of a search result in a search result user interface, the instructions comprising:
-
receiving search results provided by a search engine in response to an inputted search term from a user; collecting data that represents a performance of each search result of the search results from at least one of a plurality of various sources of performance data that includes implicit data collected by quantifying interactions of various users during various sessions with each of the search results, wherein the performance data from one of the plurality of various sources is initially incomparable against the performance data from another of the plurality of various sources and the search results are initially incomparable against each other absent normalization, wherein the plurality of various sources reflecting the performance data quantifying interactions of various users; normalizing the collected performance data in accordance with a relative importance associated with each of the various sources of the performance data, wherein normalizing comprises formatting the performance data in a common measurement that is associated with each of the search results such that the search results are comparable against each other and such that normalized performance data of search results is comparable against expected performance data of search results; comparing the normalized performance data against the expected performance data for each search result; based on the comparison, determining whether the normalized collected performance data diverges from the expected performance data by a quantified threshold and identifying a search result, of the search results, as underperforming when it is associated with divergent normalized collected performance data; diagnosing at least one possible cause for the underperforming search result based on a result of the comparison between the normalized performance data and the expected performance data; utilizing the at least one possible cause to select from a set of predefined corrective actions, wherein one of the set of corrective actions is adjusting an operation of the search engine accordance with the normalized collected performance data. - View Dependent Claims (46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66)
-
Specification