Removing substitution rules based on user interactions
First Claim
1. A computer-implemented method comprising:
- identifying a plurality of revised search queries associated with a substitution rule, wherein the substitution rule is used to generate the revised search queries from a plurality of original search queries such that each of the revised search queries includes a substitute query term for an original query term that is included in each of the original search queries;
for each of the revised search queries associated with the substitution rule, accessing query log data that identifies (i) a set of search results that was presented to a user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) a particular, user-selected search result that was selected by the user from among the set of search results;
determining a quality metric for the substitution rule based at least on the query log data that identifies, for each of the revised search queries associated with the substitution rule, (i) the set of search results that was presented to the user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) the particular, user-selected search result that was selected by the user from among the set of search results;
determining that the quality metric does not satisfy a threshold; and
in response to determining that the quality metric does not satisfy the threshold, removing the substitution rule.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for removing substitution rules. According to one implementation, a method includes identifying a plurality of revised search queries associated with a substitution rule, wherein the substitution rule is used to generate the revised search queries from each of a plurality of original search queries to include a substitute query term for an original query term included in the original search queries; for each of the revised search queries, accessing query log data that identifies (i) a set of search results generated by the respective revised search query, and (ii) a particular, user-selected search result in the set of search results; determining a quality metric for the substitution rule using the query log data; determining that the quality metric does not satisfy a threshold; and in response to determining that the quality metric does not satisfy the threshold, removing the substitution rule.
107 Citations
24 Claims
-
1. A computer-implemented method comprising:
-
identifying a plurality of revised search queries associated with a substitution rule, wherein the substitution rule is used to generate the revised search queries from a plurality of original search queries such that each of the revised search queries includes a substitute query term for an original query term that is included in each of the original search queries; for each of the revised search queries associated with the substitution rule, accessing query log data that identifies (i) a set of search results that was presented to a user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) a particular, user-selected search result that was selected by the user from among the set of search results; determining a quality metric for the substitution rule based at least on the query log data that identifies, for each of the revised search queries associated with the substitution rule, (i) the set of search results that was presented to the user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) the particular, user-selected search result that was selected by the user from among the set of search results; determining that the quality metric does not satisfy a threshold; and in response to determining that the quality metric does not satisfy the threshold, removing the substitution rule. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A non-transitory computer-readable medium storing software having stored thereon instructions, which, when executed by one or more computers, cause the one or more computers to perform operations of:
-
identifying a plurality of revised search queries associated with a substitution rule, wherein the substitution rule is used to generate the revised search queries from a plurality of original search queries such that each of the revised search queries includes a substitute query term for an original query term that is included in each of the original search queries; for each of the revised search queries associated with the substitution rule, accessing query log data that identifies (i) a set of search results that was presented to a user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) a particular, user-selected search result that was selected by the user from among the set of search results; determining a quality metric for the substitution rule based at least on the query log data that identifies, for each of the revised search queries associated with the substitution rule, (i) the set of search results that was presented to the user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) the particular, user-selected search result that was selected by the user from among the set of search results; determining that the quality metric does not satisfy a threshold; and in response to determining that the quality metric does not satisfy the threshold, removing the substitution rule. - View Dependent Claims (12, 13, 14, 15, 16, 17)
-
-
18. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising; identifying a plurality of revised search queries associated with a substitution rule, wherein the substitution rule is used to generate the revised search queries from a plurality of original search queries such that each of the revised search queries includes a substitute query term for an original query term that is included in each of the original search queries; for each of the revised search queries associated with the substitution rule, accessing query log data that identifies (i) a set of search results that was presented to a user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) a particular, user-selected search result that was selected by the user from among the set of search results; determining a quality metric for the substitution rule based at least on the query log data that identifies, for each of the revised search queries associated with the substitution rule, (i) the set of search results that was presented to the user in response to the respective revised search query and that includes one or more search results that were not selected by the user, and (ii) the particular, user-selected search result that was selected by the user from among the set of search results; determining that the quality metric does not satisfy a threshold; and in response to determining that the quality metric does not satisfy the threshold, removing the substitution rule. - View Dependent Claims (19, 20, 21, 22, 23, 24)
Specification