Click or skip evaluation of query term optionalization rule

• US 9,141,672 B1
• Filed: 12/27/2012
• Issued: 09/22/2015
• Est. Priority Date: 01/25/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Popovici, Dan, Hoffman, Jeremy D.
• Primary Examiner(s): HICKS, MICHAEL J
• Application Number:
  US13/727,968
• Time in Patent Office: 999 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9032 Query formulation
• CPC Class Codes: G06F 16/90324 using system suggestions
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for click or skip evaluation of optionalization rules. In one aspect, a method includes accessing query log data that identifies, for an initial search query, one or more revised search queries that were identified by designating a query term of the initial search query as optional according to one or more optionalization rules, and a particular search result selected by the user. A determination is made, using the query log data, that the particular search result corresponds to the one or more revised search queries that were identified by designating the query term of the initial search query as optional according to one or more optionalization rules. In response to this determination, a click count is incremented for the one or more optionalization rules that corresponds to the particular search result.

Claims

1. A computer-implemented method comprising:

accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;

accessing query log data that identifies, for an initial search query that includes the particular query term;

one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,

search results that were identified using the initial search query,

search results that were identified using the one or more revised search queries, and

a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;

in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and

after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.

2. The method of claim 1, comprising:

ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and

in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.

3. The method of claim 2, comprising associating a first weight with the click count and a second weight with the skip count.

4. The method of claim 3, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:

5. The method of claim 2, comprising assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.

6. The method of claim 5, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.

7. The method of claim 5, comprising:

determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and

removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.

8. The method of claim 1, wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,

wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,

wherein the initial search query includes the particular other query term, and

wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.

9. The method of claim 1, wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises:

determining, using the query log data, that the particular search result does not include the particular query term; and

adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term in response to (i) determining that the particular search result is one of the search results that were identified using the one or more revised search queries, and (ii) determining that the particular search result does not include the particular query term included in the initial search query.

10. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;

accessing query log data that identifies, for an initial search query that includes the particular query term;

one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,

search results that were identified using the initial search query,

search results that were identified using the one or more revised search queries, and

a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;

in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and

after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query team, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.

11. The medium of claim 10, wherein the operations comprise:

ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and

in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.

12. The medium of claim 11, wherein the operations comprise assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.

13. The medium of claim 12, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.

14. The medium of claim 12, wherein the operations comprise:

determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and

removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.

15. The medium of claim 11, wherein the operations comprise associating a first weight with the click count and a second weight with the skip count.

16. The medium of claim 15, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:

17. The medium of claim 10, wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,

wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,

wherein the initial search query includes the particular other query term, and

wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.

18. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;

accessing query log data that identifies, for an initial search query that includes the particular query term;

one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries,

search results that were identified using the initial search query,

search results that were identified using the one or more revised search queries, and

a particular search result selected by a user from among a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries;

in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term; and

after adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the click count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.

19. The system of claim 18, wherein the operations comprise:

ranking the search results included in the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries; and

in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term.

20. The system of claim 19, wherein the operations comprise assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.

21. The system of claim 20, wherein the operations comprise:

determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and

removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.

22. The system of claim 19, wherein the operations comprise associating a first weight with the click count and a second weight with the skip count.

23. The system of claim 22, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:

24. The system of claim 18, wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,

wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,

wherein the initial search query includes the particular other query term, and

wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.

25. A computer-implemented method comprising:

accessing data that identifies one or more query term optionalization rules that are specific to a particular query term, the one or more query term optionalization rules indicating whether the particular query term should be made optional in revisions of search queries that include the particular query term;

accessing query log data that identifies, for an initial search query that includes the particular query term;

one or more revised search queries that a search engine system generated in response to the initial search query using the one or more query term optionalization rules that are specific to the particular query term, wherein the particular query term is optional in the one or more revised search queries

search results that were identified using the initial search query,

search results that were identified using the one or more revised search queries,

a ranking of a set of search results that includes (i) one or more of the search results that were identified using the initial search query and (ii) one or more of the search results that were identified using the one or more revised search queries, and

a particular search result selected by a user from among the set of search results that includes (i) the one or more of the search results that were identified using the initial search query and (ii) the one or more of the search results that were identified using the one or more revised search queries;

determining, using the query log data, that a search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries;

in response to determining that the search result that is ranked above the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a skip count for the one or more query term optionalization rules that are specific to the particular query term; and

after adjusting the skip count for the one or more query term optionalization rules that are specific to the particular query term, determining, based at least on the skip count for the one or more query term optionalization rules that are specific to the particular query term, whether to revise a subsequently-received search query that includes the particular query term using the one or more query term optionalization rules that are specific to the particular query term.

26. The method of claim 25, comprising:

determining, using the query log data, that the particular search result is one of the search results that were identified using the one or more revised search queries; and

in response to determining that the particular search result is one of the search results that were identified using the one or more revised search queries, adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term.

27. The method of claim 26, comprising assigning a score to the one or more query term optionalization rules that are specific to the particular query term based at least on the click count and the skip count.

28. The method of claim 27, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term is based on a ratio of (i) the click count to (ii) the click count and the skip count.

29. The method of claim 27, comprising:

determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy a threshold; and

removing the one or more query term optionalization rules that are specific to the particular query term from a set of query term optionalization rules that includes other query term optionalization rules that are specific to other query terms, the other query term optionalization rules indicating whether the other query terms should be made optional in revisions of search queries that include the other query terms, based on determining that the score assigned to the one or more query term optionalization rules that are specific to the particular query term does not satisfy the threshold.

30. The method of claim 27, wherein the score assigned to the one or more query term optionalization rules that are specific to the particular query term satisfies:

31. The method of claim 26, comprising associating a first weight with the click count and a second weight with the skip count.

32. The method of claim 25, wherein the one or more revised search queries were generated by the search engine system using one or more query term optionalization rules that are specific to the particular query term and are associated with a particular context,

wherein the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context indicate whether the particular query term should be made optional in revisions of search queries that include the particular query term and the particular other query term,

wherein the initial search query includes the particular other query term, and

wherein adjusting the click count for the one or more query term optionalization rules that are specific to the particular query term comprises adjusting a click count for the one or more query term optionalization rules that are specific to the particular query term and are associated with the particular context.