Modifying search result ranking based on implicit user feedback

• US 9,235,627 B1
• Filed: 12/30/2013
• Issued: 01/12/2016
• Est. Priority Date: 11/02/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kim, Hyung-Jin, Tong, Simon, Shazeer, Noam M., Diligenti, Michelangelo
• Primary Examiner(s): Lu, Kuen
• Application Number:
  US14/143,622
• Time in Patent Office: 743 Days
• Field of Search: 707/723
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

The present disclosure includes systems and techniques relating to ranking search results of a search query. In general, the subject matter described in this specification can be embodied in a computer-implemented method that includes determining a measure of relevance for a document result within a context of a search query for which the document result is returned, the determining being based on a first number in relation to a second number, the first number corresponding to longer views of the document result, and the second number corresponding to at least shorter views of the document result; and outputting the measure of relevance to a ranking engine for ranking of search results, including the document result, for a new search corresponding to the search query. The subject matter described in this specification can also be embodied in various corresponding computer program products, apparatus and systems.

Claims

1. A computer-implemented method comprising:

determining a first number representing a count of longer views of a document following selections of search results, where the search results reference the document in response to submissions of a search query to a search engine;

determining a second number representing a count of shorter views of the document following the selections of the search results, where the search results reference the document in response to submissions of the search query;

determining a measure of relevance for the document with respect to the search query based on a ratio between the first number and the second number; and

outputting the measure of relevance to a ranking engine for ranking of search results, including new search results referencing the document, for a new search responsive to a new submission of the search query.

2. The method of claim 1, wherein determining the measure of relevance comprises dividing the first number by a count of total views of the document.

3. The method of claim 1, further comprising:

tracking individual selections of the search results referencing the document within the context of the search query for which the search results are returned;

weighting document views resulting from the selections based on viewing length information to produce weighted views of the document result; and

combining the weighted views of the document result to determine the first number.

4. The method of claim 3, wherein determining the measure of relevance comprises dividing the first number by the second number.

5. The method of claim 3, wherein the weighting comprises applying a continuous function to the document views resulting from the selections.

6. The method of claim 3, wherein the weighting comprises applying a discontinuous function to the document views resulting from the selections.

7. The method of claim 6, wherein applying the discontinuous function comprises:

classifying the individual selections of the document result into viewing time categories; and

assigning weights to the individual selections based on results of the classifying.

8. The method of claim 3, wherein the weighting comprises weighting the document views based on the viewing length information in conjunction with a viewing length differentiator.

9. The method of claim 8, wherein the viewing length differentiator includes a factor governed by a determined category of the search query, and the weighting comprises weighting the document views based on the determined category of the search query.

10. The method of claim 8, wherein the viewing length differentiator includes a factor governed by a determined type of a user generating the individual selections, and the weighting comprises weighting the document views based on the determined type of the user.

11. A system comprising:

one or more computers; and

a computer program product, encoded on a non-transitory computer-readable medium, operable to cause the one or more computers to perform operations comprising;

determining a first number representing a count of longer views of a document following selections of search results, where the search results reference the document in response to submissions of a search query to a search engine;

determining a second number representing a count of shorter views of the document following the selections of the search results, where the search results reference the document in response to submissions of the search query;

determining a measure of relevance for the document with respect to the search query based on a ratio between the first number and the second number; and

outputting the measure of relevance to a ranking engine for ranking of search results, including new search results referencing the document, for a new search responsive to a new submission of the search query.

12. The system of claim 11, wherein determining the measure of relevance comprises dividing the first number by a count of total views of the document.

13. The system of claim 11, the operations further comprising:

tracking individual selections of the search results referencing the document within the context of the search query for which the search results are returned;

weighting document views resulting from the selections based on viewing length information to produce weighted views of the document result; and

combining the weighted views of the document result to determine the first number.

14. The system of claim 13, wherein determining the measure of relevance comprises dividing the first number by the second number.

15. The system of claim 13, wherein the weighting comprises applying a continuous function to the document views resulting from the selections.

16. The system of claim 13, wherein the weighting comprises applying a discontinuous function to the document views resulting from the selections.

17. The system of claim 16, wherein applying the discontinuous function comprises:

classifying the individual selections of the document result into viewing time categories; and

assigning weights to the individual selections based on results of the classifying.

18. The system of claim 13, wherein the weighting comprises weighting the document views based on the viewing length information in conjunction with a viewing length differentiator.

19. The system of claim 18, wherein the viewing length differentiator includes a factor governed by a determined category of the search query, and the weighting comprises weighting the document views based on the determined category of the search query.

20. The system of claim 18, wherein the viewing length differentiator includes a factor governed by a determined type of a user generating the individual selections, and the weighting comprises weighting the document views based on the determined type of the user.