Modifying search result ranking based on a temporal element of user feedback

• US 9,092,510 B1
• Filed: 04/30/2007
• Issued: 07/28/2015
• Est. Priority Date: 04/30/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Stets, Robert J. Jr., Paskin, Mark Andrew
• Primary Examiner(s): Mamillapalli, Pavan
• Application Number:
  US11/742,447
• Time in Patent Office: 3,011 Days
• Field of Search: 707/723, 707/727
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline

Abstract

In general, the subject matter described in this specification can be embodied in a method that includes: obtaining user feedback associated with quality of an electronic document; adjusting a measure of relevance for the electronic document based on a temporal element of the user feedback; and outputting the measure of relevance to a ranking engine for ranking of search results, including the electronic document, for a search for which the electronic document is returned. Obtaining the user feedback can include receiving user selections of documents presented by a document search service, the method can include evaluating the user selections in accordance with an implicit user feedback model to determine the measure of relevance, and adjusting the measure of relevance can include adjusting the measure of relevance in accordance with the implicit user feedback model.

Claims

1. A computer-implemented method comprising:

obtaining user feedback associated with quality of a document including receiving user selections of the document and a plurality of other documents presented by a document search service;

adjusting a measure of relevance for the document, including;

determining a first click trend for the document, the first click trend comprising a plurality of first date intervals and, for each of the first date intervals, a respective volume of user selections of the document in search results provided by the document search service within the date interval;

determining a second click trend for the other documents, the second click trend comprising a plurality of second date intervals and, for each of the second date intervals, a respective volume of user selections of the other documents in search results provided by the document search service within the date interval, where contributions of the user selections of the document and the user selections of the other documents to the first click trend and the second click trend, respectively, are based on temporal distance between a current time and times of the user selections;

comparing the first click trend to the second click trend and determining that a first rate of change for the first click trend is larger than a second rate of change for the second click trend; and

as a consequence of determining that the first rate of change is larger than the second rate of change, increasing the measure of relevance for the document; and

outputting the measure of relevance to a ranking engine for ranking of search results, for a search that identifies the document as a search result, the ranking of search results including ranking the document using the measure of relevance.

2. The method of claim 1 further comprising determining the measure of relevance for the document within a context of a search query for which the document is returned by the document search service.

3. The method of claim 1, wherein adjusting the measure of relevance comprises weighting the user selections based on recency of document viewing grouped into time span categories.

4. The method of claim 3, wherein the weighting of the user selections is initiated only after a total number of selections within a context of a search query surpasses a threshold.

5. The method of claim 3, wherein the time span categories comprise less than two weeks old, two to four weeks old, four to six weeks old, six to eight weeks old, and more than eight weeks old, and the weighting comprises:

applying no weighting to selections that are less than two weeks old;

applying at least a fifteen percent reduction weighting to selections that are two to four weeks old;

applying at least a fifty percent reduction weighting to selections that are four to six weeks old;

applying at least a seventy five percent reduction weighting to selections that are six to eight weeks old; and

applying at least a ninety percent reduction weighting to selections that are more than eight weeks old.

6. A computer program product, encoded on a computer-readable storage medium, operable to cause data processing apparatus to perform operations comprising:

obtaining user feedback associated with quality of a document including receiving user selections of the document and a plurality of other documents presented by a document search service;

adjusting a measure of relevance for the document, including;

determining a first click trend for the document, the first click trend comprising a plurality of first date intervals and, for each of the first date intervals, a respective volume of user selections of the document in search results provided by the document search service within the date interval;

determining a second click trend for the other documents, the second click trend comprising a plurality of second date intervals and, for each of the second date intervals, a respective volume of user selections of the other documents in search results provided by the document search service within the date interval, where contributions of the user selections of the document and the user selections of the other documents to the first click trend and the second click trend, respectively, are based on temporal distance between a current time and times of the user selections;

comparing the first click trend to the second click trend and determining that a first rate of change for the first click trend is larger than a second rate of change for the second click trend; and

as a consequence of determining that the first rate of change is larger than the second rate of change, increasing the measure of relevance for the document; and

outputting the measure of relevance to a ranking engine for ranking of search results, for a search that identifies the document as a search result, the ranking of search results including ranking the document using the measure of relevance.

7. The computer program product of claim 6 further comprising determining the measure of relevance for the document within a context of a search query for which the document is returned by the document search service.

8. The computer program product of claim 6, wherein adjusting the measure of relevance comprises weighting the user selections based on recency of document viewing grouped into time span categories.

9. The computer program product of claim 8, wherein the weighting of the user selections is initiated only after a total number of selections within a context of a search query surpasses a threshold.

10. The computer program product of claim 8, wherein the time span categories comprise less than two weeks old, two to four weeks old, four to six weeks old, six to eight weeks old, and more than eight weeks old, and the weighting comprises:

applying no weighting to selections that are less than two weeks old;

applying at least a fifteen percent reduction weighting to selections that are two to four weeks old;

applying at least a fifty percent reduction weighting to selections that are four to six weeks old;

applying at least a seventy five percent reduction weighting to selections that are six to eight weeks old; and

applying at least a ninety percent reduction weighting to selections that are more than eight weeks old.

11. A system of one or more computers configured to perform operations comprising:

obtaining user feedback associated with quality of a document including receiving user selections of the document and a plurality of other documents presented by a document search service;

adjusting a measure of relevance for the document, including;

determining a first click trend for the document, the first click trend comprising a plurality of first date intervals and, for each of the first date intervals, a respective volume of user selections of the document in search results provided by the document search service within the date interval;

determining a second click trend for the other documents, the second click trend comprising a plurality of second date intervals and, for each of the second date intervals, a respective volume of user selections of the other documents in search results provided by the document search service within the date interval, where contributions of the user selections of the document and the user selections of the other documents to the first click trend and the second click trend, respectively, are based on temporal distance between a current time and times of the user selections;

comparing the first click trend to the second click trend and determining that a first rate of change for the first click trend is larger than a second rate of change for the second click trend; and

as a consequence of determining that the first rate of change is larger than the second rate of change, increasing the measure of relevance for the document; and

outputting the measure of relevance to a ranking engine for ranking of search results, for a search that identifies the document as a search result, the ranking of search results including ranking the document using the measure of relevance.

12. The system of claim 11, the operations further comprising determining the measure of relevance for the document within a context of a search query for which the document is returned by the document search service.

13. The system of claim 11, wherein adjusting the measure of relevance comprises weighting the user selections based on recency of document viewing grouped into time span categories.

14. The system of claim 13, wherein the weighting of the user selections is initiated only after a total number of selections within a context of a search query surpasses a threshold.

15. The system of claim 13, wherein the time span categories comprise less than two weeks old, two to four weeks old, four to six weeks old, six to eight weeks old, and more than eight weeks old, and the weighting comprises:

applying no weighting to selections that are less than two weeks old;

applying at least a fifteen percent reduction weighting to selections that are two to four weeks old;

applying at least a fifty percent reduction weighting to selections that are four to six weeks old;

applying at least a seventy five percent reduction weighting to selections that are six to eight weeks old; and

applying at least a ninety percent reduction weighting to selections that are more than eight weeks old.