Combining user feedback

• US 8,832,083 B1
• Filed: 07/23/2010
• Issued: 09/09/2014
• Est. Priority Date: 07/23/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chen, Zhihui, Kim, Hyung-Jin, Kacholia, Varun, Stoutamire, David P., Adams, Henele I., Khandelwal, Nitin
• Primary Examiner(s): Le, Uyen
• Application Number:
  US12/842,345
• Time in Patent Office: 1,509 Days
• Field of Search: None
• US Class Current: 707/723
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Obtaining search results responsive to a query, each search result identifying a respective resource and each resource having a respective rank determined by a primary ranking algorithm. The method includes obtaining primary user feedback data and secondary user feedback data for the resource. The primary and secondary user feedback data representing previous user interactions with the resource when the resource was presented as a search result responsive to the query. The method also includes applying a weight to the secondary user feedback data, the weight being at least partially based on a threshold quantity of the primary user feedback data. The method also includes aggregating the primary user feedback data and the weighted secondary user feedback data and modifying the respective rank of the resource as a search result for the query based at least partially on the aggregated data.

Claims

1. A computer-implemented method comprising:

obtaining search results responsive to a query, each search result identifying a respective resource and each resource having a respective rank determined by a primary ranking algorithm;

for one or more of the respective resources;

obtaining primary user feedback data for the resource, the primary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by the primary ranking algorithm;

obtaining secondary user feedback data for the resource, the secondary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by a different secondary ranking algorithm;

applying a weight to the secondary user feedback data, the weight being at least partially based on a threshold quantity of the primary user feedback data;

aggregating the primary user feedback data and the weighted secondary user feedback data and modifying the respective rank of the resource as a search result for the query based at least partially on the aggregated data; and

wherein obtaining, applying and aggregating are performed by one or more data processing apparatus.

2. The method of claim 1, wherein the primary user feedback data includes a first plurality of user interactions where the duration of each of the user interactions in the first plurality satisfies a first threshold;

and

in which the secondary user feedback data includes a second plurality of user interactions where the duration of each of the user interactions in the second plurality satisfies a second threshold.

3. The method of claim 2, wherein the weight is further based on a count of user interactions in the primary user feedback data and a count of user interactions in the secondary user feedback data.

4. The method of claim 3, wherein the weight is inversely proportional to the count of user interactions in the secondary user feedback data.

5. The method of claim 3, wherein the weight is inversely proportional to the count of user interactions in the primary user feedback data.

6. The method of claim 1, wherein the search results refer to video resources.

7. The method of claim 1, wherein the search results refer to web resources.

8. The method of claim 1, wherein applying the weight to the secondary user feedback data comprises multiplying a count of targeted user interactions in the secondary user data by the weight to create the weighted secondary user feedback data.

9. The method of claim 8, wherein aggregating the primary user feedback data and the weighted secondary user feedback data comprises combining a count of targeted interactions in the primary user feedback data to the weighted target interactions count.

10. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

obtaining search results responsive to a query, each search result identifying a respective resource and each resource having a respective rank determined by a primary ranking algorithm;

for one or more of the respective resources;

obtaining primary user feedback data for the resource, the primary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by the primary ranking algorithm;

obtaining secondary user feedback data for the resource, the secondary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by a different secondary ranking algorithm;

applying a weight to the secondary user feedback data, the weight being at least partially based on a threshold quantity of the primary user feedback data; and

aggregating the primary user feedback data and the weighted secondary user feedback data and modifying the respective rank of the resource as a search result for the query based at least partially on the aggregated data.

11. The medium of claim 10, wherein the primary user feedback data includes a first plurality of user interactions where the duration of each of the user interactions in the first plurality satisfies a first threshold;

and

in which the secondary user feedback data includes a second plurality of user interactions where the duration of each of the user interactions in the second plurality satisfies a second threshold.

12. The medium of claim 11, wherein the weight is further based on a count of user interactions in the primary user feedback data and a count of user interactions in the secondary user feedback data.

13. The medium of claim 12, wherein the weight is inversely proportional to the count of user interactions in the secondary user feedback data.

14. The medium of claim 12, wherein the weight is inversely proportional to the count of user interactions in the primary user feedback data.

15. The medium of claim 10, wherein the search results refer to video resources.

16. The medium of claim 10, wherein the search results refer to web resources.

17. The medium of claim 10, wherein applying the weight to the secondary user feedback data comprises multiplying a count of targeted user interactions in the secondary user data by the weight to create the weighted secondary user feedback data.

18. The medium of claim 17, wherein aggregating the primary user feedback data and the weighted secondary user feedback data comprises combining a count of targeted interactions in the primary user feedback data to the weighted target interactions count.

19. A system comprising:

one or more computers; and

a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

obtaining search results responsive to a query, each search result identifying a respective resource and each resource having a respective rank determined by a primary ranking algorithm;

for one or more of the respective resources;

obtaining primary user feedback data for the resource, the primary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by the primary ranking algorithm;

obtaining secondary user feedback data for the resource, the secondary user feedback data representing a plurality of previous user interactions with the resource when the resource was presented as a search result responsive to the query and wherein the search result had a rank determined by a different secondary ranking algorithm;

applying a weight to the secondary user feedback data, the weight being at least partially based on a threshold quantity of the primary user feedback data; and

aggregating the primary user feedback data and the weighted secondary user feedback data and modifying the respective rank of the resource as a search result for the query based at least partially on the aggregated data.

20. The system of claim 19, wherein the primary user feedback data includes a first plurality of user interactions where the duration of each of the user interactions in the first plurality satisfies a first threshold;

and

in which the secondary user feedback data includes a second plurality of user interactions where the duration of each of the user interactions in the second plurality satisfies a second threshold.

21. The system of claim 20, wherein the weight is further based on a count of user interactions in the primary user feedback data and a count of user interactions in the secondary user feedback data.

22. The system of claim 20, wherein the weight is inversely proportional to the count of user interactions in the secondary user feedback data.

23. The system of claim 22, wherein the weight is inversely proportional to the count of user interactions in the primary user feedback data.

24. The system of claim 19, wherein the search results refer to video resources.

25. The system of claim 19, wherein the search results refer to web resources.

26. The system of claim 19, wherein applying the weight to the secondary user feedback data comprises multiplying a count of targeted user interactions in the secondary user data by the weight to create the weighted secondary user feedback data.

27. The system of claim 26, wherein aggregating the primary user feedback data and the weighted secondary user feedback data comprises combining a count of targeted interactions in the primary user feedback data to the weighted target interactions count.