Watch time based ranking

• US 10,133,809 B1
• Filed: 06/29/2015
• Issued: 11/20/2018
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Lawry, James, Kressler, Bryan M., Angelov, Stanislav Plamenov, Elson, David Kliger, Kaiserlian, Christian, Agraz, David, Hylton, Jeremy, Pham, Phong Thanh
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US14/754,253
• Time in Patent Office: 1,240 Days
• Field of Search: 707725, 707603, 707706, 707709, 707722, 707723, 707748
• CPC Class Codes: G06F 16/3331 Query processing
• CPC Class Codes: G06F 16/3346 using probabilistic model
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/4826 using recommendation lists, e.g. of programs or channels sorted out according to their score
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for ranking search results. One of the methods includes identifying one or more sessions for a query and associating watch times of the respective resources watched in the sessions with the query. One or more watch time signals are calculated for a first resource and the query based on the watch times associated with the query. A first search result responsive to the query is obtained, wherein the first search result identifies the first resource and has an associated score S. A new score S′ is calculated based on a least S and a watch time function, the watch time function being a function of the one or more watch time signals. The new score S′ is provided to a process for ranking search results including the first search result.

Claims

1. A method comprising:

identifying one or more sessions for a first user query, wherein

each session comprises a respective chain of respective resources attended to by a respective user that

begins with a respective first resource that was identified by a respective first search result in response to the first user query from the respective user and continues with one or more respective subsequent resources that were each visited in the session by the user by invoking a link associated with a preceding resource in the chain;

associating, with the first user query, a total user attention time of the respective resources attended to in the one or more sessions for the first user query, the total user attention time being a total of individual user attention times of the resources in the one or more sessions;

responsive to receiving a second user query matching the first user query after the one or more sessions have ended;

obtaining one or more second search results responsive to the second user query and one or more respective second search result scores for the second search results, each second search result identifying a corresponding resource;

adjusting the respective second search result scores based on the total user attention time associated with the first user query;

ranking the second search results based on the adjusted search result scores; and

in response to the second search query, providing the ranked second search results.

2. The method of claim 1, wherein the first user query is the same as the second user query.

3. The method of claim 1, wherein the resources in a session comprise video content items, and wherein an individual user attention time of a resource is a length of time that a user spent watching a video content item.

4. The method of claim 3, further comprising:

computing the watch times for a particular session using pings received during the session, wherein each ping identifies a point in a video content item that a user has reached.

5. The method of claim 1, wherein the resources in a session comprise web pages, and wherein an individual user attention time of a resource is a length of time that a user spent reviewing a web page.

6. The method of claim 1, wherein adjusting the respective second search result scores based on the total user attention time comprises:

for each second search result, adjusting the second search result score by a multiplier that is a function of a signal that is derived from (i) the individual user attention times of the resources corresponding to the second search result (ii) the total user attention time associated with the first query.

7. The method of claim 6, wherein the signal is a document-query fraction (dqf) score.

8. The method of claim 6, wherein the signal is a relative document performance (rdf) score.

9. The method of claim 6, wherein the multiplier is based on a utility function of the signal, wherein the utility function is a convexity function, a low-data backoff function, or a scaling function, or a combination of two or more of them.

10. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

identifying one or more sessions for a first user query, wherein

each session comprises a respective chain of respective resources attended to by a respective user that

begins with a respective first resource that was identified by a respective first search result in response to the first user query from the respective user and continues with one or more respective subsequent resources that were each visited in the session by the user by invoking a link associated with a preceding resource in the chain;

associating, with the first user query, a total user attention time of the respective resources attended to in the one or more sessions for the first user query, the total user attention time being a total of individual user attention times of the resources in the one or more sessions;

responsive to receiving a second user query matching the first user query after the one or more sessions have ended;

obtaining one or more second search results responsive to the second user query and one or more respective second search result scores for the second search results, each second search result identifying a corresponding resource;

adjusting the respective second search result scores based on the total user attention time associated with the first user query;

ranking the second search results based on the adjusted search result scores; and

in response to the second search query, providing the ranked second search results.

11. The system of claim 10, wherein the first user query is the same as the second user query.

12. The system of claim 10, wherein the resources in a session comprise video content items, and wherein a watch time of a resource is a length of time that a user spent watching a video content item.

13. The system of claim 12, the operations further comprising:

computing the watch times for a particular session using pings received during the session, wherein each ping identifies a point in a video content item that a user has reached.

14. The system of claim 10, wherein the resources in a session comprise web pages, and wherein a user attention time of a resource is a length of time that a user spent reviewing a web page.

15. The system of claim 10, wherein adjusting the respective second search result scores based on the total user attention time comprises:

for each second search result, adjusting the second search result score by a multiplier that is a function of a signal that is derived from (i) the individual user attention times of the resources corresponding to the second search result (ii) the total user attention time associated with the first query.

16. The system of claim 15, wherein the signal is a document-query fraction (dqf) score.

17. The system of claim 15, wherein the signal is a relative document performance (rdf) score.

18. The system of claim 15, wherein the multiplier is based on a utility function of the signal, wherein the utility function is a convexity function, a low-data backoff function, or a scaling function, or a combination of two or more of them.

19. A non-transitory computer storage medium encoded with a computer program, the computer program comprising instructions that when executed by a computing device cause the computing device to perform operations comprising:

identifying one or more sessions for a first user query, wherein

each session comprises a respective chain of respective resources attended to by a respective user that

begins with a respective first resource that was identified by a respective first search result in response to the first user query from the respective user and continues with one or more respective subsequent resource that were each visited in the session by the user by invoking a link associated with a preceding resource in the chain;

associating, with the first search query, a total user attention time of the respective resources attended to in the one or more sessions for the first user query, the total user attention time being a total of individual user attention times of the resources in the one or more sessions;

responsive to receiving a second user query matching the first user query after the one or more sessions have ended;

obtaining one or more second search results responsive to the second user query and one or more respective second search result scores for the second search results, each second search result identifying a corresponding resource;

adjusting the respective second search result scores based on the total user attention time associated with the first user query;

ranking the second search results based on the adjusted search result scores; and

in response to the second search query, providing the ranked second search results.

20. The non-transitory computer storage medium of claim 19, wherein adjusting the respective second search result scores based on the total user attention time comprises:

for each second search result, adjusting the second search result score by a multiplier that is a function of a signal that is derived from (i) the individual user attention times of the resources corresponding to the second search result (ii) the total user attention time associated with the first query.

21. The non-transitory computer storage medium of claim 20, wherein the signal is a document-query fraction (dqf) score.

22. The non-transitory computer storage medium of claim 20, wherein the signal is a relative document performance (rdf) score.

23. The non-transitory computer storage medium of claim 20, wherein the multiplier is based on a utility function of the signal, wherein the utility function is a convexity function, a low-data backoff function, or a scaling function, or a combination of two or more of them.