Selecting and ordering groups of titles

• US 10,698,909 B2
• Filed: 10/20/2016
• Issued: 06/30/2020
• Est. Priority Date: 06/06/2014
• Status: Active Grant

• Current Assignee: Netflix, Inc.
• Original Assignee: Netflix, Inc.
• Inventors: Alvino, Christopher, Gomez-Uribe, Carlos, Hammond, Che, Stipins, III, Janis
• Primary Examiner(s): Dang, Thanh-Ha
• Application Number:
  US15/299,301
• Publication Number:
  US 20170039199A1
• Time in Patent Office: 1,349 Days
• Field of Search: 707709
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0631 Item recommendations

Abstract

Techniques for selecting and ordering groups of titles to present as recommendations. In one embodiment, for example, a method performed by one or more computing devices of an online services comprises selecting, for each of a plurality of row positions, a group of titles to fill the row position based at least in part on a relevance score computed for the group of titles. The relevance score is based at least in part on a personalized ranking for a particular user of titles in the groups in titles selected to fill the row positions. The groups of titles selected to fill to the row positions are presented as recommendations to the particular user as a sequence of rows in which each row in the sequence corresponds to one of the selected groups of titles.

Claims

1. A method comprising:

determining, from a plurality of titles, a candidate group of titles;

computing, by one or more computing devices, a score for each title included in the candidate group;

pre-selecting, by the one or more computing devices and before interacting with a first application executing on a user device, a first display list from the candidate group by;

selecting, from the candidate group and based at least in part on the score for each title included in the candidate group, a selected set of titles,

ordering the selected set of titles to generate the first display list in order to fill a first plurality of row positions included in a first plurality of rows, and

storing the first display list;

in response to a first network interaction with the first application executing on the user device, generating a first network response to the first network interaction, wherein the first network response includes the first display list; and

sending the first network response to the user device, wherein;

the first network response causes the first application executing on the user device to display the first plurality of row positions included in the first plurality of rows, and

the first plurality of row positions are filled with at least a portion of the selected set of titles based on the first display list.

2. The method of claim 1, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a relevance score that is based at least in part on a personalized ranking of the first title by a first user of the user device.

3. The method of claim 1, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a diversity score that is based at least in part on a similarity between the first title and a first subset of titles included in the selected set of titles that have already been pre-selected.

4. The method of claim 1, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a coverage score that is based at least in part on a similarity between the first title and a positive user interaction by a first user of the user device.

5. The method of claim 1, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a weighted combination of:

a relevance score computed for the first title,

a diversity score computed for the first title, and

a coverage score computed for the first title.

6. The method of claim 1, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a row score, wherein the row score comprises a weighted combination of:

a relevance score computed for the first title,

a diversity score computed for the first title, and

a coverage score computed for the first title.

7. The method of claim 1, wherein pre-selecting the first display list before interacting with the first application executing on the user device comprises:

generating and storing the first display list when while the first application is not executing or when the first application is not communicating with the one or more computing devices.

8. The method of claim 1, wherein the first display list comprises:

a set of row indices corresponding to the row positions included in the first plurality of rows; and

for each row position index included in the set of row indices, an identifier of at least one title included in the selected set of titles that is to fill the row position corresponding to the row position index.

9. One or more non-transitory, computer-readable media including instructions that, when executed by one or more processors, cause the one or more processors to perform steps of:

determining, from a plurality of titles, a candidate group of titles;

computing, by one or more computing devices, a score for each title included in the candidate group;

pre-selecting, by the one or more computing devices and before interacting with a first application executing on a user device, a first display list from the candidate group by;

selecting, from the candidate group and based at least in part on the score for each title included in the candidate group, a selected set of titles,

ordering the selected set of titles to generate the first display list in order to fill a first plurality of row positions included in a first plurality of rows, and

storing the first display list;

in response to a first network interaction with the first application executing on the user device, generating a first network response to the first network interaction, wherein the first network response includes the first display list; and

sending the first network response to the user device, wherein;

the first network response causes the first application executing on the user device to display the first plurality of row positions included in the first plurality of rows, and

the first plurality of row positions are filled with at least a portion of the selected set of titles based on the first display list.

10. The one or more non-transitory, computer-readable media of claim 9, wherein computing the score for each title included in the candidate group comprises computing a relevance score and a diversity score.

11. The one or more non-transitory, computer-readable media of claim 9, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group:

a relevance score that is based at least in part on a personalized ranking of the first title by a first user of the user device; and

a coverage score that is based at least in part on a similarity between the first title and a positive user interaction by the first user.

12. The one or more non-transitory, computer-readable media of claim 9, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a row score, wherein the row score comprises a weighted combination of:

a relevance score computed for the first title,

a diversity score computed for the first title, and

a coverage score computed for the first title.

13. The one or more non-transitory, computer-readable media claim 9, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a weighted combination of:

a relevance score computed for the first title,

a diversity score computed for the first title, and

a coverage score computed for the first title.

14. A system comprising:

one or more computing devices that;

determine, from a plurality of titles, a candidate group of titles;

compute a score for each title included in the candidate group;

pre-selecting, before interacting with a first application executing on a user device, a first display list from the candidate group by;

selecting, from the candidate group and based at least in part on the score for each title included in the candidate group, a selected set of titles,

ordering the selected set of titles to generate the first display list in order to fill a first plurality of row positions included in a first plurality of rows, and

storing the first display list;

in response to a first network interaction with the first application executing on the user device, generates a first network response to the first network interaction, wherein the first network response includes the first display list; and

sends the first network response to the user device, wherein;

the first network response causes the first application executing on the user device to display the first plurality of row positions included in the first plurality of rows, and

at least the first plurality of row positions are filled with at least a portion of the selected set of titles based on the first display list.

15. The system of claim 14, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group, a relevance score that is based at least in part on:

a personalized ranking of the first title by a first user of the user device; and

an un-personalized ranking of the first title.

16. The system of claim 14, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group:

a relevance score; and

a diversity score, wherein the diversity score is based at least in part on a similarity between the first title and a first subset of titles included in the selected set of titles that have already been pre-selected.

17. The system of claim 14, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group:

a relevance score; and

a coverage score, wherein the coverage score is based at least in part on a similarity between the first title and a positive user interaction by a first user of the user device.

18. The system of claim 14, wherein computing the score for each title included in the candidate group comprises computing, for at least a first title included in the candidate group:

a relevance score; and

a coverage score, wherein the coverage score is based at least in part on a similarity between the first title and a first subset of titles included in the candidate set that were recently consumed.