Media content rankings for discovery of novel content

• US 9,430,532 B2
• Filed: 07/30/2013
• Issued: 08/30/2016
• Est. Priority Date: 07/30/2013
• Status: Active Grant

• Current Assignee: Netflix, Inc.
• Original Assignee: Netflix, Inc.
• Inventors: Basilico, Justin D.
• Primary Examiner(s): Waldron, Scott A
• Assistant Examiner(s): Davanlou, Soheila
• Application Number:
  US13/954,780
• Publication Number:
  US 20150039608A1
• Time in Patent Office: 1,127 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/735 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06N 20/00 Machine learning

Abstract

A content provider system ranks media content items with respect to a particular user based on selection scores determined for each of the media content items. The selection scores may be determined using a particular model that calculates a predicted selection score based on feature values associated with the content item with respect to the particular user. The feature values may indicate properties of the media content item, the particular user, or the particular user'"'"'s relationship with the content item, including information about the novelty of the media content item with respect to the user. The particular model may be trained with sample user consumption data points that represent various combinations of media content items and users. The data point information evaluated during the training of the particular model may cause the model to assign higher selection scores to content items that are novel in particular ways.

Claims

1. A method comprising:

in a data processing system configured to generate personalized rankings of a first plurality of media content items in a catalog;

determining a plurality of user consumption values, wherein each user consumption value of the plurality of user consumption values is associated with a user consumption data point of a plurality of user consumption data points, and wherein each user consumption data point is associated with a user and a media content item in a second plurality of media content items;

determining, for each user consumption data point of the plurality of user consumption data points, a plurality of feature values associated with the user consumption data point, wherein the plurality of feature values include a particular feature value indicating whether the user associated with the user consumption data point consumed the media content item associated with the user consumption data point within a particular amount of time;

training a selection value prediction model based on each feature value of the plurality of feature values associated with each user consumption data point of the plurality of user consumption data points;

for each media content item of the first plurality of media content items, determining, based on the selection value prediction model, a predicted selection score for an input user and the media content item;

ranking the first plurality of media content items in the catalog for the input user based on the predicted selection score for each media content item in the first plurality of media content items;

wherein the method is performed using one or more processors.

2. The method of claim 1, wherein a certain user consumption value associated with a certain user consumption data point is determined based upon:

certain novelty information indicating a novelty of a certain content item corresponding to the certain user consumption data point with respect to a certain user corresponding to the certain user consumption data point;

user consumption information indicating whether the certain content item was selected for consumption by the certain user.

3. The method of claim 2, wherein the novelty of the certain content item with respect to the certain user is determined based on novelty of a related content item with respect to the certain user, wherein the related content item is different than the certain content item.

4. The method of claim 2, wherein the certain novelty information indicates novelty of a similar content item with respect to the certain user, wherein the similar content item is different than the certain content item and is determined to be similar to the certain content item.

5. The method of claim 2, wherein the certain novelty information indicates at least one of:

whether the certain content item was consumed for at least a particular amount of time;

whether the certain content item was reviewed;

whether an option to consume the certain content item was displayed to the input user;

whether the certain content item was rated;

or

whether the certain content item was added to queue of content items for consumption.

6. The method of claim 1, wherein, for each user consumption data point, the particular feature value included in the plurality of feature values is a non-binary novelty value.

7. The method of claim 1, wherein, for each user consumption data point, the particular feature value included in the plurality of feature values is a binary value.

8. The method of claim 1, wherein, for each user consumption data point, the plurality of feature values include an additional particular feature value indicating novelty information about the media content item with respect to the user associated with the user consumption data point, wherein the additional particular feature value is different than the particular feature value.

9. The method of claim 1, wherein, for each user consumption data point, the plurality of feature values further include an additional particular feature value indicating popularity of the media content item with respect to one or more users other than the user associated with the user associated with the user consumption data point.

10. The method of claim 1, wherein the selection value prediction model is a regression function.

11. The method of claim 1, wherein the selection value prediction model is a classification function comprising a plurality of weights that respectively correspond with the plurality of feature values associated with the user consumption data point of the plurality of user consumptions data points.

12. The method of claim 1, wherein the selection value prediction model comprises a plurality of weight values.

13. The method of claim 12, wherein, for each media content item of the first plurality of media content items, determining the predicted selection score comprises summing a plurality of products, wherein each product in the plurality of products is based on a weight value of the plurality of weight values and a corresponding feature value in the plurality of feature values.

14. The method of claim 1, further comprising causing display of media content information representing the media content items, wherein the media content information is ordered according to a result of the ranking.

15. The method of claim 1, wherein each user consumption value of the plurality of user consumption values represents observed user consumption of the media content item associated with the user consumption data point that is associated with the user consumption value.

16. The method of claim 1, further comprising:

determining an unfavorable user consumption value for a certain user consumption data point associated with a certain user and a certain content item in response to determining;

that the certain content item was selected for consumption by the certain user; and

that the certain content item was not novel with respect to the certain user.

17. A data processing system for generating personalized ranking of a first plurality of media content items in a catalog, the system comprising:

a memory;

one or more processors coupled to the memory and configured to;

determine a plurality of user consumption values, wherein each user consumption value of the plurality of user consumption values is associated with a user consumption data point of a plurality of user consumption data points, and wherein each user consumption data point is associated with a user and a media content item in a second plurality of media content items;

determine, for each user consumption data point of the plurality of user consumption data points, a plurality of feature values associated with the user consumption data point, wherein the plurality of feature values include a particular feature value indicating whether the user associated with the user consumption data point consumed the media content item associated with the user consumption data point within a particular amount of time;

training a selection value prediction model based on each feature value of the plurality of feature values associated with each user consumption data point of the plurality of user consumption data points;

for each media content item of the first plurality of media content items, determine, based on the selection value prediction model, a predicted selection score for an input user and an media content item;

rank the first plurality of media content items in the catalog for the input user based on the predicted selection score for each media content item in the first plurality of media content items.

18. The data processing system of claim 17, wherein a certain user consumption value associated with a certain user consumption data point is determined based upon:

certain novelty information indicating a novelty of a certain content item corresponding to the certain user consumption data point with respect to a certain user corresponding to the certain user consumption data point;

user consumption information indicating whether the certain content item was selected for consumption by the certain user.

19. The data processing system of claim 18, wherein the novelty of the certain content item with respect to the certain user is determined based on novelty of a related content item with respect to the certain user, wherein the related content item is different than the certain content item.

20. The data processing system of claim 18, wherein the certain novelty information indicates novelty of a similar content item with respect to the certain user, wherein the similar content item is different than the certain content item and is determined to be similar to the certain content item.

21. The data processing system of claim 18, wherein the certain novelty information indicates at least one of:

whether the certain content item was consumed for at least a particular amount of time;

whether the certain content item was reviewed;

whether an option to consume the certain content item was displayed to the input user;

whether the certain content item was rated;

or

whether the certain content item was added to queue of content items for consumption.

22. The data processing system of claim 17, wherein, for each user consumption data point, the particular feature value included in the plurality of feature values is a non-binary novelty value.

23. The data processing system of claim 17, wherein, for each user consumption data point, the particular feature value included in the plurality of feature values is a binary value.

24. The data processing system of claim 17, wherein, for each user consumption data point, the plurality of feature values include an additional particular feature value indicating novelty information about the media content item with respect to the user associated with the user consumption data point, wherein the additional particular feature value is different than the particular feature value.

25. The data processing system of claim 17, wherein, for each user consumption data point, the plurality of feature values further include an additional particular feature value indicating popularity of the media content item with respect to one or more users other than the user associated with the user associated with the user consumption data point.

26. The data processing system of claim 17, wherein the selection value prediction model is a regression function.

27. The data processing system of claim 17, wherein the selection value prediction model is a classification function comprising a plurality of weights that respectively correspond with the feature values associated with the user consumption data point of the plurality of user consumptions data points.

28. The data processing system of claim 17, wherein the selection value prediction model comprises a plurality of weight values.

29. The data processing system of claim 28, wherein, for each media content item of the first plurality of media content items, determining the predicted selection score comprises summing a plurality of products, wherein each product in the plurality of products is based on a weight value of the plurality of weight values and a corresponding feature value in the plurality of feature values.

30. The data processing system of claim 17, further configured to cause display of media content information representing the media content items, wherein the media content information is ordered according to a result of the ranking.

31. The data processing system of claim 17, wherein each user consumption value of the plurality of user consumption values represents observed user consumption of the media content item associated with the user consumption data point that is associated with the user consumption value.

32. The data processing system of claim 17, further configured to:

determine an unfavorable user consumption value for a certain user consumption data point associated with a certain user and a certain content item in response to determining;

that the certain content item was selected for consumption by the certain user; and

that the certain content item was not novel with respect to the certain user.

33. A method comprising:

in a data processing system configured to generate personalized rankings of a first plurality of media content items in a catalog;

determining a plurality of user consumption values, wherein each user consumption value of the plurality of user consumption values is associated with a user consumption data point of a plurality of user consumption data points, and wherein each user consumption data point is associated with a user and a media content item in a second plurality of media content items, wherein the first plurality of media content items is different than the second plurality of media content items;

determining, for each user consumption data point of the plurality of user consumption data points, a plurality of feature values associated with the user consumption data point, wherein the plurality of feature values include a particular feature value indicating whether the user associated with the user consumption data point consumed the media content item associated with the user consumption data point within a particular amount of time;

training a selection value prediction model based on each feature value of the plurality of feature values associated with each user consumption data point of the plurality of user consumption data points, wherein the selection value prediction model comprises a plurality of weight values;

for each media content item of the first plurality of media content items, determining a predicted selection score for an input user and the media content item by summing a plurality of products, wherein each product in the plurality of products is based on a weight value of the plurality of weight values and a corresponding feature value in the plurality of feature values;

ranking the first plurality of media content items in the catalog for the input user based on the predicted selection score for each media content item in the first plurality of media content items;

wherein the method is performed using one or more processors.