Prediction of media selection consumption using analysis of user behavior

• US 9,462,313 B1
• Filed: 08/31/2012
• Issued: 10/04/2016
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Sbaiz, Luciano, Berent, Jesse
• Primary Examiner(s): Saltarelli, Dominic D
• Application Number:
  US13/600,640
• Time in Patent Office: 1,495 Days
• Field of Search: 705/14.55, 705/14.53, 706/46, 725/5, 725/14, 715/700
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: H04H 60/46 for recognising users' preferences
• CPC Class Codes: H04N 21/251 Learning process for intelligent management, e.g. learning user preferences for recommending movies details of learning user preferences for the retrieval of video data in a video database G06F16/739; computer systems using learning methods G06N3/08
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/44226 on social networks
• CPC Class Codes: H04N 21/4662 characterized by learning algorithms
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38

Abstract

Techniques are shown for predicting the number of times a media selection will be consumed by one or more users at a target time. Examples of user behavior during the consumption of a media selection are chosen as input features. A partitioner separates a set of media selections into a training subset and an evaluation subset. The input features are transformed into feature vectors, and a learned function is derived to define a relationship between the feature vector for the training subset and the number of times a media selection from the training subset is consumed. The learned function is then applied to a feature vector for the evaluation subset to test its accuracy.

Claims

1. A method of facilitating prediction of a number of times a selection of media content will be consumed by one or more users at a target time, the method comprising:

identifying a set of media selections;

selecting a plurality of input features, the input features comprising examples of user behavior occurred during consumption of the identified set of media selections;

separating the identified set of media selections into a training set and an evaluation set;

upon separating the identified set of media selections into the training set and the evaluation set, transforming selected input features from the plurality of input features into a training set feature vector corresponding to a media selection from the training set, the training set feature vector comprising features descriptive of the user behavior during consumption of the media selection from the training set;

deriving a learned function defining a relationship between the training set feature vector corresponding to the media selection from the training set and a number of times the media selection from the training set is consumed at the target time; and

applying the learned function to a feature vector of input features from a media selection from the evaluation set to predict the number of times the media selection from the evaluation set is to be consumed at the target time.

2. The method according to claim 1, wherein:

the selecting examples of user behavior as input features comprises one or more of selecting the number of searches for other information conducted by a user during consumption of the media selection, selecting the number of times a user clicks on one or more links within the media selection, selecting the total number of media selections a user consumes during a time frame, selecting where in a series of media selections being consumed by a user the media selection is located, selecting whether the media selection was discovered by a user as a result of following a link from a social media site, selecting whether the media selection was discovered by a user as a result of a referral received by email, selecting whether the media selection was discovered independently by a user, selecting whether the media selection was consumed repeatedly by a particular user, selecting whether consumption of the media selection was ended prematurely as a result of a selection by a user to end consumption, or selecting whether a user makes annotations or performs other inputs during or following consumption of the media selection.

3. A method of facilitating prediction of an output value regarding a set of media selections consumed by one or more users, the method comprising:

determining a plurality of input features, the input features comprising examples of user behavior occurred during consumption of one or more media selections from the set of media selections;

selecting input features from the determined examples of user behavior to include in a learned function for predicting an output value from the selected input features;

partitioning the set of media selections into a training subset of media selections and an evaluation subset of media selections prior to determining a learned function for the prediction;

extracting the selected input features from the training subset of media selections;

determining an output value from the training subset;

transforming the extracted training subset input features into a training feature vector corresponding to a media selection from the training subset, the training feature vector containing multiple features descriptive of the user behavior during consumption of the media selection from the training subset;

determining the learned function that defines a relationship between the training feature vector corresponding to the media selection from the training subset and the output value from the training subset;

extracting the selected input features from the evaluation subset of media selections;

transforming the extracted evaluation subset input features into an evaluation feature vector containing multiple features descriptive of the user behavior during consumption of the media selections in the evaluation subset; and

applying the learned function to the evaluation feature vector to calculate a predicted output value for the evaluation subset of media selections.

4. The method according to claim 3, wherein:

the predicted output value for the evaluation subset of media selections is the number of times one or more media selections from the evaluation subset will be consumed by one or more users at a future target time.

5. The method according to claim 3, further comprising:

comparing the predicted output value for the evaluation subset of media selections to a measurement of the actual value of the output at the target time; and

determining whether the predicted output value is within a predetermined error threshold from the actual value of the output to evaluate performance of the learned function.

6. The method according to claim 3, wherein:

the determining examples of user behavior to include as representative input features comprises one or more of determining the number of searches for other information conducted by a user during consumption of the media selection, determining the number of times a user clicks on one or more links within the media selection, determining the total number of media selections a user consumes during a time frame, determining where in a series of media selections being consumed by a user the media selection is located, determining whether the media selection was discovered by a user as a result of following a link from a social media site, determining whether the media selection was discovered by a user as a result of a referral received by email, determining whether the media selection was discovered independently by a user, determining whether the media selection was consumed repeatedly by a particular user, determine whether consumption of the media selection was ended prematurely as a result of a selection by a user to end consumption, or determining whether a user makes annotations or performs other inputs during or following consumption of the media selection.

7. The method according to claim 3, wherein:

the media selections comprise video selections, and consumption of the media selections comprises viewing of the video selections.

8. The method according to claim 3, wherein:

the media selections comprise audio recordings, and consumption of the media selections comprises listening to the audio recordings.

9. The method according to claim 3, wherein:

the media selections comprise audio-visual recordings, and consumption of the media selections comprises watching and listening to the audio-visual recordings.

10. The method according to claim 3, wherein:

the transforming training subset input features into a training feature vector comprises aggregating all input features for all sessions of the same media selection.

11. The method according to claim 3, wherein:

the determining a learned function comprises classification wherein input features are categorized in accordance with their determined effect on the determined output value.

12. The method according to claim 3, wherein:

the determining a learned function comprises regression analysis wherein a determination is made of the amount the determined output value changes for a variance in one or more of the input features.

13. The method according to claim 3, wherein:

the determining a learned function comprises a consideration of the kind of information included in the extracted input features, wherein the kind of information can be one of integer-valued information, real-valued information, categorical information, or continuous information.

14. The method according to claim 3, wherein:

the determining a learned function comprises including different weightings for some of the extracted input features.

15. The method according to claim 3, wherein:

the determining a learned function comprises including consideration of any observable interactions between the extracted input features.

16. The method according to claim 3, further comprising:

the training subset of media selections and the evaluation subset of media selections are collected from one or more of local sources, remote sources, simultaneously, at different times, manually, or automatically.

17. A system for predicting the number of times a selection of media from a set of media selections will be consumed by one or more users at a future target time, the system comprising:

a processing device; and

a memory, coupled to the processing device, the memory to store;

a media selection utility configured to select a representative set of media selections for training and evaluation;

a media selection partitioning utility configured to partition the set of media selections into a training subset of media selections and an evaluation subset of media selections prior to determining a learned function for the predicting;

an input feature definitional utility configured to identify a plurality of input features, the input features comprising examples of user behavior occurring during consumption of the representative set of media selections, the input features for use in predicting the number of times a selection of media will be consumed by users;

a training set feature extractor configured to extract input features from the training subset of media selections;

a training set output value determiner configured to determine an output value from the training subset of media selections;

a feature vector creator configured to transform the extracted training subset input features into a training feature vector corresponding to a media selection from the training subset, the training feature vector containing multiple features descriptive of the user behavior during consumption of the media selection from the training subset;

a training set learned function derivation utility configured to determine the learned function that defines a relationship between the training feature vector corresponding to the media selection from the training subset and the determined output value from the training subset; and

an evaluation set learned function implementer configured to run the learned function on an evaluation feature vector and calculate a predicted output value for the evaluation subset of media selections.

18. The system according to claim 17, further comprising:

an evaluation set feature extractor configured to extract select input features from the evaluation subset of media selections; and

the feature vector creator further configured to transform the extracted evaluation subset input features into the evaluation feature vector containing multiple features descriptive of the user behavior during consumption of the media selections in the evaluation subset.

19. The system according to claim 18, further comprising:

an evaluation set output value comparator configured to compare the predicted output value for the evaluation subset of media selections to an actual measured output value for the evaluation subset of media selections to evaluate performance of the learned function.

20. The system according to claim 17, wherein:

the input feature definitional utility is configured to identify examples of user behavior during consumption of media selections as input features, wherein the examples of user behavior comprise one or more of the number of searches for other information conducted by a user during consumption of the media selection, the number of times a user clicks on links within the media selection, the total number of media selections a user consumes during a time frame, where in a series of media selections being consumed by a user the media selection is located, whether the media selection was discovered by a user as a result of following a link from a social media site, whether the media selection was discovered by a user as a result of a referral received by email, whether the media selection was discovered independently by a user, whether the media selection was consumed repeatedly by a particular user, whether consumption of the media selection was ended prematurely as a result of a selection by a user to end consumption, or whether a user makes annotations or performs other inputs during or following consumption of the media selection.

21. The system according to claim 17, wherein:

the media selection utility is configured to select video clips for viewing by users.

22. The system according to claim 17, wherein:

the media selection utility is configured to select audio recordings for listening by users.

23. The system according to claim 17, wherein:

the media selection utility is configured to select audio-visual recordings for viewing and listening to by users.

24. A non-transitory computer-readable media storing instructions that, when executed by a computing device, cause the computing device to perform operations that facilitate prediction of the number of times a selection of media from a set of media selections will be consumed by one or more users at a future target time, the operations comprising:

determining a plurality of input features, the input features comprising examples of user behavior occurring during consumption of the media selection;

selecting input features from the determined examples of user behavior to include in a learned function for predicting the number of times a selection of media will be consumed by one or more users at a target time;

partitioning the set of media selections into a training subset of media selections and an evaluation subset of media selections prior to determining a learned function for the prediction;

extracting the select input features from the training subset of media selections;

determining the number of times a media selection from the training subset of media selections is actually consumed by one or more users at a target time;

transforming the extracted training subset input features into a training feature vector corresponding to a media selection from the training subset, the training feature vector containing multiple features descriptive of the user behavior during consumption of the media selection from the training subset;

determining the learned function that defines a relationship between the training feature vector corresponding to the media selection from the training subset and the determined number of times a media selection from the training subset is consumed at a target time;

extracting the select input features from the evaluation subset of media selections;

transforming the extracted evaluation subset input features into an evaluation feature vector containing multiple features descriptive of the user behavior during consumption of the media selections in the evaluation subset; and

applying the learned function to the evaluation feature vector to calculate a predicted number of times a media selection from the evaluation subset of media selections will be consumed by one or more users at a target time.

25. The non-transitory computer-readable media according to claim 24, the operations further comprising:

the determining examples of user behavior to include as representative input features comprises one or more of determining the number of searches for other information conducted by a user during consumption of the media selection, determining the number of times a user clicks on links within the media selection, determining the total number of media selections a user consumes during a time frame, determining where in a series of media selections being consumed by a user the media selection is located, determining whether the media selection was discovered by a user as a result of following a link from a social media site, determining whether the media selection was discovered by a user as a result of a referral received by email, determining whether the media selection was discovered independently by a user, determining whether the media selection was consumed repeatedly by a particular user, determine whether consumption of the media selection was ended prematurely as a result of a selection by a user to end consumption, or determining whether a user makes annotations or performs other inputs during or following consumption of the media selection.

26. The non-transitory computer-readable media according to claim 24 storing instructions that, when executed by a computing device, cause the computing device to perform operations that facilitate predicting the number of times a video selection from a set of video selections will be viewed by one or more users.

27. The non-transitory computer-readable media according to claim 24 storing instructions that, when executed by a computing device, cause the computing device to perform operations that facilitate predicting the number of times an audio recording from a set of audio recordings will be listened to by one or more users.

28. The non-transitory computer-readable media according to claim 24 storing instructions that, when executed by a computing device, cause the computing device to perform operations that facilitate predicting the number of times an audio-visual recording from a set of audio-visual recordings will be viewed and listened to by one or more users.