Video content analysis for automatic demographics recognition of users and videos

• US 8,924,993 B1
• Filed: 11/10/2011
• Issued: 12/30/2014
• Est. Priority Date: 11/11/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Aradhye, Hrishikesh Balkrishna, Sbaiz, Luciano, Yagnik, Jay N., Strobl, Reto, Niebles Duque, Juan Carlos
• Primary Examiner(s): Flynn, Nathan
• Assistant Examiner(s): Parra, Omar S
• Application Number:
  US13/294,156
• Time in Patent Office: 1,146 Days
• Field of Search: None
• US Class Current: 725/9
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18

Abstract

A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics.

In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos.

In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.

Claims

1. A computer-implemented method for generating a user classifier model, the method comprising:

for each viewer of a plurality of viewers of digital videos, the viewers including viewers from a plurality of different households, and for a demographic attribute;

identifying a value of the demographic attribute for the viewer;

identifying a period of time during which the viewer viewed digital videos via a video hosting service;

identifying a plurality of digital videos viewed by the viewer during the identified period of time;

for the identified period of time, determining a set of features corresponding to the viewed digital videos, the features comprising content features derived from audiovisual content of the plurality of digital videos viewed during the period, and

view-block features related to a division of the plurality of digital videos into subsets of videos viewed within a threshold amount of time of each other; and

associating the determined set of features with the demographic attribute value; and

for each demographic attribute value of a plurality of the demographic attribute values identified for the demographic attribute, training, by a computer processor, a user classifier model for the value of the demographic attribute using the sets of features associated with the demographic attribute value as input for an ensemble classifier training algorithm.

2. The computer-implemented method of claim 1, further comprising:

for an anonymous viewer for whom a value of the demographic attribute is not already known, identifying a period of time during which the anonymous viewer viewed videos;

identifying a plurality of videos viewed by the anonymous viewer during the identified period;

determining a set of features for the period of time comprising at least one of distributions of attributes of the plurality of videos viewed during the period, and view-block features related to a division of the plurality of videos into a set of view-blocks, each view-block defining a subset of videos viewed within a threshold amount of time of each other; and

determining whether the anonymous viewer has the demographic attribute value by applying the user classifier model for the demographic attribute value to the determined set of features.

3. The computer-implemented method of claim 1, wherein the features include at least one of a set consisting of:

a distribution of viewings of the plurality of videos viewed by the viewer over hours of a day, and a distribution of viewings of the plurality of videos viewed by the viewer over days of a week.

4. The computer-implemented method of claim 1, wherein the view-block features comprise distributions of viewings of the plurality of videos viewed by the viewer over the view-blocks.

5. The computer-implemented method of claim 1, wherein the view-block features comprise at least one of a set consisting of:

an average number of videos viewed per view-block, a median time duration of the view-blocks, and a total number of the view-blocks.

6. The computer-implemented method of claim 1, wherein identifying a value of the demographic attribute for the viewer comprises identifying the value in a user profile of the viewer, the value in the user profile having been specified by the user.

7. The computer-implemented method of claim 1, wherein the set of features comprises distributions of attributes of the plurality of digital videos viewed during the period.

8. The computer-implemented method of claim 7, wherein the distributions of attributes comprise distributions of demographic attribute values of uploaders of the plurality of videos viewed by the viewer.

9. The computer-implemented method of claim 7, further comprising:

obtaining a set of category scores for each of the plurality of videos viewed by the viewer by applying a set of category classifier models to the plurality of videos viewed by the viewer;

wherein the distributions of attributes comprise distributions of categories represented by the category scores.

10. The computer-implemented method of claim 7, further comprising:

obtaining, for each video of the plurality of videos viewed by the viewer, a score for a demographic attribute value by applying a video classifier model to the video, the video classifier model corresponding to the demographic attribute value and producing a score representing how strongly the video represents the demographic attribute value; and

forming a demographic distribution of the scores;

wherein the distributions of attributes comprise the demographic distribution.

11. A non-transitory computer-readable storage medium having executable computer program instructions embodied therein for generating a user classifier model, actions of the computer program instructions comprising:

for each viewer of a plurality of viewers of digital videos, the viewers including viewers from a plurality of different households, and for a demographic attribute;

identifying a value of the demographic attribute for the viewer;

identifying a period of time during which the viewer viewed digital videos via a video hosting service;

identifying a plurality of digital videos viewed by the viewer during the identified period of time;

for the identified period of time, determining a set of features corresponding to the viewed digital videos, the features comprising;

view-block features related to a division of the plurality of digital videos into subsets of videos viewed within a threshold amount of time of each other; and

associating the determined set of features with the demographic attribute value; and

training a user classifier model for a value of the demographic attribute using the sets of features associated with the demographic attribute value.

12. The non-transitory computer-readable storage medium of claim 11, the actions further comprising:

for an anonymous viewer for whom a value of the demographic attribute is not already known, identifying a period of time during which the anonymous viewer viewed videos;

identifying a plurality of videos viewed by the anonymous viewer during the identified period;

determining a set of features for the period of time comprising at least one of distributions of attributes of the plurality of videos viewed during the period, and view-block features related to a division of the plurality of videos into a set of view-blocks, each view-block defining a subset of videos viewed within a threshold amount of time of each other; and

determining whether the anonymous viewer has the demographic attribute value by applying the user classifier model for the demographic attribute value to the determined set of features.

13. The non-transitory computer-readable storage medium of claim 11, wherein the features include at least one of a set consisting of:

a distribution of viewings of the plurality of videos viewed by the viewer over hours of a day, and a distribution of viewings of the plurality of videos viewed by the viewer over days of a week.

14. The non-transitory computer-readable storage medium of claim 11, wherein the view-block features comprise distributions of viewings of the plurality of videos viewed by the viewer over the view-blocks.

15. The non-transitory computer-readable storage medium of claim 11, wherein the view-block features comprise at least one of a set consisting of:

an average number of videos viewed per view-block, a median time duration of the view-blocks, and a total number of the view-blocks.

16. The non-transitory computer-readable storage medium of claim 11, wherein the features comprise distributions of demographic attribute values of uploaders of the plurality of videos viewed by the viewer.

17. A computer system for generating a user classifier model, the system comprising:

a computer processor; and

a computer-readable storage medium having executable computer program instructions embodied therein that when executed by the computer processor perform actions comprising;

for each viewer of a plurality of viewers of digital videos, the viewers including viewers from a plurality of different households, and for a demographic attribute;

identifying a value of the demographic attribute for the viewer;

identifying a period of time during which the viewer viewed digital videos via a video hosting service;

identifying a plurality of digital videos viewed by the viewer during the identified period of time;

for the identified period of time, determining a set of features corresponding to the viewed digital videos, the features comprising at least one feature selected from the group consisting of;

distributions of attributes of the plurality of digital videos viewed during the period, and

view-block features related to a division of the plurality of digital videos into subsets of videos viewed within a threshold amount of time of each other; and

associating the determined set of features with the demographic attribute value; and

training a user classifier model for a value of the demographic attribute using the sets of features associated with the demographic attribute value.

18. The computer system of claim 17, the actions further comprising:

obtaining a set of category scores for each of the plurality of videos viewed by the viewer by applying a set of category classifier models to the plurality of videos viewed by the viewer;

wherein the distributions of attributes comprise distributions of categories represented by the category scores.

19. The computer system of claim 17, the actions further comprising:

obtaining, for each video of the plurality of videos viewed by the viewer, a score for a demographic attribute value by applying a video classifier model to the video, the video classifier model corresponding to the demographic attribute value and producing a score representing how strongly the video represents the demographic attribute value; and

forming a demographic distribution of the scores;

wherein the distributions of attributes comprise the demographic distribution.

20. The computer system of claim 17, wherein identifying a value of the demographic attribute for the viewer comprises identifying the value in a user profile of the viewer.