Video content analysis for automatic demographics recognition of users and videos

• US 10,210,462 B2
• Filed: 11/24/2014
• Issued: 02/19/2019
• Est. Priority Date: 11/11/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Duque, Juan Carlos Niebles, Aradhye, Hrishikesh, Sbaiz, Luciano, Yagnik, Jay, Strobl, Reto
• Primary Examiner(s): Vincent, David R
• Application Number:
  US14/552,001
• Publication Number:
  US 20150081604A1
• Time in Patent Office: 1,548 Days
• Field of Search: 706 12, 706 20, 706 45
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18

Abstract

A demographics analysis trains classifier models for predicting demographic attribute values of videos and users not already having known demographics. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of videos using video features such as demographics of video uploaders, textual metadata, and/or audiovisual content of videos. In one embodiment, the demographics analysis system trains classifier models for predicting demographics of users (e.g., anonymous users) using user features based on prior video viewing periods of users. For example, viewing-period based user features can include individual viewing period statistics such as total videos viewed. Further, the viewing-period based features can include distributions of values over the viewing period, such as distributions in demographic attribute values of video uploaders, and/or distributions of viewings over hours of the day, days of the week, and the like.

Claims

1. A computer-implemented method of using a video classifier model that correlates demographic information with characteristics of video content to recommend videos, the method comprising:

accessing, using a hardware processor, a set of videos from a digital video repository;

for a given demographic attribute value of a demographic attribute, and for each video in the set of videos from the digital video repository, identifying, using the hardware processor, viewers of the video that have the demographic attribute value from a view log;

forming, using the hardware processor, a positive training set for the demographic attribute value by selecting videos in the set of videos from the digital video repository having a percentage of viewers that exceeds a threshold percentage value with the demographic attribute value and forming a negative training set for the demographic attribute value by selecting a subset of remaining videos from the digital video repository that are not included in the positive training set;

determining, using the hardware processor, for videos in the positive training set, features comprising demographic attribute values of uploaders of the videos in the positive training set;

training, using the hardware processor, the video classifier model for the demographic attribute value using the determined features of the videos of the positive training set;

determining, using the hardware processor, whether a given video is associated with the demographic attribute value by applying the trained video classifier model to the given video;

generating, using the hardware processor, a list of recommended videos for a viewer having the demographic attribute value, wherein the list of recommended videos includes the given video that was determined by the application of the trained video classifier model to be associated with the demographic attribute value; and

in response to detecting one or more viewers of the digital video repository that have the demographic attribute value, causing, using the hardware processor, the list of recommended videos to be presented to the one or more viewers.

2. The computer-implemented method of claim 1, further comprising:

for a new video for which a value of the demographic attribute is not already known, determining features comprising a demographic attribute value of an uploader of the new video;

determining whether the new video has the demographic attribute value by applying the video classifier model for the demographic attribute value to the determined features.

3. The computer-implemented method of claim 1, wherein identifying viewers of the video that have the demographic attribute value comprises identifying the demographic attribute value in user profiles of the viewers.

4. The computer-implemented method of claim 1, further comprising training a video classifier model for each of a plurality of other demographic attribute values of the demographic attribute.

5. The computer-implemented method of claim 4, further comprising:

for a new video for which the values of the demographic attribute are not already known;

determining features of the new video, the features comprising a demographic attribute value of an uploader of the new video; and

for each value of the other demographic attribute values of the demographic attribute;

obtaining a score by applying the video classifier model for the value to the determined features of the new video.

6. The computer-implemented method of claim 5, further comprising forming a demographic distribution for the demographic attribute using the obtained scores.

7. The computer-implemented method of claim 5, further comprising:

sorting the scores into an order; and

using the order to identify values of the demographic attribute to which the new video most strongly appeals.

8. A computer-readable storage medium storing instructions executable by a computer processor, the instructions comprising:

instructions for accessing a set of videos from a digital video repository;

instructions for, for a given demographic attribute value of a demographic attribute, and for each video in the set of videos from the digital video repository, identifying viewers of the video that have the demographic attribute value from a view log;

instructions for forming a positive training set for the demographic attribute value by selecting videos in the set of videos from the digital video repository having a percentage of viewers that exceeds a threshold percentage value with the demographic attribute value and forming a negative training set for the demographic attribute value by selecting a subset of remaining videos from the digital video repository that are not included in the positive training set;

instructions for determining, for videos in the positive training set, features comprising;

demographic attribute values of uploaders of the videos in the positive training set, and content features derived from audiovisual content of the videos in the positive training set;

instructions for training the video classifier model for the demographic attribute value using the determined features of the videos in the positive training set;

instructions for determining whether a given video is associated with the demographic attribute value by applying the trained video classifier model to the given video;

instructions for generating a list of recommended videos for a viewer having the demographic attribute value, wherein the list of recommended videos includes the given video that was determined by the application of the trained video classifier model to be associated with the demographic attribute value; and

instructions for causing the list of recommended videos to be presented to one or more viewers in response to detecting the one or more viewers of the digital video repository that have the demographic attribute value.

9. The computer-readable storage medium of claim 8, further comprising:

for a new video for which a value of the demographic attribute is not already known, determining features comprising a demographic attribute value of an uploader of the new video;

determining whether the new video has the demographic attribute value by applying the video classifier model for the demographic attribute value to the determined features.

10. The computer-readable storage medium of claim 8, wherein identifying viewers of the video that have the demographic attribute value comprises identifying the demographic attribute value in user profiles of the viewers.

11. The computer-readable storage medium of claim 8, further comprising training a video classifier model for each of a plurality of other demographic attribute values of the demographic attribute.

12. The computer-readable storage medium of claim 11, further comprising:

for a new video for which the values of the demographic attribute are not already known;

determining features of the new video, the features comprising a demographic attribute value of an uploader of the new video; and

for each value of the other demographic attribute values of the demographic attribute;

obtaining a score by applying the video classifier model for the value to the determined features of the new video.

13. The computer-readable storage medium of claim 12, further comprising forming a demographic distribution for the demographic attribute using the obtained scores.

14. The computer-readable storage medium of claim 12, further comprising:

sorting the scores into an order; and

using the order to identify values of the demographic attribute to which the new video most strongly appeals.

16. The computer-readable storage medium of claim 8, further comprising:

for a new video for which a value of the demographic attribute is not already known, determining features comprising a demographic attribute value of an uploader of the new video;

determining whether the new video has the demographic attribute value by applying the video classifier model for the demographic attribute value to the determined features.

17. The computer-readable storage medium of claim 8, wherein identifying viewers of the video that have the demographic attribute value comprises identifying the demographic attribute value in user profiles of the viewers.

18. The computer-readable storage medium of claim 8, further comprising training a video classifier model for each of a plurality of other demographic attribute values of the demographic attribute.

15. A computer system comprising:

a computer processor; and

a computer-readable storage medium having instructions executable by the computer processor, the instructions comprising;

instructions for accessing a set of videos from a digital video repository;

instructions for, for a given demographic attribute value of a demographic attribute, and for each video in the set of videos from the digital video repository, identifying viewers of the video that have the demographic attribute value from a view log;

instructions for forming a positive training set for the demographic attribute value by selecting videos in the set of videos from the digital video repository having a percentage of viewers that exceeds a threshold percentage value with the demographic attribute value and forming a negative training set for the demographic attribute value by selecting a subset of remaining videos from the digital video repository that are not included in the positive training set;

instructions for determining, for videos in the positive training set, features comprising;

demographic attribute values of uploaders of the videos in the positive training set, and content features derived from audiovisual content of the videos in the positive training set;

instructions for training the video classifier model for the demographic attribute value using the determined features of the videos in the positive training set;

instructions for determining whether a given video is associated with the demographic attribute value by applying the trained video classifier model to the given video;

instructions for generating a list of recommended videos for a viewer having the demographic attribute value, wherein the list of recommended videos includes the given video that was determined by the application of the trained video classifier model to be associated with the demographic attribute value; and

instructions for causing the list of recommended videos to be presented to one or more viewers in response to detecting the one or more viewers of the digital video repository that have the demographic attribute value.