Video content analysis for automatic demographics recognition of users and videos

• US 8,301,498 B1
• Filed: 06/04/2012
• Issued: 10/30/2012
• Est. Priority Date: 01/27/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cortes, Corinna, Kumar, Sanjiv, Makadia, Ameesh, Mann, Gideon, Yagnik, Jay, Zhao, Ming
• Primary Examiner(s): Rosen, Elizabeth
• Assistant Examiner(s): GOODBODY, JOAN T
• Application Number:
  US13/488,126
• Publication Number:
  US 20120272259A1
• Time in Patent Office: 148 Days
• Field of Search: 705/14.49
• US Class Current: 705/14.49
• CPC Class Codes: G06F 16/735 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/783 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 16/787 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/4668 for recommending content, e.g. movies
• CPC Class Codes: H04N 21/4826 using recommendation lists, e.g. of programs or channels sorted out according to their score

Abstract

A video demographics analysis system selects a training set of videos to use to correlate viewer demographics and video content data. The video demographics analysis system extracts demographic data from viewer profiles related to videos in the training set and creates a set of demographic distributions, and also extracts video data from videos in the training set. The video demographics analysis system correlates the viewer demographics with the video data of videos viewed by that viewer. Using the prediction model produced by the machine learning process, a new video about which there is no a priori knowledge can be associated with a predicted demographic distribution specifying probabilities of the video appealing to different types of people within a given demographic category, such as people of different ages within an age demographic category.

Claims

1. A computer-implemented method of generating a prediction model for videos, comprising:

accessing a plurality of videos stored in a video repository, each video associated with a plurality of viewers who have viewed the video;

for each video, creating a demographic distribution for at least one demographic attribute based at least in part on viewer demographic data associated with the plurality of viewers who have viewed the video;

for each video, generating feature vectors based at least in part on video content of frames of the video;

generating by a computer system a prediction model that correlates the feature vectors for the plurality of videos and the demographic distributions for the plurality of videos; and

storing the prediction model.

2. The computer-implemented method of claim 1, wherein the demographic attribute is one of age and gender.

3. The computer-implemented method of claim 1, wherein the demographic attribute is one of occupation, household income, and location.

4. The computer-implemented method of claim 1, wherein the prediction model is generated using support vector machines.

5. The computer-implemented method of claim 1, further comprising altering the generated feature vectors using a dimensionality reduction algorithm.

6. The computer-implemented method of claim 1, wherein the generated feature vectors further include features vectors generated based on audio content of the video.

7. The computer-implemented method of claim 1, wherein the generated feature vectors include feature vectors that are generated at least in part on metadata associated with the video.

8. The computer-implemented method of claim 1, further comprising:

performing object segmentation on a frame of the video, thereby identifying a visual object of the frame;

wherein generating feature vectors based at least in part on the content of the video comprises generating feature vectors for the identified visual object.

9. A computer-implemented method for determining demographics of a video, comprising:

accessing a prediction model that correlates viewer demographic attributes with feature vectors extracted from a plurality of videos viewed by a plurality of viewers, wherein the viewer demographic attributes include age and gender;

receiving a video;

generating, from video content of frames of the video, a set of feature vectors; and

identifying, by a computer system, demographic attribute values by applying the prediction model to the generated set of feature vectors.

10. The computer-implemented method of claim 9, wherein identifying demographic attribute values comprises:

identifying a set of feature vectors of the prediction model that is most similar to the generated set of feature vectors; and

identifying, in the prediction model, demographic attribute values most strongly correlated with the identified feature vectors.

11. A computer-implemented method for identifying demographics associated with a viewer, comprising:

accessing a prediction model that correlates viewer demographic attributes with feature vectors generated from a plurality of videos viewed by a plurality of viewers;

identifying a set of videos viewed by a given viewer;

generating, from video content of frames of the set of videos, a set of feature vectors;

applying the feature vectors to the prediction model to identify viewer demographic attribute values most strongly correlated with the feature vectors of the prediction model; and

identifying, by a computer system, viewer demographic attribute values most strongly correlated with the given viewer based at least in part on the identified viewer demographic attribute values.

12. A computer-implemented method for identifying videos associated with given demographic attribute values, comprising:

accessing a prediction model that correlates viewer demographic attributes with feature vectors generated from a plurality of videos viewed by a plurality of viewers;

receiving a plurality of videos;

for each video of the plurality of received videos;

generating, from video content of frames of the video, a set of feature vectors;

applying the feature vectors generated from the video to the prediction model to identify viewer demographic attribute values most strongly correlated with the feature vectors of the prediction model;

storing the identified viewer demographic attribute values in association with the video;

selecting, by a computer system, videos having highest values for the given demographic attribute values; and

displaying identifiers of the selected videos.

13. A non-transitory computer readable storage medium storing a computer program executable by a processor for generating a prediction model for videos, the actions of the computer program comprising:

accessing a plurality of videos stored in a video repository, each video associated with a plurality of viewers who have viewed the video;

for each video, creating a demographic distribution for at least one demographic attribute based at least in part on viewer demographic data associated with the plurality of viewers who have viewed the video;

for each video, generating feature vectors based at least in part on video content of frames of the video;

generating by a computer system a prediction model that correlates the feature vectors for the plurality of videos and the demographic distributions for the plurality of videos; and

storing the prediction model.

14. The non-transitory computer readable storage medium of claim 13, wherein the generated feature vectors include features vectors generated based on audio content of the video.

15. The non-transitory computer readable storage medium of claim 13, wherein the prediction model is generated using support vector machines.

16. A computer system for generating a prediction model for videos, comprising:

a computer processor;

a video repository storing a plurality of videos, each video associated with a plurality of viewers who have viewed the video;

a non-transitory computer-readable storage medium storing executable instructions that when executed by the computer processor perform actions comprising;

accessing a plurality of videos stored in the video repository;

for each video, creating a demographic distribution for at least one demographic attribute based at least in part on viewer demographic data associated with the plurality of viewers who have viewed the video;

for each video, generating feature vectors based at least in part on video content of frames of the video;

generating a prediction model that correlates the feature vectors for the plurality of videos and the demographic distributions for the plurality of videos; and

storing the prediction model.

17. The computer system of claim 16, wherein the demographic attribute is one of age and gender.

18. The computer system of claim 16, wherein the prediction model is generated using support vector machines.

19. The computer system of claim 16, wherein the generated feature vectors include features vectors generated based on audio content of the video.

20. The computer system of claim 16, wherein the generated feature vectors include feature vectors that are generated at least in part on metadata associated with the video.