Targeted advertisement based on face clustering for time-varying video

• US 8,769,556 B2
• Filed: 10/28/2011
• Issued: 07/01/2014
• Est. Priority Date: 10/28/2011
• Status: Active Grant

• Current Assignee: Symbol Technologies, LLC (Zebra Technologies Corporation)
• Original Assignee: Motorola Solutions, Inc.
• Inventors: Guo, Feng, Super, Boaz J., Arora, Himanshu
• Primary Examiner(s): Goodarzi, Nasser
• Assistant Examiner(s): PIERORAZIO, MICHAEL
• Application Number:
  US13/284,282
• Publication Number:
  US 20130111509A1
• Time in Patent Office: 977 Days
• Field of Search: 725 9- 13, 348169-172, 348/262, 348/335, 348340-344, 348359-361, 348372-376, 348/207, 3482151-2201, 209/569, 209583-584, 382100-154, 382162-172, 382/181, 382187-189, 382251-253, 382318-319, 382/325, 382/225, 434/116
• US Class Current: 725/10
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: H04H 60/45 for identifying users
• CPC Class Codes: H04H 60/65 for using the result on users' side
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/44226 on social networks
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

A method and apparatus for providing targeted advertisements is provided herein. In particular, targeted advertisements are provided to users based on face clustering for time-varying video. During operation video is continuously obtained of users of the system. Users'"'"' faces are detected and measured. Measurements of users'"'"' faces are then clustered. Once the clusters are available, advertisements are targeted at clusters rather than individual users.

Claims

1. A method for associating an action with facial data obtained from a video feed, the method comprising the steps of:

receiving the video feed;

detecting faces of users within the video feed;

representing each detected face as data comprising one or more attributes for each face;

representing the data in an n-dimensional vector space, wherein an n-dimensional vector space has one or more dimensions;

performing a clustering method to group the data to form a set of clusters within the n-dimensional vector space, wherein there exists one cluster per user and one user per cluster, multiple clusters per user, or multiple users per single cluster within the n-dimensional vector space;

after formation of the set of clusters, performing the steps of;

detecting an additional face of a user;

representing the additionally-detected face as additional data comprising one or more attributes;

representing the additional data in an n-dimensional vector space;

determining whether the additional face is associated with at least one of the clusters in the set of clusters in the n-dimensional vector space and if associated with at least one of the clusters in the set of clusters, modifying the associated cluster, otherwise determining whether the additional face is not associated with any of the clusters in the set of clusters, and if not associated with any of the clusters in the set of clusters, inserting the additional data into a buffer; and

associating an action to a cluster in the set of clusters in the n-dimensional vector space.

2. The method of claim 1 additionally comprising:

determining whether the additional face is ambiguously associated with a cluster in the set of clusters.

3. The method of claim 2 further comprising the step of:

discarding the additional data.

4. The method of claim 1 wherein determining whether the additional face is associated with at least one of the clusters in the set of clusters in the n-dimensional vector space comprises:

determining whether the additional face has a spatial overlap with a detected face in a previous frame of the video feed, where the detected face in a previous frame is associated with a cluster in the set of clusters.

5. The method of claim 1 additionally comprising:

determining whether two faces are associated with the same cluster in the set of clusters; and

discarding the two faces.

6. The method of claim 1 additionally comprising:

creating a classifier for a cluster in the set of clusters; and

classifying the additional face.

7. The method of claim 1 additionally comprising:

determining whether a buffer is full of data; and

adding a cluster to the set of clusters when the buffer is full.

8. The method of claim 1 additionally comprising:

determining whether a buffer contains at least a predetermined number of faces, a buffer Is using at least a predetermined amount of memory, or a predetermined amount of video from the video feed has been processed; and

adding a cluster to the set of clusters.

9. The method of claim 1 wherein the step of associating the action comprises the step of associating a playing of an advertisement based on a cluster in the set of clusters in the n-dimensional vector space.

10. The method of claim 1 further comprising the step of:

applying a k-means algorithm for at least one value of k and selecting a value of k based on a normalized difference between average dissimilarities of data points to their own clusters and their average dissimilarities to points of a most similar cluster other than their own.

11. The method of claim 1 further comprising the step of:

accessing a database to determine targeted advertisements to play based on a cluster in the set of clusters in the n-dimensional vector space.

12. An apparatus comprising:

a detector, wherein the detector is configured for receiving the video feed, detecting faces of users within the video feed, representing each detected face as data comprising one or more attributes for each face, representing the data in an n-dimensional vector space, wherein an n-dimensional vector space has one or more dimensions, performing a clustering method to group the data to form a set of clusters within the n-dimensional vector space, after formation of the set of clusters, performing the steps of detecting an additional face of a user, representing the additionally-detected face as additional data comprising one or more attributes representing the additional data in an n-dimensional vector space, determining whether the additional face is associated with at least one of the clusters in the set of clusters in the n-dimensional vector space and if associated with at least one of the clusters in the set of clusters, modifying the associated cluster, otherwise determining whether the additional face is not associated with any of the clusters in the set of clusters, and associating an action to a cluster in the set of clusters in the n-dimensional vector space,

wherein there exists one cluster per user and one user per cluster, multiple clusters per user, or multiple faces in a single cluster within the n-dimensional feature space; and

a buffer for storing the additional data when the additional data is not associated with any of the clusters in the set of clusters.

13. The apparatus of claim 12 wherein the action comprises a playing of an advertisement to be played with the identified clusters.

14. The apparatus of claim 12 wherein the advertisements are associated with the identified clusters by determining programs watched for various clusters.

15. The apparatus of claim 12 wherein the advertisements are associated with the identified clusters by determining what advertisements are skipped for each cluster.

16. The apparatus of claim 12 wherein the advertisements are associated with the identified clusters by determining known physical or social traits typically associated with the cluster characteristics.

17. The apparatus of claim 16 wherein the detector accumulates faces that do not belong to a cluster into the storage, and later creates additional clusters from the faces in the storage.