System and method to assign a digital image to a face cluster

• US 8,781,179 B2
• Filed: 10/29/2012
• Issued: 07/15/2014
• Est. Priority Date: 12/02/2009
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Liu, Zhu, Begeja, Lee
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US13/662,790
• Publication Number:
  US 20130070975A1
• Time in Patent Office: 624 Days
• Field of Search: None
• US Class Current: 382/118
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A method includes accessing a digital image including a plurality of faces including a first face and a second face. The method includes identifying a plurality of identification regions of the digital image including a first identification region associated with the first face and a second identification region associated with the second face. The method also includes assigning the digital image to a first face cluster of a plurality of face clusters when a difference between data descriptive of the first identification region and data descriptive of a face cluster identification region of the first face cluster satisfies a threshold. The method further includes assigning the digital image to a second face cluster of the plurality of face clusters based at least partially on a frequency of the second face and the first face appearing together in an image.

Claims

1. A method comprising:

at a computer, accessing a digital image representing a plurality of faces including a first face and a second face, wherein the digital image is part of a frame of video data;

at the computer, identifying a plurality of identification regions of the digital image including a first identification region associated with the first face and a second identification region associated with the second face;

at the computer, assigning the digital image to a first face cluster of a plurality of face clusters when a difference between data descriptive of the first identification region and data descriptive of a face cluster identification region of the first face cluster satisfies a threshold; and

at the computer, assigning the digital image to a second face cluster of the plurality of face clusters based on an analysis of a portion of the video data to determine a frequency at which the first face and the second face appear together.

2. The method of claim 1, wherein the video data comprises a sequence of shots and wherein the portion of the video data that is analyzed corresponds to a particular shot of the sequence of shots.

3. The method of claim 1, further comprising determining a probability of the second face and the first face appearing together in the digital image by;

accessing a face cluster library that includes the first face cluster and the second face cluster;

determining a number of images of the face cluster library that are assigned to both the first face cluster and the second face cluster in which the first face and the second face appear together; and

determining the probability that the second face and the first face appear together in the digital image based on the number of images.

4. The method of claim 3, wherein the face cluster library includes images selected from two or more video data files.

5. The method of claim 1, further comprising, after assigning the digital image to the first face cluster, analyzing images assigned to the first face cluster to select a particular digital image of the first face cluster as a representative image of the first face cluster, the representative image having a difference value with respect to other digital images of the first face cluster that is smaller than difference values of each of the other digital images.

6. The method of claim 1, further comprising iteratively merging the plurality of face clusters based on difference values descriptive of differences between digital images assigned to each of the plurality of face clusters until a minimum difference value between each of the plurality of face clusters fails to satisfy a second threshold.

7. The method of claim 6, wherein each of the digital images is initially assigned to its own face cluster.

8. The method of claim 1, further comprising determining eigenface data for the first identification region, wherein the difference is based on the eigenface data.

9. The method of claim 1, wherein identifying the plurality of identification regions of the digital image includes:

searching for faces in the digital image;

when the first face is detected, selecting a first portion of the digital image based on characteristics of the first face as a first face region of the plurality of identification regions; and

determining a data set descriptive of the first face region based on the first portion of the digital image.

10. The method of claim 9, further comprising selecting a second portion of the digital image that includes the first face, the second portion larger than the first portion, wherein the second portion comprises a head region of the plurality of identification regions.

11. The method of claim 10, wherein the first portion has a first center approximately aligned with a second center of the second portion, wherein the first portion has a first characteristic length based on the characteristics of the first face, and wherein the second portion has a second characteristic length that is a predetermined multiple of the first characteristic length.

12. The method of claim 11, wherein the first characteristic length is approximately a distance between eyebrows and mouth of the first face in the digital image.

13. The method of claim 9, further comprising selecting a third portion of the digital image that does not include the first face, wherein the third portion comprises a torso region of the plurality of identification regions.

14. The method of claim 13, wherein the first portion has a first characteristic length based on the characteristics of the first face, wherein the third portion has a third characteristic length that is a predetermined multiple of the first characteristic length, and wherein the third characteristic length specifies dimensions of the third portion and an offset of the third portion from the first portion.

15. A computer-readable storage device, comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:

accessing digital image including a plurality of faces including a first face and a second face, wherein the digital image is part of a frame of video data;

identifying a plurality of identification regions of the digital image including a first identification region associated with the first face and a second identification region associated with the second face;

assigning the digital images to a first face cluster of a plurality of face clusters when a difference between data descriptive of the first identification region and data descriptive of a face cluster identification region of the first face cluster satisfies a threshold; and

assigning the digital image to a second face cluster of the plurality of face clusters based on an analysis of a portion of the video data to determine a frequency at which the first face and the second face appear together.

16. The computer-readable storage device of claim 15, wherein assigning the digital image to the first face cluster comprises:

applying a weighting to data descriptive of identification regions associated with the first face to create weighted data descriptive of identification regions associated with the first face; and

determining a first difference between the weighted data descriptive of the identification regions associated with the first face and the data descriptive of the face cluster identification region.

17. A system, comprising:

a processor;

a memory accessible to the processor, the memory including processor-executable instructions that, when executed by the processor, cause the processor to perform operations comprising;

accessing a digital image including a plurality of faces including a first face and a second face, wherein the digital image is part of a frame of video data;

identifying a plurality of identification regions of the digital image including a first identification region associated with the first face and a second identification region associated with the second face;

assigning the digital images to a first face cluster of a plurality of face clusters based on a comparison between data descriptive of the first identification region and data descriptive of a face cluster identification region of the first face cluster; and

assigning the digital image to a second face cluster of the plurality of face clusters based on an analysis of a portion of the video data to determine a frequency at which the first face and the second face appear together.

18. The system of claim 17, wherein a data record associated with the digital image includes information identifying a location of the digital image within the video data.

19. The system of claim 17, wherein the first face cluster and the second face cluster are accessed from a face cluster library.

20. The system of claim 19, wherein the face cluster library includes images from a plurality of video data files.