Enabling persistent recognition of individuals in images

• US 8,180,112 B2
• Filed: 01/21/2008
• Issued: 05/15/2012
• Est. Priority Date: 01/21/2008
• Status: Expired due to Fees

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Kurtz, Andrew Frederick, Das, Madirakshi
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/017,131
• Publication Number:
  US 20090185723A1
• Time in Patent Office: 1,576 Days
• Field of Search: 382/118, 382/103, 382/115, 382/116, 382/117, 382/155, 382/218, 382/228
• US Class Current: 382/108
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition
• CPC Class Codes: G06V 40/50 Maintenance of biometric data or enrolment thereof

Abstract

A method and system for accumulating and using images of individuals, and associated image-derived data, on an continuous basis to create recognition models that facilitate ongoing recognition of individuals in images or photo collections.

Claims

1. A computer implemented method for enabling computer recognition of a particular person in images, comprising the steps of:

analyzing one or more facial images of a person using a facial recognition model associated with the person to identify the person and to determine changes in facial appearance of the person;

calculating a subject recognition confidence value for the person responsive to the facial recognition model and the determined changes in facial appearance present in the one or more facial images, wherein the subject recognition confidence value is calculated from similarity score distributions, the similarity score distributions being probability distributions indicative of the probability of matching instances of the same person and the probability of matching instances of different persons;

comparing the calculated subject recognition confidence value for the person to prior subject recognition confidence values calculated using prior images of the person to identify changes in the subject recognition confidence values;

analyzing the identified changes in the subject recognition confidence values of determine whether a new facial recognition model for the person should be generated based on the determined changes in the facial appearance of the person;

generating the new facial recognition model from current of subsequent images of the person is it is so determined by the step of analyzing the identified changes; and

using the new facial recognition model for subsequent person recognition tasks.

2. The method according to claim 1 wherein the step of analyzing the identified changes includes tracking changes in the subject recognition confidence values over time to identify significant changes in the confidence values.

3. The method according to claim 2 further comprising the step of measuring significant changes in the subject recognition confidence values relative to change detection criteria including total change, percent change, threshold change, rate of change, or duration of change.

4. The method according to claim 1 wherein the step of calculating the subject recognition confidence value includes using normalized ratios of probabilities calculated from the similarity score distributions.

5. The method according to claim 4 wherein the similarity score distributions are household similarity score distributions determined for a set of persons that are members of a household.

6. The method according to claim 5 wherein the step of calculating similarity score distributions includes-using-pair-wise facial classifiers for one or more pairs of individuals in the household.

7. The method according to claim 1 further comprising the step of analyzing the facial images relative to one or more defined facial metrics, wherein the facial metrics are associated with predetermined facial features that are tracked using a facial recognition model.

8. The method according to claim 1 further comprising the step of acquiring the plurality of facial images of the person from an image capture device, an image collection, or a combination thereof.

9. The method according to claim 1 wherein the facial recognition model is selected from a 3-D composite model, an active shape model, an active appearance model, or combinations thereof.

10. The method according to claim 1 wherein at least two different facial recognition models are concurrently provided for the person, each facial recognition model corresponding to a different appearance condition of the person.

11. The method according to claim 1 wherein the new facial recognition model is generated by updating the existing facial recognition model to include metadata that is indicative of the changes in appearance of the person or to include an auxiliary facial recognition model that is indicative of the changes in appearance of the person.

27. The method according to claim 1 wherein the changes in the appearance of the person include facial texture changes, facial geometry changes, facial coloring changes, changes due to the use of cosmetics, changes resulting from the presence, absence or styling of hair, or combinations thereof.

28. The method according to claim 1 wherein the new facial recognition models are used in person recognition tasks that include providing identification of the person in new images or in images that are present within or being added to an image collection.

12. A computer implemented method for enabling computer recognition of a particular person in images, comprising the steps of:

acquiring at least one digital facial image of the person using an image capture device;

analyzing the at least one digital facial image of the person relative to a predetermined facial recognition model associated with the person to identify the person and to determine changes in facial appearance of the person;

calculating a subject recognition confidence value for the person responsive to the facial recognition model and the determined changes in facial appearance present in the at least on facial image, wherein the subject recognition confidence value is calculated from similarity score distributions, the similarity score distributions being probability distributions indicative of the probability of matching instances of the same person and the probability of matching instances of different persons;

comparing the calculated subject recognition confidence value for the person to prior subject recognition confidence values calculated using prior images of the person to identify changes in the subject recognition confidence values;

analyzing the identified change in the subject recognition confidence values to determine whether a new facial recognition model for the person should be generated based on the determined changes in the facial appearance of the person;

acquiring at least one additional facial image of the person using the image capture device;

generating the new facial recognition model using at least the one additional facial image of the person; and

using the new facial recognition model for subsequent person recognition tasks.

13. The method according to claim 12 wherein the step of analyzing the identified change includes tracking changes in the subject recognition confidence values over time to identify significant changes in the subject recognition confidence values.

14. The method according to claim 12 wherein the step of calculating the subject recognition confidence value includes using normalized ratios of probabilities calculated from the similarity score distributions.

15. The method according to claim 12 further comprising the step of analyzing the facial images relative to one or more defined facial metrics, wherein the facial metrics are associated with predetermined facial features that are tracked using a facial recognition model.

16. The method according to claim 12, wherein the image capture device is a device selected from an electronic picture frame, a video phone, a web camera, or a digital camera.

17. The method according to claim 12 wherein the step of generating the new facial recognition model comprises the step of normalizing the facial images acquired by the image capture device relative to grey scale and size.

18. The method according to claim 12 wherein the additional facial image of the person are tested to determine whether they have an image quality level sufficient for use in generating the new facial recognition model.

19. The method according to claim 12 wherein the facial recognition model is selected from a 3-D composite model, an active shape model, an active appearance model, or combinations thereof.

20. The method according to claim 12 further comprising the step of identifying whether the person is a subject for whom a facial recognition model exists or a non-subject for whom a facial recognition model does not exist.

21. The method according to claim 20 wherein the step of identifying whether the person is a subject or non-subject includes the step of utilizing a reference image comprising selected facial features to represent the facial appearance.

22. The method according to claim 20 further comprising the steps of providing the image capture device with a microphone that enables capture of voice characteristics of the person, and analyzing the voice characteristics to provide recognition cues for the person.

29. The method according to claim 12 wherein at least two different facial recognition models are concurrently provided for the person, each facial recognition model corresponding to a different appearance condition of the person.

30. The method according to claim 13 wherein the identification of significant changes in the subject recognition confidence values is responsive to one or more change detection criteria including a total change criterion, a percent change criterion, a threshold change criterion, a rate of change criterion, or a duration of change criterion.

31. The method according to claim 14 wherein the similarity score distributions are determined for a plurality of persons that are members of a household.

32. The method according to claim 31 wherein the step of calculating similarity score distributions includes pair-wise facial classifiers for at least one pair of persons in the household.

23. A non-transitory tangible computer readable storage medium storing executable instructions for causing a processor to perform a method for enabling computer recognition of a particular person in images, said method comprising:

accessing a facial recognition model for a person and associated prior subject recognition confidence values calculated using prior images of the person from processor-accessible memory;

analyzing one or more facial images of the person using the facial recognition model for the person to identify the person and to determine changes in facial appearance of the person;

calculating a subject recognition confidence value for the person responsive to the facial recognition model and the determined changes in facial appearance present in the one or more facial images, wherein the subject recognition confidence value is calculated from similarity score distributions indicative of the probability of matching instances of the same person and the probability of match instances of different persons;

comparing the calculated subject recognition confidence value for the person to the prior subject recognition confidence values to identify changes in the subject recognition confidence values;

analyzing the identified changes in the subject recognition confidence values to determine whether a new facial recognition model for the person should be generated based on the identified changes in the appearance of the person;

generating the new facial recognition model from current of subsequent images of the person if it is so determined by the step of analyzing the identified changes; and

storing the new facial recognition model in the processor accessible memory for use with subsequent person recognition tasks.

24. A system comprising:

an image capture device for capturing facial images of a person;

storing memory coupled to the image capture device for storing the captured facial images of the person. a previously generated facial recognition model for the person, and one or more prior subject recognition confidence values calculated using prior images of the person;

means for calculating a subject recognition confidence values for the person responsive to the facial recognition model and the determined changes in the facial appearance present in the one or more captured facial images, wherein the subject recognition confidence value is calculated from similarity score distributions, the similarity score distributions being probability distributions indicative of the probability of matching instances of the same person and the probability of matching instances of different persons;

means for comparing the calculated subject recognition confidence value to the prior subject recognition confidence values to identify changes in the subject recognition confidence values and to determine whether a new facial recognition model for the person should be generated based on the determined changes in the facial appearance of the person; and

means for generating a new facial recognition model using current or subsequent images of the person if a need is so determined by the comparison means.

25. A computer implemented method for enabling computer recognition of a person in images, the person being a member of a household including a plurality of persons, comprising the steps of:

generating facial recognition models for two or more persons in the household;

generating at least one pair-wise facial classifier for at least one pair of persons in the household, the pair-wise facial classifier using at least one facial feature that discriminates appearance differences for the pair of persons;

analyzing one or more facial images of the person using the facial recognition models and the pair-wise facial classifier to identify the person; and

further to identify changes facial appearance attributes present in the one or more facial images;

calculating a subject recognition confidence value for the person responsive to the facial recognition model and the identified changes in the facial appearance attributes of the person in the one or more facial images;

comparing the calculated subject recognition confidence value for the person to prior subject recognition confidence values calculated using prior images of the person to identify changes in the subject recognition confidence values associated with either individual;

analyzing the identified changes in the subject recognition confidence values to determine whether a new facial recognition model for the person should be generated based on the identified changes in the facial appearance attributes of the person;

generating the new facial recognition model from current or subsequent images of the person if it is so determined by the step of analyzing the identified changes; and

using the new facial recognition model for subsequent person recognition tasks.

26. The method of claim 25 wherein pair-wise facial classifiers for different pairs of individuals in the household are based on different facial features.