Face recognition using face tracker classifier data

• US 8,687,078 B2
• Filed: 12/04/2009
• Issued: 04/01/2014
• Est. Priority Date: 12/05/2008
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Corcoran, Peter
• Primary Examiner(s): MOE, AUNG SOE
• Application Number:
  US12/631,711
• Publication Number:
  US 20100141786A1
• Time in Patent Office: 1,579 Days
• Field of Search: 382224-228, 382/118, 348/222.1
• US Class Current: 348/222.1
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A face recognition technique includes using a multi-classifier face detector to determine above a threshold probability that region of an image includes a face. Further probability values are determined for a set of classifiers for the region to provide a recognition profile. Face detection and recognition probabilities are determined for at least one classifier of the set. The recognition profile is compared against a predetermined recognition profile to determine a degree of match.

Claims

1. A method of in-camera recognition of a specific face within a digital image as part of an acquisition process, comprising:

using a lens, image sensor and processor of a portable digital image acquisition device to acquire digital images;

generating in the device, capturing or otherwise obtaining in the device a sequence of relatively low resolution images including a face;

identifying groups of pixels that correspond to the face within a plurality of the relatively low resolution images;

tracking said face within the plurality of the relatively low resolution images;

determining multiple real-time probabilities that the face corresponds to a specific person within the plurality of the relatively low resolution images;

averaging the multiple real-time probabilities to obtain a cumulative probability that the face belongs to the specific person; and

when the cumulative probability exceeds a predetermined threshold, initiating a workflow, image processing or other pre-or post-image acquisition action on the portable digital image acquisition device based on the recognition of the face as belonging to the specific person.

2. The method of claim 1, wherein the initiating comprises displaying a name or other identifier of the specific person recognized in association with the face on a display of the portable image acquisition device.

3. The method of claim 1, further comprising repeating the process for multiple different persons, and wherein when the cumulative probability of a particular face belonging to a second specific person is below said predetermined threshold by less than a predetermined amount but exceeds that for any other specific person, then initiating a workflow, image processing or other pre-or post-image acquisition action on the portable digital image acquisition device based on the recognition of the face as belonging to the second specific person.

4. The method of claim 1, further comprising repeating the process for multiple different persons, and wherein when the cumulative probability of a particular face belonging to any of the specific person or the multiple different persons is below the same or a different threshold, then identifying the face as unknown.

5. The method of claim 1, further comprising repeating the process for multiple different persons, and wherein when the cumulative probabilities of a particular face belonging to two or more of the specific person and the multiple different persons is above the threshold, then identifying the face as being associated jointly with the two or more persons.

6. The method of claim 1, further comprising training a set of face recognition classifiers associated with the specific person, and wherein the determining of the real-time probabilities comprises using said face recognition classifiers.

7. The method of claim 6, wherein the face recognition classifiers comprise both census-type and Haar-type classifiers.

8. The method of claim 6, wherein the face recognition classifiers comprise classifiers also used in the tracking or the identifying, or both.

9. The method of claim 1, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

10. A method of recognizing a face within an image, comprising:

using a lens, an image sensor and a processor of a portable digital image acquisition device to acquire a digital image;

determining a first plurality of probability values by applying a first set of classifiers to data of the digital image;

determining a first cumulative probability value by summing the first plurality of probability values;

determining whether the first cumulative probability value exceeds a first threshold;

in response to determining that the first cumulative probability value exceeds the first threshold, determining that the at least one facial region is present within the digital image; and

determining whether the at least one facial region matches a first recognition profile of a first person by;

selecting a second set of classifiers based on the first recognition profile;

determining a second plurality of probability values by applying the second set of classifiers to the at least one facial region;

determining a second cumulative probability value by summing the second plurality of probability values; and

determining whether the second cumulative probability value exceeds a second threshold;

wherein the second set of classifiers is specific to the first recognition profile of the first person.

11. The method of claim 10, wherein the second set of classifiers is a subset of the first set of classifiers.

12. The method of claim 11, further comprising:

determining whether the at least one facial region matches a second recognition profile of a second person by;

selecting a third set of classifiers based on the second recognition profile;

determining a third plurality of probability values by applying the third set of classifiers to the at least one facial region;

determining a third cumulative probability value by summing the third plurality of probability values; and

determining whether the third cumulative probability value exceeds a third threshold;

wherein the third set of classifiers is specific to the second recognition profile of the second person.

13. The method of claim 12, wherein the method further comprises comparing the second recognition profile with the first recognition profile to determine a degree of match.

14. The method of claims 13, further comprising selecting one of the first and second recognition profiles as providing a better degree of match.

15. The method of claim 13, wherein at least one of the first set of classifiers differs from at least one classifier of the second set of classifiers.

16. A digital image acquisition device capable of real-time in-camera recognition of a specific face within a digital image as part of an acquisition process, comprising:

a lens and an image sensor to acquire digital images including sequences of relatively low resolution images;

a processor programmed by processor-readable code embedded within one or more digital storage media, wherein the processor-readable code comprises;

a face detector component to program the processor to identify a groups of pixels that correspond to the face within one or more of the relatively low resolution images;

a face tracker component to program the processor to track said face within a plurality of the relatively low resolution images; and

a face recognition component to program the processor to determine multiple real-time probabilities that the face corresponds to a specific person within the plurality of the low resolution images, and to average the multiple real-time probabilities to obtain a cumulative probability that the face belongs to the specific person; and

wherein when the processor determines that the cumulative probability exceeds a predetermined threshold, said processor is further programmed to initiate a workflow, image processing or other pre-or post-image acquisition action, or combinations thereof, on the portable digital image acquisition device based on the recognition of the face as belonging to a specific person.

17. The device of claim 16, further comprising a display to indicate thereon a name or other identifier of the specific person recognized in association with the face.

18. The device of claim 16, wherein the processor-readable code programs the processor to repeat the process for multiple different persons, and when the cumulative probability of a particular face belonging to a second specific person is below said predetermined threshold by less than a predetermined amount but exceeds that for any other specific person, then to initiate a workflow, image processing or other pre-or post-image acquisition action on the portable digital image acquisition device based on the recognition of the face as belonging to the second specific person.

19. The device of claim 16, wherein the processor-readable code programs the processor to repeat the process for multiple different persons, and when the cumulative probability of a particular face belonging to any of the specific person or the multiple different persons is below the same or a different threshold, then to identify the face as unknown.

20. The device of claim 16, wherein the processor-readable code programs the processor to repeat the process for multiple different persons, and when the cumulative probabilities of a particular face belonging to two or more of the specific person and the multiple different persons is above the threshold, then identifying the face as being associated jointly with the two or more persons.

21. The device of claim 16, wherein the face recognition component comprises a training component to train a set of face recognition classifiers associated with the specific person, and to determine the real-time probabilities using said face recognition classifiers.

22. The device of claim 21, wherein the face recognition classifiers comprise both census-type and Haar-type classifiers.

23. The device of claim 21, wherein the face recognition classifiers comprise classifiers also used in the tracking or the identifying, or both.

24. The device of claim 16, wherein the face tracker and recognition components are configured to operate simultaneously.

25. A digital image acquisition device capable of in-camera recognition of a face within a digital image as part of an acquisition process, the device configured to:

use a lens, an image sensor and a processor of a portable digital image acquisition device to acquire a digital image;

determine a first plurality of probability values by applying a first set of classifiers to data of the digital image;

determine a first cumulative probability value by summing the first plurality of probability values;

determine whether the first cumulative probability value exceeds a first threshold;

in response to determining that the first cumulative probability value exceeds the first threshold, determine that the at least one facial region is present within the digital image; and

determine whether the at least one facial region matches a first recognition profile of a first person by;

selecting a second set of classifiers based on the first recognition profile;

determining a second plurality of probability values by applying the second set of classifiers to the at least one facial region;

determining a second cumulative probability value by summing the second plurality of probability values; and

determining whether the second cumulative probability value exceeds a second threshold;

wherein the second set of classifiers is specific to the first recognition profile of the first person.

26. The device of claim 25, wherein the second set of classifiers is a subset of the first set of classifiers.

27. The device of claim 26, further configured to:

determine whether the at least one facial region matches a second recognition profile of a second person by;

selecting a third set of classifiers based on the second recognition profile;

determining a third plurality of probability values by applying the third set of classifiers to the at least one facial region;

determining a third cumulative probability value by summing the third plurality of probability values; and

determining whether the third cumulative probability value exceeds a third threshold;

wherein the third set of classifiers is specific to the second recognition profile of the second person.

28. The device of claim 25, further configured to compare the second recognition profile with the first recognition profile to determine a degree of match.

29. The device of claim 28, wherein the device is further configured to select one of the first and second recognition profiles as providing a better degree of match.

30. The device of claim 28, wherein at least one of the first set of classifiers differs from at least one classifer of the second set of classifiers.

31. One or more non-transitory processor-readable storage media having code embedded therein for programming a processor to perform a method of in-camera recognition of a specific face within a digital image as part of an acquisition process, wherein the method comprises:

using a processor of a portable digital image acquisition device;

generating in the device, capturing or otherwise obtaining in the device a sequence of relatively low resolution images including a face;

identifying groups of pixels that correspond to the face within a plurality of the relatively low resolution images;

tracking said face within the plurality of the relatively low resolution images;

determining multiple real-time probabilities that the face corresponds to a specific person within the plurality of the relatively low resolution images;

averaging the multiple real-time probabilities to obtain a cumulative probability that the face belongs to the specific person; and

when the cumulative probability exceeds a predetermined threshold, initiating a workflow, image processing or other pre-or post-image acquisition action on the portable digital image acquisition device based on the recognition of the face as belonging to a specific person.

32. The one or more processor-readable storage media of claim 31, wherein the initiating comprises displaying a name or other identifier of the specific person recognized in association with the face on a display of the portable image acquisition device.

33. The one or more processor-readable storage media of claim 31, wherein the method further comprises repeating the process for multiple different persons, and wherein when the cumulative probability of a particular face belonging to a second specific person is below said predetermined threshold by less than a predetermined amount but is exceeds that for any other specific person, then initiating a workflow, image processing or other pre-or post-image acquisition action on the portable digital image acquisition device based on the recognition of the face as belonging to the second specific person.

34. The one or more processor-readable storage media of claim 31, wherein the method further comprises repeating the process for multiple different persons, and wherein when the cumulative probability of a particular face belonging to any of the specific person or the multiple different persons is below the same or a different threshold, then identifying the face as unknown.

35. The one or more processor-readable storage media of claim 31, wherein the method further comprises repeating the process for multiple different persons, and wherein when the cumulative probabilities of a particular face belonging to two or more of the specific person and the multiple different persons is above the threshold, then identifying the face as being associated jointly with the two or more persons.

36. The one or more processor-readable storage media of claim 31, wherein the method further comprises training a set of face recognition classifiers associated with the specific person, and wherein the determining of the real-time probabilities comprises using said face recognition classifiers.

37. The one or more processor-readable storage media of claim 36, wherein the face recognition classifiers comprise both census-type and Haar-type classifiers.

38. The one or more processor-readable storage media of claim 36, wherein the face recognition classifiers comprise classifiers also used in the tracking or the identifying, or both.

39. The one or more processor-readable storage media of claim 31, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

40. One or more non-transitory processor-readable storage media having one or more instructions embedded therein for programming a processor to perform a method of in-camera recognition of a face within a digital image as part of an acquisition process, the one or more instructions, when executed, cause performing:

using a lens, an image sensor and a processor of a portable digital image acquisition device to acquire a digital image;

determining a first plurality of probability values by applying a first set of classifiers to data of the digital image;

determining a first cumulative probability value by summing the first plurality of probability values;

determining whether the first cumulative probability value exceeds a first threshold;

in response to determining that the first cumulative probability value exceeds the first threshold, determining that the at least one facial region is present within the digital image; and

determining whether the at least one facial region matches a first recognition profile of a first person by;

selecting a second set of classifiers based on the first recognition profile;

determining a second plurality of probability values by applying the second set of classifiers to the at least one facial region;

determining a second cumulative probability value by summing the second plurality of probability values; and

determining whether the second cumulative probability value exceeds a second threshold;

wherein the second set of classifiers is specific to the first recognition profile of the first person.

41. The one or more processor-readable storage media of claim 40, wherein the second set of classifiers is a subset of the first set of classifiers.

42. The one or more processor-readable storage media of claim 41, comprising additional instructions which, when executed cause performing:

determining whether the at least one facial region matches a second recognition profile of a second person by;

selecting a third set of classifiers based on the second recognition profile;

determining a third plurality of probability values by applying the third set of classifiers to the at least one facial region;

determining a third cumulative probability value by summing the third plurality of probability values; and

determining whether the third cumulative probability value exceeds a third threshold;

wherein the third set of classifiers is specific to the second recognition profile of the second person.

43. The one or more processor-readable storage media of claim 40, comprising additional instructions which, when executed, cause performing:

comparing the second recognition profile with the first recognition profile to determine a degree of match.

44. The one or more processor-readable storage media of claim 43, comprising additional instructions which, when executed, cause performing:

selecting one of the first and second recognition profiles as providing a better degree of match.

45. The one or more processor-readable storage media of claim 43, wherein at least one of the first set of classifiers differs from at least one classifier of the second set of classifiers.