FACE RECOGNITION USING FACE TRACKER CLASSIFIER DATA

• US 20100141787A1
• Filed: 12/04/2009
• Published: 06/10/2010
• Est. Priority Date: 12/05/2008
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Ireland Limited (Adeia Inc.)
• Inventors: Bigioi, Petronel, Corcoran, Peter
• Granted Patent: US 8,411,912 B2
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method of determining face recognition profiles for a group persons includes determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of the group, and recording probability scores which are analyzed for each classifier, including determining a mean and variance for each classifier for the first person. The process is repeated for one or more other persons of the group. A sub-set of classifiers is determined which best differentiates between the first person and the one or more other persons. The sub-set of classifiers is stored in association with the first person as a recognition profile.

Claims

1. A method of in-camera face recognition training of a specific face within digital images acquired with a portable camera-enabled device, comprising:

using a lens, image sensor and processor of a portable camera-enabled device to acquire digital images;

generating in the device, capturing or otherwise obtaining in the device multiple different images that include a face of a specific person;

identifying groups of pixels that correspond to the face within the multiple different images;

selecting sets of classifiers as matching the faces identified in the multiple different images;

statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;

repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;

normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;

generating a face recognition profile for the specific person based on the normalized face classifiers of the average face of the specific person; and

programming the processor such that when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, then the processor initiates a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device based on the recognition of the face as belonging to the specific person.

2. The method of claim 1, wherein the different images comprise two or more different poses of the face.

3. The method of claim 2, wherein the different images comprises two or more different lighting conditions of the face.

4. The method of claim 1, wherein the different images comprises two or more different lighting conditions of the face.

5. The method of claim 1, wherein the statistically analyzing comprises determining mean and variance values for the sets of classifiers.

6. The method of claim 1, further comprising tracking said face within the plurality of the relatively low resolution images.

7. The method of claim 6, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

8. The method of claim 6, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

9. The method of claim 1, further comprising determining probabilities for classifiers based upon which the sets of classifiers are selected.

10. A method of determining face recognition profiles for each of a group of at least two persons comprising analyzing a set of images where more than one face image of each person is available, comprising:

(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;

(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;

(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;

(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;

(e) repeating steps (a) through (d) for one or more further persons of the group of persons;

(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and

(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

11. The method of claim 10, further comprising deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

12. The method of claim 10, further comprising storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

13. The method of claim 12, further comprising:

determining and storing base recognition profile data for the one or more further persons of the group of persons;

determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and

storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

14. The method of claim 13, further comprising deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

15. The method of claim 14, further comprising storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.

16. A portable camera-enabled device capable of in-camera face recognition and face recognition training of a specific face within acquired digital images, comprising:

a lens and an image sensor to acquire digital images;

a processor programmed by processor-readable code embedded within one or more digital storage media, wherein the processor-readable code comprises;

a face detector component to program the processor to identify a groups of pixels that correspond to a face within the digital images;

a face recognition training component to program the processor to generate a face recognition profile for a specific person determined by programming the processor to perform the following;

selecting sets of classifiers as matching the faces identified in the multiple different images;

statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;

repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;

normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;

a face recognition component to determine when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, and to initiate a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device when the face belonging to the specific person is thereby recognized.

17. The device of claim 16, wherein the multiple different images comprise two or more different poses of the face.

18. The device of claim 17, wherein the multiple different images comprises two or more different lighting conditions of the face.

19. The device of claim 16, wherein the multiple different images comprises two or more different lighting conditions of the face.

20. The device of claim 16, wherein the multiple statistically analyzing comprises determining mean and variance values for the sets of classifiers.

21. The device of claim 16, wherein the processor-readable code further comprises a face tracker component to program the processor to track said face within a plurality of images.

22. The device of claim 21, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

23. The device of claim 21, wherein the face tracking and recognition components are configured to operate simultaneously.

24. The device of claim 16, wherein the processor-readable code further for programming the processor to determine probabilities for classifiers based upon which the sets of classifiers are selected.

25. A portable camera-enabled device capable of in-camera determination of face recognition profiles for each of a group of at least two persons when more than one face image of each person is available, comprising:

a lens;

an image sensor;

a processor; and

a processor readable medium having code embedded therein for programming the processor to perform a face recognition method that comprises;

(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;

(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;

(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;

(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;

(e) repeating steps (a) through (d) for one or more further persons of the group of persons;

(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and

(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

26. The device of claim 25, wherein the method further comprises deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

27. The device of claim 25, wherein the method further comprises storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

28. The device of claim 27, wherein the method further comprises:

determining and storing base recognition profile data for the one or more further persons of the group of persons;

determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and

storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

29. The device of claim 28, wherein the method further comprises deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

30. The device of claim 29, wherein the method further comprises storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.

31. One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of in-camera face recognition training of a specific face within digital images acquired with a portable camera-enabled device, wherein the method comprises:

using a processor of a portable camera-enabled device;

generating in the device, capturing or otherwise obtaining in the device multiple different images that include a face of a specific person;

identifying groups of pixels that correspond to the face within the multiple different images;

selecting sets of classifiers as matching the faces identified in the multiple different images;

statistically analyzing the sets of classifiers to generate a reference classifier profile of the face associated with the specific person;

repeating the generating, identifying, selecting and statistically analyzing one or more times to generate one or more further reference classifier profiles of the face associated with the specific person;

normalizing the reference classifier profiles to determine normalized face classifiers of an average face associated with the specific person;

generating a face recognition profile for the specific person based on the normalized face classifiers of the average face of the specific person; and

programming the processor such that when a probability exceeds a predetermined threshold that an identified face corresponds to the specific person associated with the face recognition profile, then the processor initiates a workflow, image processing or other pre- or post-image acquisition action on the portable camera-enabled device based on the recognition of the face as belonging to the specific person.

32. The one or more processor-readable storage media of claim 31, wherein the different images comprise two or more different poses of the face.

33. The one or more processor-readable storage media of claim 32, wherein the different images comprises two or more different lighting conditions of the face.

34. The one or more processor-readable storage media of claim 31, wherein the different images comprises two or more different lighting conditions of the face.

35. The one or more processor-readable storage media of claim 31, wherein the statistically analyzing comprises determining mean and variance values for the sets of classifiers.

36. The one or more processor-readable storage media of claim 31, wherein the method further comprises tracking said face within the plurality of the relatively low resolution images.

37. The one or more processor-readable storage media of claim 36, wherein the selecting comprises aligning face region subwindows and determining probabilities for classifiers based upon which the sets of classifiers are selected.

38. The one or more processor-readable storage media of claim 36, wherein the tracking is performed in parallel with determining whether the identified face corresponds to the specific person.

39. The one or more processor-readable storage media of claim 31, wherein the method further comprises determining probabilities for classifiers based upon which the sets of classifiers are selected.

40. One or more processor-readable storage media having code embedded therein for programming a processor to perform a method of in-camera determination of face recognition profiles for each of a group of at least two persons when more than one face image of each person is available, wherein the method comprises:

(a) determining with a multi-classifier face detector that a face region within a digital image has above a threshold probability of corresponding to a first person of a group of persons;

(b) recording a probability score for each classifier stage of the multi-classifier face detector, along with an overall probability score, including associating the probability scores with the first person;

(c) repeating (a) and (b) for one or more further images each containing a face region that has above the threshold probability of corresponding to the first person;

(d) analyzing the probability scores for each classifier and the overall probability score, including determining a mean and variance for each classifier for the first person;

(e) repeating steps (a) through (d) for one or more further persons of the group of persons;

(f) determining a sub-set of classifiers which best differentiates between the first person and the one or more further persons of the group; and

(g) storing the sub-set of classifiers including associating said sub-set of classifiers with the first person as a recognition profile.

41. The one or more processor-readable storage media of claim 40, wherein the method further comprises deriving the one or more further images each containing a face region of the first person from a set of preview stream images acquired within a same digital imaging device as the digital image.

42. The one or more processor-readable storage media of claim 40, wherein the method further comprises storing the mean and variance for each classifier and the overall probability score, including associating these with the first person as a base recognition profile.

43. The one or more processor-readable storage media of claim 42, wherein the method further comprises:

determining and storing base recognition profile data for the one or more further persons of the group of persons;

determining a new sub-set of classifiers which best differentiates the first person from the one or more further persons, and

storing the new sub-set of classifiers as a new recognition profile for the first person, including associating the new sub-set of classifiers with the first person.

44. The one or more processor-readable storage media of claim 43, wherein the method further comprises deriving multiple face images of the first person from a set of preview stream images, and determining the base recognition profile on these multiple face images from the set of preview stream images.

45. The one or more processor-readable storage media of claim 44, wherein the method further comprises storing the base recognition profile of the first person and the base recognition profile data of the one or more further persons of the group of persons within a same digital imaging device used to capture the digital image and set of preview stream images.