Method for outlining and aligning a face in face processing of an image

• US 7,720,284 B2
• Filed: 09/08/2006
• Issued: 05/18/2010
• Est. Priority Date: 09/08/2006
• Status: Active Grant

• Current Assignee: Omron Corporation, Tsinghua University
• Original Assignee: Omron Corporation, Tsinghua University
• Inventors: Zhang, Li, Tsukiji, Shuichiro, Lao, Shihong, Ai, Haizhou
• Primary Examiner(s): Couso; Jose L
• Application Number:
  US11/517,485
• Publication Number:
  US 20080063263A1
• Time in Patent Office: 1,348 Days
• Field of Search: 382/118, 382155-156, 382/159, 382/224, 382/227, 706/2, 706/26, 706/45, 706/52
• US Class Current: 382/173
• CPC Class Codes: G06V 10/7553 based on shape, e.g. active shape models [ASM]
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships

Abstract

Method and apparatus for face alignment by building a hierarchical classifier network. The hierarchical classifier network connects the tasks of face detection and face alignment into a smooth coarse-to-fine procedure. Texture classifiers are trained to recognize feature texture at different scales for different resolution layers. A multi-layer structure is employed to organize the texture classifiers, which begins with one classifier at the first layer and gradually refines the localization of feature points using additional texture classifiers in subsequent layers.

Claims

1. A method of training texture classifiers to detect an object in an image, the method being preformed by a computing system and comprising:

specifying a plurality of resolution levels for an object to be detected;

specifying one or more feature points for each resolution level;

collecting positive image samples from a window centered on each feature point at each resolution level from a plurality of object images;

collecting negative image samples from windows not centered on feature points at each resolution level from a plurality of object images; and

training a texture classifier for each feature point of each resolution level using a boosting learning algorithm.

2. The method of claim 1, further comprising using a LUT weak feature as a weak classifier in the boosting learning algorithm.

3. The method of claim 2, wherein the boosting learning algorithm is RealAdaBoost.

4. The method of claim 1, wherein the boosting learning algorithm is RealAdaBoost.

5. The method of claim 1, wherein the negative image samples are randomly collected from within a region of interest centered on each feature point.

6. The method of claim 1, wherein the number of feature points is progressively larger for each successive resolution level.

7. The method of claim 6, wherein the image samples taken are progressively smaller for each successive resolution level.

8. The method of claim 1, wherein only one feature point is specified for the first resolution level.

9. The method of claim 8, wherein the feature point specified for the first resolution level is related to at least one of an eye, nose, and mouth.

10. The method of claim 8, wherein a texture classifier trained for the one feature point of the first resolution level is trained to detect a face.

11. The method of claim 10, wherein texture classifiers trained for feature points of resolution levels subsequent to the first resolution level are trained to detect facial features.

12. The method of claim 1, further comprising training a cascade of texture classifiers for each feature point.

13. The method of claim 1, wherein a texture classifier for each feature point of each resolution level is trained using a boosting learning algorithm to boost weak classifiers constructed based on rectangular features that correspond to the positive image samples and the negative image samples.

14. The method of claim 1, wherein the feature points for a last resolution level are selected according to the final alignment desired, and wherein the feature points for the other resolution levels are constructed by merging the feature points in a subsequent resolution level.

15. The method of claim 14, wherein two feature points of a subsequent layer are merged to create a new feature point in a previous level only if they are close enough so that the texture classifier of the new feature point covers most of the area of the texture classifiers of the two feature points before mergence.

16. A method for detecting and aligning an object in an image, the method being performed by a computing system and comprising:

locating a first feature point corresponding to the object in an image using a texture classifier at a first resolution level;

estimating shape and pose parameters for the object at the first resolution level;

locating a set of second feature points corresponding to features of the object using texture classifiers at a second resolution level, that is finer than the first resolution level;

wherein search areas for the texture classifiers at the second resolution level are constrained according to the shape and pose parameters for the object at the first resolution level.

17. The method of claim 16, wherein the size of the object is normalized before estimating the shape and pose parameters for the object at the first resolution level.

18. The method of claim 16, wherein the shape and pose parameters for the object at the first resolution level are estimated using a max-a-posterior inference.

19. The method of claim 16, wherein the texture classifier at the first resolution level is implemented in a nested cascade structure.

20. The method of claim 16, wherein the second set of feature points are located using a statistical inference based on the shape and pose parameters for the object at the first resolution level and the texture classifiers at the second resolution level.

21. The method of claim 20, wherein the statistical inference is a Bayesian inference.

22. The method of claim 20, wherein solving the statistical inference yields the shape and pose parameters for the second set of feature points.

23. The method of claim 16, wherein the object is a human face.

24. The method of claim 16, further comprising:

locating multiple sets of feature points corresponding to facial features using texture classifiers, wherein the texture classifiers for each subsequent set of feature points are finer than the texture classifiers for the previous set of feature points,

wherein search areas for the texture classifiers for each subsequent set of feature points are constrained according to the shape and pose parameters for the set of feature points for the previous set of feature points.

25. A digital computing apparatus for training texture classifiers, comprising:

an image input unit for inputting a plurality of images containing an object to be detected;

a memory storage unit for storing positive image samples and negative image samples collected from the plurality of images containing the object; and

a central processing unit for training a set of texture classifiers corresponding to a set of feature points for a plurality of resolution levels of the object using a boosting learning algorithm;

wherein the positive image samples are collected by the central processing unit by specifying a plurality of resolution levels for a face, specifying a set of one or more feature points for each resolution level, and collecting positive image samples from a window centered on each feature point at each resolution level from a plurality of face images,

wherein said negative image samples are collected by the central processing unit from a window positioned away from the center on each feature point at each resolution level from a plurality of face images.

26. The digital computing apparatus of claim 25, wherein a weak classifier of the boosted learning algorithm uses an LUT weak feature.

27. The digital computing apparatus of claim 26, wherein the boosting learning algorithm used is RealAdaBoost.

28. The digital computing apparatus of claim 25, wherein the boosting learning algorithm used is RealAdaBoost.

29. A digital computing apparatus for detecting and aligning an object in an image, comprising:

an image input unit for inputting an image containing an object to be detected and aligned;

a memory storage unit for storing texture classifiers; and

a central processing unit for locating a first feature point corresponding to the object using a texture classifier at a first resolution level, estimating shape and pose parameters for the object at the first resolution level, locating a second set of feature points corresponding to features of the object using texture classifiers at a second resolution level, that is finer than the first resolution level;

wherein search areas for the texture classifiers at the second resolution level are constrained according to the shape and pose parameters for the object at the first resolution level.

30. A program encoded on a computer readable medium for training texture classifiers, said program when executed by a processor causing the processor to execute the steps of:

specifying a plurality of resolution levels for an object to be detected;

specifying one or more feature points for each resolution level;

collecting positive image samples from a window centered on each feature point at each resolution level from a plurality of object images;

collecting negative image samples from windows not centered on feature points at each resolution level from a plurality of object images; and

training a texture classifier for each feature point of each resolution level using a boosting learning algorithm.

31. The program of claim 30, wherein a weak classifier of the boosted learning algorithm uses an LUT weak feature.

32. The program of claim 31, wherein the boosting learning algorithm used is RealAdaBoost.

33. The program of claim 30, wherein the boosting learning algorithm used is RealAdaBoost.

34. A program encoded on a computer readable medium for detecting and aligning an object in an image, said program when executed by a processor causing the processor to execute the steps of:

locating a first feature point corresponding to the object in an image using a texture classifier at a first resolution level;

estimating shape and pose parameters for the object at the first resolution level;

locating a second set of feature points corresponding to features of the object using texture classifiers at a second resolution level, that is finer than the first resolution level;

wherein search areas for the texture classifiers at the second resolution level are constrained according to the shape and pose parameters for the object at the first resolution level.