Cascaded face model

• US 8,144,976 B1
• Filed: 12/06/2010
• Issued: 03/27/2012
• Est. Priority Date: 12/06/2010
• Status: Expired due to Fees

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Shiell, Derek, Xiao, Jing
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US12/961,350
• Time in Patent Office: 477 Days
• Field of Search: 382/118, 382/154, 382/159, 382/276, 382/285, 345/6, 345/419, 345/420, 345/422, 345/621, 345/630, 345/664, 348/50, 348/267, 700/98, 700/163, 700/182, 704/250, 704/255, 704/266, 715/225, 715/771, 715/970
• US Class Current: 382/159
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

An Active Appearance Model AAM is trained using expanded library having examples of true outlier images. The AAM creates a first statistical fitting pair (a model image of the class of object and corresponding statistical model fitting) using characteristic features drawn only from the expanded library. All images within the expanded library that the first statistical fitting pair cannot align are collected into a smaller, second library of true outlier cases. A second statistical fitting pair is created using characteristic features drawn only from the second library, and samples not aligned by the second statistical fitting pair are collected into a still smaller, third library. This process is repeated until a desired percentage of all the images within the initial, expanded library have been aligned. In operation, the AAM applies each of its created statistical fitting pairs, in turn, until it has successfully aligned a submitted test image, or until a stop criterion has been reached.

Claims

1. An Active Appearance Model (AAM) machine, comprising:

(A) a learn module providing a plurality of statistical fitting pairs, each statistical fitting pair consisting of a model image of a class of object and a corresponding statistical model fitting function defining shape and appearance features of said class of object, wherein;

(i) a first statistical fitting pair within said plurality of statistical fitting pairs includes a first model image and corresponding first statistical model fitting function defined from feature information drawn from a first image library of true image samples of said class of object, said first statistical fitting pair being effective for fitting said first model image to less than 100% of the images within said first image library, the images of said first image library to which said first model image cannot be fit defining a second image library of true image samples;

(ii) a second statistical fitting pair within said plurality of statistical fitting pairs includes a second model image and a corresponding second statistical model fitting function defined from feature information drawn from only said second image library of true image samples;

(B) an AAM input for receiving an input image;

(C) an align module for conditionally and separately accessing each statistical fitting pair within said plurality of statistical fitting pairs, and for each accessed statistical fitting pair, applying a fitting sequence wherein the accessed statistical fitting pair'"'"'s corresponding statistical model fitting function is optimized to determine a best fit of its corresponding model image to said input image through iterative applications of said corresponding statistical model fitting function to produce an aligned image if its corresponding model image can successfully be fitted to said input image, and if its corresponding model image cannot be successfully fitted to said input image, then sequentially accessing each remaining statistical fitting pair, in turn, until a statistical fitting pair is accessed wherein the model image of the currently accessed statistical fitting pair is successfully fitted to said input image or until all statistical fitting pairs within said plurality of statistical fitting pairs have been accessed; and

(D) an AAM output for outputting said aligned image.

2. The Active Appearance Model machine of claim 1, wherein said align module ceases to access said statistical fitting pairs when said align image is found or when all statistical fitting pairs have been accessed.

3. The Active Appearance Model machine of claim 1, wherein said align module accesses said statistical fitting pairs in a fixed sequenced determined by the size of the library of true image samples from which each statistical fitting pair is defined.

4. The Active Appearance Model machine of claim 3, wherein said first statistical fitting pair is first in said fixed sequence, and it is defined from the largest of the libraries of true images, and the last statistical fitting pair in said fixed sequence is defined from the smallest of the libraries of true images.

5. The Active Appearance Model machine of claim 1, wherein a third statistical fitting pair within said plurality of statistical fitting pairs is defined from information drawn from only images from said second image library of true image samples that were not fitted by said second statistical fitting pair.

6. The Active Appearance Model machine of claim 1, wherein:

said second statistical fitting pair is effective for fitting said second model image to less than 100% of the images within said second image library, the images of said second image library to which said second model image cannot be fitted defining a third image library of true image samples, and

a third statistical fitting pair within said plurality of statistical fitting pairs includes a third model image and a corresponding third statistical model fitting function defined from feature information drawn only from said third image library of true image samples.

7. The Active Appearance Model machine of claim 6, wherein said first, second, and third statistical fitting pairs are first, second, and third in said fixed sequence, and each subsequent statistical fitting pair within said fixed sequence is defined from feature information drawn from only a corresponding library of true image samples constructed from unfitted images from its immediately previous statistical fitting pair within said fixed sequence.

8. The Active Appearance Model machine of claim 6, wherein each statistical fitting pair within said plurality of statistical fitting pairs is defined for the same, said class of object.

9. The Active Appearance Model machine of claim 1, wherein each statistical fitting pair constitutes a separate sub-learn-module and said align module is comprised of a plurality of align sub-modules maintaining a one-to-one relationship with a corresponding sub-learn-module, and the output of each of said align sub-module is selectively coupled to said AAM output so that only the align sub-module corresponding to the currently accessed sub-learn-module is coupled to said AAM output at any given time.

10. The Active Appearance Model machine of claim 9, wherein all the align sub-modules are coupled to said AAM output via a common multiplexer.

11. A method of implementing an Active Appearance Model system, comprising the following steps:

(1) providing;

(A) a learn module having a plurality of statistical fitting pairs individually accessible in a predefined sequence, each statistical fitting pair consisting of a model image of a class of object and a corresponding statistical model fitting function defining shape and appearance features of said class of object, wherein;

(i) a first statistical fitting pair within said plurality of statistical fitting pairs includes a first model image and corresponding first statistical model fitting function defined from feature information drawn from only a first image library of true image samples of said class of object, said first statistical fitting pair being effective for fitting said first model image to less than 100% of the images within said first image library, the images of said first image library to which said first model image cannot be fit defining a second image library of true image samples;

(ii) a second statistical fitting pair within said plurality of statistical fitting pairs includes a second model image and a corresponding second statistical model fitting function defined from feature information drawn from only said second image library of true image samples;

(B) an align module;

(2) accessing a new input test image;

(3) said align module accessing the next individual statistical fitting pair in said predefined sequence, and applying a fitting sequence wherein the accessed statistical fitting pair'"'"'s corresponding statistical model fitting function is optimized to determine a best fit of its corresponding model image to said input test image through iterative applications of said corresponding statistical model;

(4) IF said aligned module successfully fitted said corresponding model image to said input test image, THEN outputting said fitted image as an aligned image;

(5) IF not all of said plurality of statistical fitting pairs have been accessed, THEN returning to step (3).

12. The method of claim 11, further having the following step immediately following step (5):

(6) Returning to step (2).

13. The method of claim 11, wherein in step (1), said predefined sequence is a fixed predefined sequence.

14. The method of claim 11, wherein:

step (1) further includes providing a counter;

each of said statistical fitting pair in said plurality of statistical fitting pairs is identifiable by a unique ID-Count-Value, and the last statistical fitting pair in said predefined sequence has the highest value ID-Count-Value;

step (2) further includes resetting said counter;

in step (3), said align module accesses the statistical fitting pair whose ID-Count-Value corresponds to the current value of said counter;

step (5) includes incrementing said counter, and if the incremented counter is not greater than said highest valued ID-Count-Value, then it is determined that not all of said plurality of statistical fitting pairs have been accessed.

15. The method of claim 11, wherein step (1) includes providing:

an AAM input for receiving an input test image; and

an AAM output for outputting said aligned image.

16. A method of training an Active Appearance Model machine having a learn module, comprising the following steps:

(A) providing a first image library of true image samples of a class of object, each image within said first image library having predefined characteristic features of said class of object identified and labeled, designating said first image library as a currently active image library;

(B) submitting the currently active image library to said learn module;

(C) having said learn module create a new statistical fitting pair consisting of a new model image and a corresponding new statistical model fitting function defined from feature information drawn only from the currently active image library;

(D) IF any images within the currently active image library cannot be aligned using the new statistical fitting pair;

THEN gathering the unfitting images into a new image sub-library of true images samples of said class of object;

(E) IF the number of images within said new image sub-library is not greater than a predefined percentage of the number of image in said first image library, THEN designating said new image sub-library as the currently active image library and returning to step (B).

17. The method of claim 16, wherein in step (C) the new statistical fitting pair is associated with the currently active image library.

18. The method of claim 16, wherein in step (C), each newly created statistical fitting pair is assigned a designated position within an accessibility sequence, the created statistical fitting pairs being accessible according to said accessibility sequence.

19. The method of claim 16, wherein the sequence of steps from step (B) to step (E) defines an alignment cycle, and step (E) further comprising:

IF the number of images within said new image sub-library is not greater than a predefined percentage of the number of image in said first image library AND the number of images within said new image sub-library is not the same as in an immediately previous alignment cycle, THEN designating said new image sub-library as the currently active image library and returning to step (B), ELSE STOP execution of said method of training an Active Appearance Model.