METHOD OF LOCATING FEATURES OF AN OBJECT

• US 20090116749A1
• Filed: 04/05/2007
• Published: 05/07/2009
• Est. Priority Date: 04/08/2006
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation), University of Manchester
• Original Assignee: Toyota Motor Europe SA (Toyota Motor Corporation), University of Manchester
• Inventors: Cootes, Timothy Francis, Cristinacce, David
• Granted Patent: US 8,594,430 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A method of locating features of an object, of a class of objects, of a class of objects, within a target image. The method comprises initialising a set of feature points within the target image, each feature point corresponding to a predetermined feature for objects of the class of objects; deriving a set of template detectors, from the set of feature points, using a statistical model of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects; comparing the set of template detectors with the target image; and updating the set of feature points within the target image in response to the result of the comparison.

Claims

1. A method of locating features of an object, of a class of objects, within a target image, the method comprising:

initialising a set of feature points within the target image, each feature point corresponding to a predetermined feature for objects of the class of objects;

deriving a set of template detectors, from the set of feature points, using a statistical model of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects;

comparing the set of template detectors with the target image; and

updating the set of feature points within the target image in response to the result of the comparison.

2. The method of claim 1, wherein the set of template detectors derived using said statistical model collectively represent only part of an object of the class of objects.

3. The method of claim 1, wherein the set of template detectors derived using said statistical model collectively represent all of an object of the class of objects.

4. The method of claim 1, further comprising:

repeating the deriving the set of template detectors from the updated set of feature points, the comparing the set of template detectors with the target image and the updating the set of feature points.

5. The method of claim 1, further comprising after said updating the set of feature points:

determining whether the average difference between the location of each updated feature point and its preceding location is below a predetermined threshold; and

outputting the set of feature points as a final set of feature points if the average difference is below the threshold.

6. The method of claim 1, further comprising:

generating the statistical model from a set of training images, each training image containing an object of the class of objects.

7. The method of claim 6, wherein said generating the statistical model comprises for each training image:

identifying a set of feature points within the training image corresponding to a predetermined feature of the object within the training image; and

generating a set of templates, each template comprising an area of image within the training image located about an identified feature point.

8. The method of claim 7, wherein said identifying a set of feature points comprises a human operator identifying the set of feature points.

9. The method of claim 7, wherein said identifying a set of feature points comprises automatically identifying the set of feature points.

10. The method of claim 7, further comprising:

generating from the sets of feature points for each training image in the set of training images a shape model representing the spatial distribution of feature points for the set of training images.

11. The method of claim 10, wherein said deriving a set of template detectors comprises:

generating a set of templates for the target image, each template comprising an area of image located about a feature point within the target image;

using the shape model to identify a subset of training images within the set of training images which have identified feature points closest to the set of feature points within the target image;

correlating the set of templates for the target image with the set of templates for each training image within the subset of training images;

selecting from the subset of training images the training image having a set of templates most correlated to the set of templates for the target image; and

setting the set of templates for the selected training image as the set of template detectors.

12. The method of claim 7, further comprising:

generating from the sets of feature points for each training image in the set of training images a shape model representing the spatial distribution of feature points for the set of training images;

generating from the sets of templates for each training image in the set of training images a texture model representing the distribution of texture for the sets of templates within the set of training images; and

combining the shape model and the texture model to form a combined shape and texture model.

13. The method of claim 12, wherein said deriving a set of template detectors comprises:

generating a set of templates for the target image, each template comprising an area of image located about a feature point within the target image;

using the combined shape and texture model to generate a modelled set of templates from the set of feature points and the set of templates for the target image; and

setting the generated set of templates as the set of template detectors.

14. The method of claim 1, wherein said comparing the set of template detectors with the target image comprises:

comparing each template detector with a respective area of the target image.

15. The method of claim 1, wherein said comparing the set of template detectors with the target image comprises:

correlating each template detector separately with the target image at a series of positions about the corresponding feature point within the target image to generate a response image, such that a set of response images are formed; and

using a non-linear optimisation to locate the positions of best correlation for each feature point, such that an updated set of feature points is generated, said updated set of feature points forming a valid set of feature points for objects of the class of objects.

16. The method of claim 1, wherein said initialising a set of feature points comprises:

using a global object detector to identify an area of image within the target image containing the object; and

initialising the set of feature points to mean feature point positions within the area of image identified by the global object detector.

17. The method of claim 16, wherein said initialising a set of feature points further comprises:

using a set of feature detectors to identify areas of image within the area of the target image identified using the global object detector containing each feature point; and

initialising each feature point to a mean feature position within the area of image identified by the feature region detector.

18. The method of claim 1, comprising:

locating features of an object within a video sequence comprising a series of target images;

initialising a set of feature points within the first target image; and

initialising a set of feature points for subsequent target images using the updated feature points from the preceding target image.

19. The method of claim 1, wherein the class of objects comprises human faces.

20. A method of modelling the visual appearance of an object, of a class of objects, within a target image, the method comprising:

initialising a set of feature points within the target image, each feature point corresponding to a predetermined feature for objects of the class of objects;

deriving a set of template detectors, from the set of feature points, using a statistical model of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects;

comparing the set of template detectors with the target image; and

updating the set of feature points within the target image in response to the result of the comparison.

21. A method of modelling the visual appearance of a human face, within a target image, the method comprising:

initialising a set of feature points within the target image, each feature point corresponding to a predetermined facial feature;

deriving a set of template detectors, from the set of feature points, using a statistical model of human faces, each template detector comprising an area of image located about the location of a facial feature;

comparing the set of template detectors with the target image; and

updating the set of feature points within the target image in response to the result of the comparison.

22. (canceled)

23. (canceled)

24. An apparatus for locating features of an object, of a class of objects, within a target image, the apparatus comprising:

means for initialising a set of feature points within the target image, each feature point corresponding to a predetermined feature for objects of the class of objects;

means for deriving a set of template detectors, from the set of feature points, using a statistical model of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects;

means for comparing the set of template detectors with the target image; and

means for updating the set of feature points within the target image in response to the result of the comparison.

25-28. -28. (canceled)

29. A computer program product comprising a computer useable medium having a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

initialise a set of feature points within the target image, each feature point corresponding to a predetermined feature for objects of the class of objects;

derive a set of template detectors, from the set of feature points, using a statistical model of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects;

compare the set of template detectors with the target image; and

update the set of feature points within the target image in response to the result of the comparison.

30. A computer apparatus for locating features of an object, of a class of objects, within a target image the apparatus comprising:

a program memory storing processor readable instructions; and

a processor configured to read and execute instructions stored in said program memory to initialise a set of feature points within the target image, derive a set of template detectors, from the set of feature points, using a statistical model of the class of objects, compare the set of template detectors with the target image, and update the set of feature points within the target image in response to the result of the comparison, each feature point corresponding to a predetermined feature for objects of the class of objects, each template detector comprising an area of image located about the location of a feature point for an object of the class of objects.