Fast Landmark Detection Using Regression Methods

• US 20080187213A1
• Filed: 02/06/2007
• Published: 08/07/2008
• Est. Priority Date: 02/06/2007
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Cha, Viola, Paul, Oh, Sang Min
• Application Number:
  US11/671,760
• Publication Number:
  US 20080187213A1
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A landmark detection technique that can quickly detect both objects of interest and landmarks within the objects in an image using regression methods. The present fast landmark detection scheme reuses existing feature values used for object detection (e.g., face detection) to find the landmarks in an object (e.g., the eyes and mouth of the face). Hence, the technique provides landmark detection functionality at almost no cost.

Claims

1. A computer-implemented process for detecting landmarks and their positions in an object detected in an input image, comprising using a computer to perform the following process actions:

creating a database comprising a plurality of training feature characterizations, each of which characterizes features of an object in an image;

for each object in the database computing landmark features that define the object and defining the ground truth locations of these landmark features;

training a regressor using a regression learning procedure to learn a relationship that defines the location of landmarks in any detected object given said feature characterizations;

inputting a portion of an input image into an object detector and outputting the location of any object found in the portion of the input image and feature characterizations used to find any object found;

inputting the feature characterizations and the location of any object found in the portion of the input image to into the trained regressor to output the landmark locations.

2. The computer-implemented process of claim 1 wherein the regression learning procedure comprises employing linear regression.

3. The computer-implemented process of claim 1 wherein the regression learning procedure comprises employing a neural network.

4. The computer-implemented process of claim 1 wherein the regression learning procedure comprises employing additive polynomial modeling.

5. The computer-implemented process of claim 1 wherein the regression learning procedure comprises employing a regression tree.

6. The computer-implemented process of claim 1 wherein the feature characterizations are raw feature values output from the object detector.

7. The computer-implemented process of claim 1 wherein the feature characterizations are transformed feature values output from the object detector.

8. The computer-implemented process of claim 1 wherein the feature characterizations are raw pixel values output from the object detector.

9. The computer-implemented process of claim 1 wherein the object detector is a face detector and wherein the landmarks are the eyes, nose and mouth of any face detected by the face detector.

10. A computer-readable medium having computer-executable instructions for performing the process recited in claim 1.

11. A system for locating landmarks in an object detected by an object detector, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to,

input an object in an image detected by an object detector that employs features to detect the object, and the features used to detect the object, into a regressor trained to find the locations of landmarks in the object; and

output the locations of the landmarks in the object.

12. The system of claim 11 wherein the object detector is a face detector.

13. The system of claim 11 wherein the regressor is trained using a regression procedure.

14. The system of claim 13 wherein the regression procedure comprises at least one of:

mean prediction;

linear regression;

a neural network;

additive polynomial modeling;

a regression tree; and

a boosted regression tree.

15. The system of claim 11 wherein the output landmarks are used for one of:

face pose estimation;

virtual makeup application; and

teleconferencing.

16. A computer-implemented process for training a regressor to detect landmarks and their positions in a face detected in an input image and using the trained regressor, comprising using a computer to perform the following process actions:

creating a training database of faces;

for each face in the training database computing landmarks that define the face and marking the ground truth locations of these landmarks; and

training a regressor using a regression learning procedure and the training database with the defined ground truth locations and features used by the face detector to learn a matrix that defines the landmarks in any detected face.

17. The computer-implemented process of claim 16 further comprising using the trained regressor to define the location of landmarks in a face detected by the face detector, comprising:

inputting a portion of an input image into an face detector and outputting the location of any face found in the portion of the input image and features used to find any face found;

inputting the features and the location of any face found in the portion of the input image to into the trained regressor to output the landmark locations.

18. The computer-implemented process of claim 17 wherein the regression procedure comprises employing a neural network and wherein the features are pixel values.

19. The computer-implemented process of claim 17 wherein the regression procedure comprises employing a regression tree and wherein the features are raw or transformed features.

20. The computer-implemented process of claim 19 wherein the regression tree is a boosted regression tree.