System and method for detecting and tracking facial features in images

• US 9,659,210 B1
• Filed: 01/13/2015
• Issued: 05/23/2017
• Est. Priority Date: 01/13/2014
• Status: Active Grant

• Current Assignee: Carnegie Mellon University
• Original Assignee: Carnegie Mellon University
• Inventors: De la Torre, Fernando, Xiong, Xuehan
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US14/596,148
• Time in Patent Office: 861 Days
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

The present invention relates to a system for detecting and tracking facial features in images and can be used in conjunction with a camera. Given a camera, the system will detect facial landmarks in images. The present invention includes software for real time, accurate facial feature detection and tracking in unconstrained images and videos. The present invention is better, more robust and faster than existing approaches and can be implemented very efficiently allowing real-time processing, even on low-power devices, such as mobile phones.

Claims

1. A system for detecting and tracking features in a series of images of an object, comprising:

a camera;

a microprocessor, in communication with said camera, said microprocessor running software for performing, for each image, the functions of;

(a) detecting the presence of said object in said image and placing said object in a bounding box;

(b) applying an average model consisting of a plurality of landmarks to the center of said bounding box;

(c) using multiple applications of a regression function to update said landmarks by mapping features of said object to said landmarks.

2. The system of claim 1 wherein said software further performs the functions of;

(d) applying a classifier to determine how closely said features map to said landmarks in said object;

(e) repeating steps (b)-(d) for the next image in said series of images if said classifier determines that the mapping of features to landmarks exceeds a predetermined, and

(f) repeating steps (a)-(d) for the next image in said series of images if said classifier determines that the mapping of features to landmarks does not exceed a predetermined threshold.

3. The system of claim 2 wherein said object is a face.

4. The system of claim 2 wherein said features of said object are expressed as a vector of histograms of gradients.

5. The system of claim 2 wherein said regression functions are based on a set of training data.

6. The system of claim 5 wherein said training data consisting of a plurality of images having landmarks manually applied thereto.