MULTI-POSE FACE FEATURE POINT DETECTION METHOD BASED ON CASCADE REGRESSION

• US 20190318158A1
• Filed: 11/30/2017
• Published: 10/17/2019
• Est. Priority Date: 12/14/2016
• Status: Active Grant

• Current Assignee: South China University of Technology
• Original Assignee: South China University of Technology
• Inventors: QIN, Huabiao, LIAO, Caiman
• Granted Patent: US 10,929,649 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A multi-pose face feature point detection method based on cascade regression comprises: extracting a pose index features and establishing corresponding optimal weak regressors; performing corresponding initialization according to different face pose orientations; using an SIFT feature of a face image as an input feature for face orientation estimation; acquiring an orientation of an input face image according to a random forest face orientation decision tree; using a feature point mean value of a face training sample under the orientation as an initial value of the input face image feature point; and extracting the pose index feature of the face image and inputting the pose index feature into the optimal weak regressor to acquire a distribution residual to update the current feature point distribution, and complete the face feature point detection. The method can achieve a stable face feature point detection effect, and is suitable for various intelligent systems such as a face detection and recognition system, a human-computer interaction system, an expression recognition system, a driver fatigue detection system, and a gaze tracking system.

Claims

1. A multi-pose face feature point detection method based on cascade regression, comprising the following steps of:

(1) extracting pose index features and establishing corresponding optimal weak regressors;

using a clustering algorithm to cluster face feature points to acquire feature point categories with adjacent positions;

extracting pose index features under corresponding poses according to the feature point categories; and

inputting the pose index features into a cascade regression algorithm, and training the pose index features to acquire the corresponding optimal weak regressors under different face poses; and

(2) performing initialization and detection on face feature points under multi-pose changes;

performing corresponding initialization according to different face pose orientations;

using an SIFT feature of a face image as an input feature for face orientation estimation;

acquiring an orientation of an input face image according to a random forest face orientation decision tree;

using a feature point mean value of a face training sample under the orientation as an initial value of the input face image feature point; and

extracting the pose index feature of the face image and inputting the pose index feature into the optimal weak regressor to acquire a distribution residual to update the current feature point distribution, and complete the face feature point detection.

2. The method according to claim 1, wherein in the step (1), the poses are divided into different training pose subsets according to the face orientation;

in each pose subset, a k-means clustering algorithm is used to cluster the same category of feature points in the internal feature points of the face, and a mean value of a clustering center is continuously updated until a sum of square errors D=[*]Σ

_i=1^kΣ

<sub>p⊂

Ci</sub>|p−

m_i|² convergence, wherein C is a clustered cluster, k is a number of clusters selected according to different pose subsets, i is a face feature point, and m_i is a mean value of an i^th cluster; and

finally, the internal feature points of the face are clustered into five categories;

left eye, right eye, nose, mouth, and chin.

3. The method according to claim 1, wherein in the step (1), the process of extracting the pose index features under different poses comprises:

in training subsets with front face orientation, randomly selecting three types of candidate feature points from the five categories of left eye, right eye, nose, mouth and chin;

in training subsets with left side face and left face orientations, randomly selecting three types of candidate feature points from the four categories of right eye, nose, mouth and chin;

similarly, in training subsets with right side face and right face orientations, randomly selecting three types of candidate feature points from the four categories of left eye, nose, mouth and chin;

after selecting corresponding feature point sets, acquiring a mean value feature of a face triangular region composed of the feature point sets.

4. The method according to claim 3, wherein the step of acquiring the mean value feature of the face triangular region comprises:

step a;

acquiring local triangular region template sets;

after selecting corresponding three types of candidate feature points in face training subsets with different poses, randomly selecting a feature point in each type as a vertex, and finally forming a closed triangular region by the three vertices, wherein the regions form N local region template sets L={L_n,n∈

[1,N]};

step b;

mapping reference points;

mapping all templates in the set L to samples, randomly selecting a reference point Q, and determining a feature point distance by recording a distance d between the reference points and a nearest vertex Q′

in the local region to complete mapping the reference points; and

step c;

solving the mean value feature of the local triangular region;

after completely mapping the reference points, calculating the feature points, and acquiring a mean value

5. The method according to claim 1, wherein in the step (2), in the face orientation estimation, the SIFT feature of the input face image is extracted as a classification feature, and input into the trained random forest decision tree, that is, a probability of each category is accumulated, and output as a category with a largest sum of predicted probabilities in all the trees, and categories with different face orientation are acquired.