Face recognition system and method

• US 9,430,694 B2
• Filed: 11/06/2014
• Issued: 08/30/2016
• Est. Priority Date: 11/06/2014
• Status: Active Grant

• Current Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Original Assignee: TCL Research America, Inc. (TCL Technology Group Corp.)
• Inventors: Iliadis, Michael, Wang, Haohong
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US14/534,688
• Publication Number:
  US 20160132717A1
• Time in Patent Office: 663 Days
• Field of Search: 382/103, 382/107, 382115-118, 382/124, 382155-167, 382/190, 382/217, 382/218
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks

Abstract

A face recognition method is provided. The method includes dividing an input video into different sets of frames and detecting faces of each frame in the input video. The method also includes generating face tracks for the whole video. Further, the method includes applying a robust collaborative representation-based classifier to recover a clean image from complex occlusions and corruptions for a face test sample and perform classification. In addition, the method also includes outputting the video containing the recognized face images.

Claims

1. A face recognition method, comprising:

dividing an input video into different sets of frames;

detecting faces of each frame in the input video;

generating face tracks for the input video;

applying a robust collaborative representation-based classifier to recover a clean image from complex occlusions and corruptions for a face test sample from one of the face tracks and performing classification; and

outputting a recognized face identity of the face test sample based on results from the classification;

wherein applying the robust collaborative representation-based classifier to recover the clean image from complex occlusions and corruptions for the face test sample and performing classification further includes;

estimating the clean image through an inductive robust principal component analysis (IRPCA) algorithm to initialize a low-rank representation with an l₁ half quadratic (LRR-HQ-L1) algorithm;

estimating a weight matrix through the LRR-HQ-L1 algorithm;

performing classification through the robust collaborative representation (RCR) algorithm; and

giving a final decision of a class identity of the face test sample based on classification results from the RCR algorithm.

2. The method according to claim 1, wherein:

an outlier detection inputs the face test sample denoted as y and a face dictionary T, and produces a weight diagonal matrix W as the weight matrix,

the outlier detection (OLD) is defined by;

W=OLD(y,T)

wherein yε

^d denotes the face test sample;

T=[T_i, . . . , T_c]ε

<sup>d×

n</sup> denotes a matrix with a set of samples of c subjects stacked in columns; and

T_iε

<sup>d×

ni</sup> denotes the n_i set of samples of the i^th subject, such that Σ

_i n_i=n.

3. The method according to claim 2, wherein:

Py represents a face image obtained by removing corruptions from the face test sample based on training data in the face dictionary, and AZ represents a mean image from the training data, the clean image is defined by;

y_c=θ

Py+(1−

θ

)AZ,

wherein 0≦

θ

≦

1 and θ

is estimated empirically.

4. The method according to claim 3, wherein:

an estimation vector f for classification is defined by;

f=argmin_f∥

W(y−

Tf)∥

₂²+λ

∥

f∥

₂²,

where fε

ⁿ is the estimated vector;

yε

^d denotes the face test sample;

T=[T_i, . . . , T_c]ε

<sup>d×

n</sup> denotes the matrix with the set of samples of c subjects stacked in columns;

λ

is a parameter greater than 0; and

W is a weight diagonal matrix.

5. The method according to claim 4, wherein:

an additive robust collaborative representation classifier (ARCR-C) inputs the face test sample y and the face dictionary T and residual of each class is defined by;

e_i(y)=Σ

_j=1^kξ

_je_i^j(y)

wherein ξ

_j is a weight of the j^th classifier;

k is an integer greater than 1;

e_iⁱ(y) denotes the residual of the i^th class of the j^th classifier and is defined by;

e_i^j(y)=∥

W_j(y−

T_if_i^j)∥

₂,

wherein W_j is a weight outlier detection matrix;

f_i^j is an estimated vector; and

T_iε

<sup>d×

ni</sup> denotes the n_i set of samples of the i^th subject, such that Σ

_i n_i=n.

6. The method according to claim 5, wherein:

the weight matrix diagonal is from a first initialization W_lrr and from a second initialization W_mean and an identity of a subject is defined by;

Identity(y)=argmin_i{ξ

₁e_i^lrr+ξ

₂e_i^mean}

yε

^d denotes the face test sample;

e_i^lrr denotes the residual of the i^th class of the first classifier;

e_i^mean denotes the residual of the i^th class of the second classifier;

ξ

₁ is a weight of the first classifier; and

ξ

₂ is a weight of the second classifier.

7. The method according to claim 1, wherein:

the classification (CLF) process inputs the face test sample y, the face dictionary T and the weight diagonal matrix W and an estimated class index is defined by;

Identity(y)=CLF(y,T,W).

8. A face recognition system, comprising one or more processors, memory, and one or more program modules stored in the memory and to be executed by the one or more processors, the one or more program modules including:

a face detection module configured to find automatically, location of faces in a sequence of video frames;

an algorithm module configured to recover a clean image from complex occlusions and corruptions for a face test sample obtained from the face detection module through an inductive robust principal component analysis (IRPCA) algorithm to initialize a low-rank representation with an l₁ half quadratic (LRR-HQ-L1) algorithm and estimate a weight matrix through the LRR-HQ-L1 algorithm;

a face classifier configured to perform classification through a robust collaborative representation (RCR) algorithm;

a dictionary configured to store face images in a database; and

an output module configured to output recognized face identity of the face test sample based on results from the face classifier;

wherein;

an outlier detection inputs a face test sample y and a face dictionary T, and produces a weight diagonal matrix W as the weight matrix,

the outlier detection (OLD) is defined by;

W=OLD(y,T)

wherein yε

^d denotes the face test sample;

T=[T_i, . . . , T_c]ε

<sup>d×

n</sup> denotes a matrix with a set of samples of c subjects stacked in columns; and

T_iε

<sup>d×

ni</sup>;

denotes the n_i set of samples of the i^th subject, such that Σ

_i n_i=n.

9. The system according to claim 8, wherein:

a classification (CLF) process inputs the face test sample y, the face dictionary T and the weight diagonal matrix W, an estimated class index is defined by;

Identity(y)=CLF(y,T,W).

10. The system according to claim 8, wherein:

Py represents a face image obtained by removing corruptions from the face test sample based on training data in the face dictionary, and AZ represents a mean image from the training data, the clean image is defined by;

y_c=θ

Py+(1−

θ

)AZ,

wherein 0≦

θ

≦

1 and θ

is estimated empirically.

11. The system according to claim 10, wherein:

an estimation vector f for classification is defined by;

f=argmin_f∥

W(y−

Tf)∥

₂²+λ

∥

f∥

₂²,

where fε

ⁿ is the estimated vector;

yε

^d denotes the face test sample;

T=[T_i, . . . , T_c]ε

<sup>d×

n</sup> denotes the matrix with the set of samples of c subjects stacked in columns;

λ

is a parameter greater than 0; and

W is a weight diagonal matrix.

12. The system according to claim 11, wherein:

an additive robust collaborative representation classifier (ARCR-C) inputs the face test sample y and the face dictionary T, residual of each class is defined by;

e_i(y)=Σ

_j=1^kξ

_je_i^j(y)

wherein ξ

_j is a weight of the j^th classifier;

k is an integer greater than 1;

e_iⁱ(y) denotes the residual of the i^th class of the j^th classifier and is defined by;

e_i^j(y)=∥

W_j(y−

T_if_i^j)∥

₂,

wherein W_j is a weight outlier detection matrix;

f_i^j is an estimated vector; and

T_iε

<sup>d×

ni</sup> denotes the n_i set of samples of the i^th subject, such that Σ

_i n_i=n.

13. The system according to claim 12, wherein:

a weight matrix diagonal is from a first initialization W_lrr and from a second initialization W_mean, an identity of a subject is defined by;

Identity(y)=argmin_i{ξ

₁e_i^lrr+ξ

₂e_i^mean}

yε

^d denotes the face test sample;

e_i^lrr denotes the residual of the i^th class of the first classifier;

e_i^mean denotes the residual of the i^th class of the second classifier;

ξ

₁ is a weight of the first classifier; and

ξ

₂ is a weight of the second classifier.