UNIFIED FACE REPRESENTATION FOR INDIVIDUAL RECOGNITION IN SURVEILLANCE VIDEOS AND VEHICLE LOGO SUPER-RESOLUTION SYSTEM

• US 20160217319A1
• Filed: 10/01/2013
• Published: 07/28/2016
• Est. Priority Date: 10/01/2012
• Status: Active Grant

• Current Assignee: Regents of the University of California (University of California)
• Original Assignee: Regents of the University of California (University of California)
• Inventors: BHANU, Bir, An, Le, Thakoor, Ninad
• Granted Patent: US 9,928,406 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/754 involving a deformation of the sample pattern or of the reference pattern; Elastic matching
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G06V 40/164 using holistic features
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

A new image-based representation and an associated reference image is disclosed called the emotion avatar image (EAI), and the avatar reference, respectively, which leverages the out-of-plane head rotation. The method is not only robust to outliers but also provides a method to aggregate dynamic information from expressions with various lengths. The approach to facial expression analysis can consist of the following steps: 1) face detection; 2) face registration of video frames with the avatar reference to form the EAI representation; 3) computation of features from EAI using both local binary patterns and local phase quantization; and 4) the classification of the feature as one of the emotion type by using a linear support vector machine classifier.

Claims

1. A method for unified face representation for individual recognition in surveillance videos, comprising:

extracting a face from a video;

generating an emotion avatar image (EAI) representation of the face from the video;

computing features from the EAI using both local binary patterns (LBP) and local phase quantization (LPQ); and

classifying the features as one of an emotion type by using a linear support vector machine classifier.

2. The method of claim 1, further comprising:

extracting the face from the video using a Viola and Jones face detector.

3. The method of claim 1, further comprising:

resizing a facial resolution of the detected face using bicubic interpolation.

4. The method of claim 1, further comprising:

forming the EAI representation using SIFT flow alignment.

5. The method of claim 1, wherein the EAI representation is a face registration of video frames with an avatar reference, which forms the EAI representation.

6. The method of claim 1, further comprising:

performing a texture descriptor on the EAI.

7. The method of claim 6, wherein the texture descriptor is a local binary pattern (LBP).

8. The method of claim 6, wherein the texture descriptor is blur-insensitive local phase quantization (LPQ) descriptor.

9. A method of recognizing a logo of a vehicle, comprising:

obtaining a limited number of high resolution logos;

populating a training dataset for each of the limited number of high resolution logos using gamma transformations;

obtaining a low resolution image of a vehicle; and

matching the low resolution image of the vehicle with the training dataset.

10. The method of claim 9, further comprising:

selecting a final output that is closest to the low-resolution image in a histogram of oriented gradients (HOG) feature space.

11. The method of claim 9, where the method is a canonical correlation analysis (CCA) based method for vehicle logo super-resolution to facilitate the recognition of a vehicle make.

12. The method of claim 9, further comprising:

performing the method using a computer system; and

obtaining the low resolution image of the vehicle is obtained from a surveillance or traffic camera.

13. The method of claim 9, further comprising:

detecting a license plate (LP) of a vehicle; and

selecting a region of interest (ROI).

14. The method of claim 13, wherein the region of interest is of a size 4LP height×

LP width immediately above the LP.

15. The method of claim 14, comprising

automatically segmenting and cropping the region of interest to detect the logos;

and

normalizing the low-resolution logos to a same size as those in the training set.

16. A computer program product comprising a non-transitory computer usable medium having computer readable code embodied therein for recognizing a logo on a vehicle, which includes the steps of:

obtaining a limited number of high resolution logos;

populating a training dataset for each of the limited number of high resolution logos using gamma transformations;

obtaining a low resolution image of a vehicle; and

matching the low resolution image of the vehicle with the training dataset.

17. The computer product of claim 16, further comprising:

selecting a final output that is closest to the low-resolution image in a histogram of oriented gradients (HOG) feature space.

18. The computer product of claim 16, where the method is a canonical correlation analysis (CCA) based method for vehicle logo super-resolution to facilitate the recognition of a vehicle make.

19. The computer product of claim 16, further comprising:

performing the method using a computer system; and

obtaining the low resolution image of the vehicle from a surveillance or traffic camera.

20. The computer product of claim 16, further comprising:

detecting a license plate (LP) of a vehicle; and

selecting a region of interest (ROI), wherein the region of interest is of a size 4LP height×

LP width immediately above the LP; and

automatically segmenting and cropping the region of interest to detect the logos.