Method and system for robust human ethnicity recognition using image feature-based probabilistic graphical models
First Claim
1. A method for determining ethnicity of people based on their facial images captured by at least a means for capturing images for a target person, comprising the following steps of:
- a) forming image filter groups in a plurality of image filters,b) computing filter responses by applying the image filters to facial images from an ethnicity-annotated facial images database,c) computing joint histograms of the filter responses within each filter group,d) deriving ethnicity-sensitive image features from the image filter groups based on contributions of the groups to a task of recognizing a given ethnic class against other ethnic classes,e) constructing probabilistic graphical models for recognizing the ethnicity of people using the ethnicity-sensitive image features as observable quantities of the probabilistic graphical models,f) training the probabilistic graphical models using the ethnicity-annotated facial images so that each of the probabilistic graphical models is dedicated to represent one ethnic class,g) extracting the ethnicity-sensitive image features of an input face, andh) determining the ethnic class of the input face based on the outputs computed from the probabilistic graphical models using the ethnicity-sensitive image features,wherein the ethnicity-sensitive image features are extracted based on specific locations from the facial images, andwherein the contribution of the grouping is computed based on comparison between the first joint histograms of the image filter responses conditioned on the given ethnic class and the second joint histograms of the image filter responses conditioned on other ethnic classes.
17 Assignments
0 Petitions
Accused Products
Abstract
The present invention is a method and system to provide a face-based automatic ethnicity recognition system that utilizes ethnicity-sensitive image features and probabilistic graphical models to represent ethnic classes. The ethnicity-sensitive image features are derived from groups of image features so that each grouping of the image features contributes to more accurate recognition of the ethnic class. The ethnicity-sensitive image features can be derived from image filters that are matched to different colors, sizes, and shapes of facial features—such as eyes, mouth, or complexion. The ethnicity-sensitive image features serve as observable quantities in the ethnic class-dependent probabilistic graphical models, where each probabilistic graphical model represents one ethnic class. A given input facial image is corrected for pose and lighting, and ethnicity-sensitive image features are extracted. The extracted image features are fed to the ethnicity-dependent probabilistic graphical models to determine the ethnic class of the input facial image.
-
Citations
8 Claims
-
1. A method for determining ethnicity of people based on their facial images captured by at least a means for capturing images for a target person, comprising the following steps of:
-
a) forming image filter groups in a plurality of image filters, b) computing filter responses by applying the image filters to facial images from an ethnicity-annotated facial images database, c) computing joint histograms of the filter responses within each filter group, d) deriving ethnicity-sensitive image features from the image filter groups based on contributions of the groups to a task of recognizing a given ethnic class against other ethnic classes, e) constructing probabilistic graphical models for recognizing the ethnicity of people using the ethnicity-sensitive image features as observable quantities of the probabilistic graphical models, f) training the probabilistic graphical models using the ethnicity-annotated facial images so that each of the probabilistic graphical models is dedicated to represent one ethnic class, g) extracting the ethnicity-sensitive image features of an input face, and h) determining the ethnic class of the input face based on the outputs computed from the probabilistic graphical models using the ethnicity-sensitive image features, wherein the ethnicity-sensitive image features are extracted based on specific locations from the facial images, and wherein the contribution of the grouping is computed based on comparison between the first joint histograms of the image filter responses conditioned on the given ethnic class and the second joint histograms of the image filter responses conditioned on other ethnic classes. - View Dependent Claims (2, 3, 4)
-
-
5. A system for determining ethnicity of people based on their facial images, comprising:
-
a) an annotation system that comprises a human annotator, an external storage with a facial image database, and a computer system that consists of a visual display, an input device, a control and processing system, and an internal storage, b) a training system that comprises a computer system having a control and processing system and an internal storage, wherein the training system is programmed to perform the following steps of; forming image filter groups in a plurality of image filters, computing filter responses by applying the image filters to facial images from an ethnicity-annotated facial images database, computing joint histograms of the filter responses within each filter group, deriving ethnicity-sensitive image features from the image filter groups based on contributions of the groups to a task of recognizing a given ethnic class against other ethnic classes, constructing probabilistic graphical models for recognizing the ethnicity of people using the ethnicity-sensitive image features as observable quantities of the probabilistic graphical models, and training the probabilistic graphical models using the ethnicity-annotated facial images so that each of the probabilistic graphical models is dedicated to represent one ethnic class, and c) an ethnicity classification system that comprises at least a means for capturing images, a computer system having a control and processing system and an internal storage, wherein the ethnicity classification system is programmed to perform the following steps of; extracting the ethnicity-sensitive image features of an input face, and determining the ethnic class of the input face based on the outputs computed from the probabilistic graphical models using the ethnicity-sensitive image features, wherein the ethnicity-sensitive image features are extracted based on specific locations from the facial images, and wherein the contribution of the grouping is computed based on comparison between the first joint histograms of the image filter responses conditioned on the given ethnic class and the second joint histograms of the image filter responses conditioned on other ethnic classes. - View Dependent Claims (6, 7, 8)
-
Specification