System and method for biometrics-based facial feature extraction

• US 6,526,161 B1
• Filed: 08/30/1999
• Issued: 02/25/2003
• Est. Priority Date: 08/30/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Yan, Yong
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US09/385,280
• Time in Patent Office: 1,275 Days
• Field of Search: 382/118, 382/154, 382/173, 382/199, 340/5.83, 340/5.53, 356/12
• US Class Current: 382/118
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

A method and image processing system are disclosed that extract facial feature information from an image using biometrics information of a face. Regions of interests such as a face, eyes, nose and mouth are defined in the image. A combination of disparity mapping, edge detection and filtering are then used to extract coordinates/positions of the facial features in the regions of interest.

Claims

1. An image processing device comprising:

a disparity detector arranged to compare locations of like pixel information in a pair of images to determine disparity information;

a region detector which identifies a region of interest in one of the images in accordance with the disparity information, the region of interest including a plurality of facial features;

a first position detector coupled to the region detector which identifies a position of a nose in accordance with the disparity information; and

a second position detector coupled to the first position detector which identifies a location of a mouth in accordance with the position of the nose,

wherein the second position detector identifies an approximate area for the location of the mouth based upon the position of the nose and determines the location of the mouth in accordance with an edge density of the approximate area and a Radon transformation.

2. The image processing device according to claim 1, wherein the position corresponds to a location where a disparity value is largest as compared to other disparity values in the region of interest.

3. The image processing device according to claim 2, wherein the first position detector includes a sub-region detector which identifies a portion of the region of interest in which the nose is located before comparing the disparity values.

4. The image processing device according to claim 1, wherein a result of the Radon transformation is weighted by a Gaussian filter.

5. The image processing device according to claim 1, wherein the pair of images are received from a stereo pair of cameras in a video conference system.

6. A method of determining positions of facial features in an image comprising the steps of:

calculating a disparity between a pair of images;

determining a face region of interest (ROI) in at least one of the images;

identifying a nose position within the face region of interest in accordance with the calculated disparity; and

identifying a mouth position within the face ROI in accordance with a Radon transformation operation on an edge density value of the at least one image.

7. The method according to claim 6, wherein the calculating step includes:

identifying locations of like pixels in each of the pair of images; and

calculating a difference between the locations of like pixels.

8. The method according to claim 6, wherein the determining step includes determining for each set of like pixels whether a disparity value between the locations falls above or below a predetermined threshold, and if so identifying a scan line or pixel column as a boundary line for the face ROI.

9. The method according to claim 6, wherein the identifying step includes:

determining a center region of the face ROI;

calculating a histogram to determine a largest disparity value within the center region;

if more than one area within the center region is determined in the step of using the histogram, determining which area is largest as compared to the other areas; and

using a center of the largest area as the position of the nose position.

10. The method according to claim 6, further comprising the step of identifying an eye position within the face ROI in accordance with a luminance value and an edge density value of the at least one image.

11. The method according to claim 10, wherein the step of identifying the eye position includes the steps of:

determining an approximate area of the eye position; and

calculating the edge density value using a Sobel operation.

12. The method according to claim 6, wherein the step of identifying the mouth position includes:

weighting a result of the Radon transformation by a Gaussian filter; and

identifying a corner of the mouth position to be where the weighted result is greater than a predetermined threshold.

13. A computer-readable memory medium including code for processing a pair of images, the code comprising:

disparity detecting code to compare locations of like pixel information in a pair of images to determine disparity information;

region detecting code to identify a region of interest in one of the images in accordance with the disparity information, the region of interest including a plurality of facial features;

first position detecting code to identify a position of a nose in accordance with the disparity information; and

second position detecting code to identify a location of a mouth in accordance with the position of the nose and in accordance with an edge density and a Radon transformation.