System and method for rapidly tacking multiple faces

• US 20020176609A1
• Filed: 08/17/2001
• Published: 11/28/2002
• Est. Priority Date: 05/25/2001
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Huang, Yea-Shuan, Hsieh, Chun-Wei
• Granted Patent: US 7,003,135 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A system and a method for rapidly tacking multiple faces are disclosed. A face-like region generator finds a face-like region by skin color, motion, and silhouette information. A face tracking engine tracks faces based on new and old faces, and skin colors provided by the face-like regions. The tracked face is fed into a face status checker for determining whether the face-like regions are old faces tracked in a previous frame or are possible new faces. If the face-like regions are old faces, a face verification engine checks whether there exists a predefined percentage of overlapping area between an old face and a skin region. If yes, the old face is still in the current frame and its position is in the center of the skin region, otherwise, the position of the old face is found by a correlation operation.

Claims

1. A system for rapidly tacking multiple faces comprising:

a face-like region generator having a skin region extractor and a motion analyzer, the skin region extractor generating a plurality of skin regions by detecting skin color pixels of an input image, the motion analyzer determining possible face-like regions from the skin regions based on moving information of the input image;

a face recorder for recording tracked faces;

a face status checker for checking the face-like regions and the faces previously tracked and recorded in the face recorder to determine whether the face-like regions are old faces which have been tracked in a previous frame or are possible new faces;

a face verification engine for determining whether the possible new faces are true new faces; and

a face tracking engine for tracking multiple faces based on the new and old faces, and the skin regions provided by the skin region extractor, wherein, when a tracked face is a new face, the face tracking engine directly adds the new face to the face recorder;

when a tracked face is an old face, the face tracking engine determines whether there exists more than a predefined percentage of overlapping area between the old face and a skin region, and if yes, it is determined that the old face is still in the current frame and its position is in the center of the skin region, otherwise, the position of the old face is determined by correlation operation.

2. The system as claimed in claim 1, wherein, if the face tracking engine determines that there exists more than a predefined percentage of overlapping area between an old face and a skin region, the face-like region is labeled as a tracked old face.

3. The system as claimed in claim 1, wherein, in the motion analyzer, luminance difference between two successive images is used as a moving information, and a pixel is defined as a moving pixel if its luminance difference between two adjacent images is larger than a threshold;

if there are more than a predefined percentage of pixels classified as moving pixels in a skin region, the region is labeled as a possible face-like region.

4. The system as claimed in claim 3, wherein the skin region extractor generates a plurality of connected skin regions based on an adaptive skin color model representing skin features of different people.

5. The system as claimed in claim 4, further comprising an adjuster for dynamically updating the adaptive skin color model in response to variation of the face recorder.

6. The system as claimed in claim 5, wherein the face-like region generator further has a silhouette analyzer for analyzing whether there exists a protrusion shape in the image so as to separate connected face regions.

7. The system as claimed in claim 6, wherein the silhouette analyzer determines that there exists a face-like region based on the following condition:

e(i)>

w

and

e(j)<

−

w

where

w=0.5*(p(j)−

p(i));

e(i)=d(p(i));

d(x)=v(x−

1)−

v(x+1);

v(x) denotes a vertical position of the first touched pixel of a connected region when tracking all pixels of an image along the x-th column from top to down.

8. The system as claimed in claim 6, wherein the face verification engine has a filter and an verification processor;

said filter filters out false faces from input possible new faces, and the other possible new faces are fed into the verification processor for finding true new faces by eigen-face analysis.

9. The system as claimed in claim 8, wherein the filter finds a false face based on a combination of compactness of a tracked region, ratio between the height and width of a face, statistic variance of a face, number of holes existed in a region, and convexity of a face.

18. The method as claimed in claim 1, wherein in step (C), if there exists more than a predefined percentage of overlapping area between the face-like region and a tracked face, the face-like region is an old face that has been previously tracked.

10. A system for rapidly tacking multiple faces comprising:

a face-like region generator having a skin region extractor, a motion analyzer and a silhouette analyzer, the skin region extractor generating a plurality of skin regions by detecting skin color pixels of an input image;

the motion analyzer determining possible face-like regions from the skin regions based on moving information of the input image, the silhouette analyzer analyzing whether there exists a protrusion shape in the image so as to separate connected regions.

a face verification engine for determining that the possible faces are new faces; and

a face tracking engine for tracking multiple faces based on the faces and the skin regions provided by the skin region extractor.

11. The system as claimed in claim 10, further comprising:

a face recorder for recording faces that have been by the face tracking engine;

a face status checker for checking the face-like regions and the faces previously tracked and recorded in the face recorder to determine whether the face-like regions are old faces which have been tracked in a previous frame or are possibly new faces, wherein only possible new faces are fed into the face verification engine, and the old faces are directly fed into the face tracking engine.

12. The system as claimed in claim 11, wherein, if the face status checker determines that there exists more than a predefined percentage of overlapping area between a face-like region and a tracked face, the face-like region is labeled as a tracked old face.

13. The system as claimed in claim 11, wherein the face verification engine has a filter and a verification processor;

the filter filters out fault faces from input possible new faces, and the other possible new faces are fed into the verification processor for finding true new faces by eigen-face analysis.

14. The system for tacking multiple faces rapidly as claimed in claim 13, wherein the filter finds a false face based on a combination of compactness of a tracked region, ratio between the height and width of a face, statistic variance of a face, number of holes existed in a region, and convexity of a face.

15. The system as claimed in claim 13, wherein, when a tracked face is a new face, the face tracking engine directly adds the new face to the face recorder;

when a tracked face is an old face, the face tracking engine determines whether there exists more than a predefined percentage of overlapping area between the old face and a skin region, and if yes, it is determined that the old face is still in the current frame and its position is in the center of the skin region, otherwise, the position of the old face is determined by correlation operation.

16. The system for tacking multiple faces rapidly as claimed in claim 11, wherein the skin region extractor generates a plurality of connected skin regions based on an adaptive skin color model representing skin features of different person.

19. The method as claimed in claim 17, wherein in step (B), luminance difference between two successive images is used as a moving information, and a pixel is defined as a moving pixel if its luminance difference between two adjacent images is larger than a threshold;

if there is more than a predefined percentage of pixels classified as moving pixels in a skin region, the region is labeled as a possible face-like region.

20. The method as claimed in claim 17, further comprising a step (B′

) between step (B) and step (C) for analyzing whether there exists a protrusion shape in the image so as to separate connected face regions.

21. The method as claimed in claim 20, wherein, in step (B′

), it is determined that there exists a face-like region based on the following conditions;

e(i)>

w

and

e(j)<

−

w,

where

w=0.5*(p(j)−

p(i));

e(i)=d(p(i));

d(x)=v(x−

1)−

v(x+1);

v(x) denotes a vertical position of the first touched pixel of a connected region when tracking all pixels of an image along the x-th column from top to down.

22. The method as claimed in claim 20, wherein, in step (D), false faces from the input possible new faces are first filtered out, and the other possible new faces are verified for finding true new faces by eigen-face analysis.

23. The method as claimed in claim 20, wherein, in step (D), the false faces are found based on a combination of compactness of a tracked region, ratio between the height and width of a face, statistic variance of a face, number of holes existed in a region, and convexity of a face.

17. A method for rapidly tacking multiple faces comprising the steps of:

(A) detecting skin color pixels of an input image for generating a plurality of skin regions;

(B) determining possible face-like regions in the skin regions based on moving information of the input image;

(C) checking the face-like regions and tracked faces previously stored to determine whether the face-like regions are old faces that have been tracked in a previous frame or are possible new faces, wherein, if the face-like regions are old faces, it is further determined whether there exists more than a predefined percentage of overlapping area between an old face and a skin region, and if yes, the old face are still in the current frame and its position is the center of the skin region, otherwise, the position of the old face is determined by correlation operation; and

(D) determining whether the possible new face is a true new face, and if yes, recording the new face.