Method for robust human face tracking in presence of multiple persons

• US 6,404,900 B1
• Filed: 01/14/1999
• Issued: 06/11/2002
• Est. Priority Date: 06/22/1998
• Status: Expired due to Term

• Current Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Qian, Richard J., Matthews, Kristine E.
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US09/231,937
• Time in Patent Office: 1,244 Days
• Field of Search: 382/171, 382/103, 382/118, 382/250, 382/174, 382/107, 382/165, 382/169, 382/236, 348/169, 367/125
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

A method for robust human face tracking in the presence of multiple facial images includes taking a frame from a color video sequence as a current input image; filtering the current input image to form a filtered image; estimating the locations and sizes of faces in the filtered image based on a projection histogram of the filtered image; estimating face motion in the filtered image; and outputting the location and size of the tracked faces within the filtered image.

Claims

1. A method for robust human face tracking in the presence of multiple facial images comprising:

taking a frame from a color video sequence as a current input image;

filtering the current input image in chromaticity space;

estimating the locations of faces in the image based on a projection histogram of the filtered image, including;

determining a sample mean μ and

standard deviation σ

using all of the samples in a distribution;

setting μ

_t(0)=μ and

δ

=max (a*σ

, b*image-width) where a and b are scaling factors, resulting in an initial trimmed mean μ

(k) within an initial trimmed interval (k);

determining a trimmed mean μ

_t (k+1) which is used to define the interval [μ

_t(k)−

δ

, μ

_t(k)+δ

];

redetermining the trimmed mean until |μ

_t (k+1)−

μ

_t(k)|<

ε

where ε

is tolerance; and

determining a robust mean μ

* within a final trimmed interval; and

extracting and outputting the tracked face regions.

2. The method of claim 1 wherein said filtering includes filtering to represent the current input image in chromaticity space.

3. The method of claim 1 which includes estimating face motion via a linear Kalman filter.

4. The method of claim 3 which further includes smoothing the face motion during a post-processing step.

5. A method for robust human face tracking in the presence of multiple facial images comprising:

taking a frame from a color video sequence as a current input image;

filtering the current input image in chromaticity space;

estimating the locations of faces in the image based on a projection histogram of the filtered image;

determining the size of a facial area by;

determining the trimmed standard deviation σ

_t based on the samples within the interval [μ

*−

δ

, μ

*+δ

]; and

setting size=c*σ

_t where c is a scaling factor; and

extracting and outputting the tracked face regions.

6. The method of claim 5 wherein said filtering includes filtering to represent the current input image in chromaticity space.

7. The method of claim 5 which further includes estimating face motion via a linear Kalman filter.

8. The method of claim 7 which further includes smoothing the face motion during a post-processing step.

9. A method for robust human face tracking in the presence of multiple facial images comprising:

taking a frame from a color video sequence as a current input image;

filtering the current input image in chromaticity space;

estimating the locations of faces in the image based on a projection histogram of the filtered image

determining the size of a facial area by;

determining the trimmed standard deviation σ

_t based on the samples within the interval [μ

*−

δ

, μ

*+δ

];

If h(μ

*+d*σ

_t)≧

g*h(μ

*) or h(μ

*−

d*σ

_t)≧

g*h(μ

*) where, e.g., d=1.0 and g=0.4, then increase σ

_t until the condition is no longer true; and

setting size=c*σ

_t, where c is a scaling factor; and

extracting and outputting the tracked face regions.

10. The method of claim 9 wherein said filtering includes filtering to represent the current input image in chromaticity space.

11. The method of claim 9 which further includes estimating face motion via a linear Kalman filter.

12. The method of claim 11 which further includes smoothing the face motion during a post-processing step.

16. The method of claim 9 which further includes smoothing the face motion during a post-processing step.

13. A method for robust human face tracking in the presence of multiple facial images comprising:

taking a frame from a color video sequence as a current input image;

filtering the current input image in chromaticity space;

estimating the locations of faces in the image based on a projection histogram of the filtered image

determining the size of a facial area by;

 

setting $\mathrm{width}=\frac{1}{\left(2<br><br>N+1\right)}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{i=-N}{\overset{N}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>h_y<br><br>\left({\mathrm{\mu <br><br>}}_y^{*}+i\right);<br><br>$

 

and

 

setting $\mathrm{height}=\frac{1}{\left(2<br><br>N+1\right)}<br><br>\text{\hspace{1em}<br><br>}<br><br>\underset{i=-N}{\overset{N}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>h_x<br><br>\left({\mathrm{\mu <br><br>}}_x^{*}+i\right),$

 

wherein N determines the number of samples used in the determination of the size of the facial image; and

extracting and outputting the tracked face regions.

14. The method of claim 13 wherein said filtering includes filtering to represent the current input image in chromaticity space.

15. The method of claim 13 which further includes estimating face motion via a linear Kalman filter.

17. A method for robust human face tracking in the presence of multiple facial images comprising:

taking a frame from a color video sequence as a current input image;

filtering the current input image in chromaticity space;

estimating the locations of faces in the image based on a projection histogram of the filtered image

determining the size of a facial area by determining the size from a projection histogram of a clipped region of the color-filtered image by;

forming the clipped projection histogram h_y^c by projecting columns in the color-filtered image within the range [μ

_x*−

Δ

, μ

_x*+Δ

], where Δ

determines the number of samples used in the determination;

determining size in the y direction based on h_y^c; and

estimating face locations for tracking facial motion; and

extracting and outputting the tracked face regions.

18. The method of claim 17 wherein said filtering includes filtering to represent the current input image in chromaticity space.

19. The method of claim 17 which further includes estimating face motion via a linear Kalman filter.

20. The method of claim 19 which further includes smoothing the face motion during a post-processing step.