Robust multi-view face detection methods and apparatuses

• US 20050013479A1
• Filed: 07/16/2003
• Published: 01/20/2005
• Est. Priority Date: 07/16/2003
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Xiao, Rong, Zhang, Lei, Zhang, Hong-Jiang, Zhu, Long
• Granted Patent: US 7,689,033 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

Face detection techniques are provided that use a multiple-stage face detection algorithm. An exemplary three-stage algorithm includes a first stage that applies linear-filtering to enhance detection performance by removing many non-face-like portions within an image, a second stage that uses a boosting chain that is adopted to combine boosting classifiers within a hierarchy “chain” structure, and a third stage that performs post-filtering using image pre-processing, SVM-filtering and color-filtering to refine the final face detection prediction. In certain further implementations, the face detection techniques include a two-level hierarchy in-plane pose estimator to provide a rapid multi-view face detector that further improves the accuracy and robustness of face detection.

Claims

1. A method for use in detecting faces within a digital image, the method comprising:

processing a set of initial candidate portions of digital image data in a boosting filter stage that uses a boosting chain to produce a set of intermediate candidate portions; and

processing said set of intermediate candidate portions in a post-filter stage to produce a set of final candidate portions.

2. The method as recited in claim 1, further comprising dividing a digital image into a plurality of portions.

3. The method as recited in claim 2, wherein at least one of said plurality of portions has a shape selected from a group of shapes comprising a rectangle and a square.

4. The method as recited in claim 1, further comprising processing said plurality of portions using a pre-filter stage that is configured to output said set of initial candidate portions selected from said plurality of portions based on at least one feature.

5. The method as recited in claim 4, wherein said feature includes at least one feature selected from a group of features comprising a Haar-like feature, an extended feature, a mirror invariant feature, and a variance feature.

6. The method as recited in claim 4, wherein said pre-filter stage includes a linear filter.

7. The method as recited in claim 6, wherein said linear filter is based on a weak learner.

8. The method as recited in claim 6, wherein said linear filter is based on a decision function of H(x)=(a₁f₁(x)>

b₁)Λ

(a₂(f₁(x)+rf₂(x))>

b₂) wherein a_i, b_i and rε

(−

1,1) are coefficients determined during a learning procedure.

9. The method as recited in claim 1, further comprising training said boosting chain using face images, non-face images, and weak classifiers.

10. The method as recited in claim 9, wherein said boosting chain includes a plurality of boosting nodes arranged in an order within said boosting chain.

11. The method as recited in claim 10, wherein said boosting chain is trained using boosting classifiers corresponding to said boosting nodes.

12. The method as recited in claim 10, wherein a sample weight initialized for a current boosting classifier is adjusted based on a classification error rate of a previous boosting node within said order.

13. The method as recited in claim 1, wherein said boosting chain includes a hierarchical chain structure.

14. The method as recited in claim 1, wherein said boosting filter stage includes an LSVM optimization.

15. The method as recited in claim 14, wherein said LSVM optimization is capable of finding a global maximum.

16. The method as recited in claim 15, wherein finding said global maximum based on:

Maximum

$L<br><br>\left(\mathrm{\beta <br><br>}\right)=\underset{i=1}{\overset{n}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\beta <br><br>}}_i-\frac{1}{2}<br><br>\underset{i,j=1}{\overset{n}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\beta <br><br>}}_i<br><br>{\mathrm{\beta <br><br>}}_j<br><br>y_i<br><br>y_j<br><br>\left(h<br><br>\left(x_i\right)·<br><br>h<br><br>\left(x_j\right)\right)$

subject to the constraints Σ

_iⁿβ

_iy_i=0 and c_i≧

β

_i≧

0 i=1, . . . , n, and wherein coefficient C_i is set according to a classification risk w and trade-off constant C over a training set

$C_i=\{\begin{array}{cc}\mathrm{wC}& \mathrm{if}<br><br>\text{\hspace{1em}<br><br>}<br><br>x_i<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{is}<br><br>\text{\hspace{1em}<br><br>}<br><br>a<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{face}<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{pattern}\\ C& \mathrm{otherwise}\end{array}.$

17. The method as recited in claim 1, wherein said post-filter stage includes image pre-processing and masking processing

18. The method as recited in claim 17, wherein said image pre-processing includes lighting correction processing.

19. The method as recited in claim 17, wherein said image pre-processing includes histogram equalization processing.

20. The method as recited in claim 1, wherein said post-filter stage includes at least a color filter process.

21. The method as recited in claim 1, wherein said post-filter stage includes at least an SVM filter process.

22. The method as recited in claim 1, further comprising outputting information associated with at least said final candidate portion.

23. The method as recited in claim 22, wherein said information identifies at least said final candidate portion.

24. The method as recited in claim 22, wherein said information includes at least said final candidate portion.

25. The method as recited in claim 22, wherein said information identifies rotation data associated with at least said final candidate portion.

26. The method as recited in claim 1, further comprising employing at least one feature-based algorithm.

27. The method as recited in claim 26, wherein said at least one feature-based algorithm uses Haar-like features.

28. The method as recited in claim 26, wherein said at least one feature-based algorithm uses extended features.

29. The method as recited in claim 26, wherein said at least one feature-based algorithm uses mirror invariant features.

30. The method as recited in claim 29, wherein said mirror invariant features are threshold configured.

31. The method as recited in claim 26, wherein said at least one feature-based algorithm uses variance features.

32. The method as recited in claim 1, further comprising performing in-plane estimation to detect an orientation of said face image data.

33. The method as recited in claim 32, wherein said orientation is with respect to an up-right position.

34. The method as recited in claim 33, further comprising performing up-right face detection based on said in-plane estimation.

35. The method as recited in claim 34, wherein said up-right face detection is configured to identify out-plane rotation variations of said face image data.

36. The method as recited in claim 35, wherein said out-plane rotation variations are within a range of Θ

=[−

45°

, 45°

].

37. The method as recited in claim 21, wherein said SVM filter process is performed in a redundancy reduced feature space.

38. The method as recited in claim 37, wherein said SVM filter process further includes performing* wavelet transformation to divided the original images into four sub-bands LL, HL, LH and HH.

39. The method as recited in claim 38, wherein said SVM filter process is configured to reduce said redundancy based on

$k^{\prime }$

⁡

( u , v ) = ∑

0 ≤

i <

4  

⁢

 

⁢

( s i ⁢

u i T ⁢

v i + r i ) 2 ,

wherein each vector u_i and v_i corresponds to an i^th sub-band portion.

40. The method as recited in claim 38, further comprising selectively cropping of four sub-band portions.

41. A computer-readable medium having computer-implementable instructions for causing at least one processing unit to perform acts comprising:

detecting possible human face image data within a digital image using a multiple stage face detection scheme that includes at least a boosting filter stage configured to process a set of initial candidate portions of digital image data using a boosting chain to produce a set of intermediate candidate portions, and a post-filter stage configured to process said set of intermediate candidate portions to produce a set of final candidate portions.

42. The computer-readable medium as recited in claim 41, further comprising dividing a digital image into a plurality of portions, and wherein at least one of said plurality of portions has a shape selected from a group of shapes comprising a rectangle and a square.

43. The computer-readable medium as recited in claim 41, wherein said multiple stage face detection scheme further includes a pre-filter stage that is configured to output said set of initial candidate portions selected from said plurality of portions based on at least one feature.

44. The computer-readable medium as recited in claim 43, wherein said feature includes at least one feature selected from a group of features comprising a Haar-like feature, an extended feature, a mirror invariant feature, and a variance feature.

45. The computer-readable medium as recited in claim 43, wherein said pre-filter stage includes a linear filter based on a weak learner.

46. The computer-readable medium as recited in claim 41, wherein said boosting chain is trained using face images, non-face images, and weak classifiers.

47. The computer-readable medium as recited in claim 46, wherein said boosting chain includes a plurality of boosting nodes arranged in an order within said boosting chain, said boosting chain is trained using boosting classifiers corresponding to said boosting nodes, and each of said boosting nodes is constructed based on its preceding node in said order.

48. The computer-readable medium as recited in claim 41, wherein said boosting chain includes a hierarchical chain structure.

49. The computer-readable medium as recited in claim 41, wherein said boosting filter stage includes an LSVM optimization.

50. The computer-readable medium as recited in claim 41, wherein said post-filter stage includes a masking process, a lighting correction process, a histogram equalization process, a color filter process, and an SVM filter process.

51. The computer-readable medium as recited in claim 41, further comprising employing at least one feature-based algorithm that uses at least one feature selected from a group of features including at least one Haar-like feature, at least one extended feature, at least one mirror invariant feature, and at least one variance features.

52. The computer-readable medium as recited in claim 41, further comprising performing in-plane estimation to predict an orientation of said face image data and applying an up-right detector to pre-rotated image data corresponding to the orientation prediction.

53. The computer-readable medium as recited in claim 50, wherein said SVM filter process is configured to reduce redundancy in a feature space associated with at least one intermediate candidate portion, and performs wavelet transformation of said at least one intermediate candidate portion to produce a plurality of sub-bands portions.

54. The computer-readable medium as recited in claim 52, further comprising selectively cropping at least one of said plurality of sub-band portions.

55. An apparatus comprising:

logic operatively configured to detect at least one human face within a digital image using a multiple stage face detection process that includes at least a boosting filter stage configured to process a set of initial candidate portions of digital image data using a boosting chain to produce a set of intermediate candidate portions, and a post-filter stage configured to process said set of intermediate candidate portions to produce a set of final candidate portions, wherein at least one of said final candidate portions includes detected face image data.

56. The apparatus as recited in claim 55, wherein said logic is further configured to divide a digital image into a plurality of portions, and wherein at least one of said plurality of portions has a shape selected from a group of shapes comprising a rectangle and a square.

57. The apparatus as recited in claim 55, wherein said multiple stage face detection scheme further includes a pre-filter stage wherein said logic is configured to output said set of initial candidate portions selected from said plurality of portions based on at least one feature.

58. The apparatus as recited in claim 57, wherein said feature includes at least one feature selected from a group of features comprising a Haar-like feature, an extended feature, a mirror invariant feature, and a variance feature.

59. The apparatus as recited in claim 57, wherein as part of said pre-filter stage said logic includes a linear filter based on a weak learner.

60. The apparatus as recited in claim 55, wherein said boosting chain is trained using face images, non-face images, and weak classifiers.

61. The apparatus as recited in claim 60, wherein to provide said boosting chain said logic includes a plurality of boosting nodes and a plurality of boot strap functions arranged in an alternating order within said boosting chain, and wherein said boosting chain is trained using boosting classifiers corresponding to said boosting nodes, and wherein at least one sample weight associated with one of said boot strap functions is adjusted based on at least one classification error of a weak classifier associated with a previous boosting node.

62. The apparatus as recited in claim 55, wherein said boosting chain is operatively arranged in a hierarchical chain structure.

63. The apparatus as recited in claim 55, wherein said boosting filter stage includes an LSVM optimization configured to determine a global maximum.

64. The apparatus as recited in claim 55, wherein as part of said post-filter stage said logic is further configured to perform at least one process selected from a group of processes that includes a lighting correction process, a histogram equalization process a color filter process, and an SVM filter process.

65. The apparatus as recited in claim 55, wherein said logic is further configured to output information associated with at least said final candidate portion, said information at least identifying said final candidate portion.

66. The apparatus as recited in claim 65, wherein said information includes rotation data associated with at least said final candidate portion.

67. The apparatus as recited in claim 55, wherein said logic is further cooperatively configured to implement at least one feature-based algorithm that uses at least one feature selected from a group of features including at least one Haar-like feature, at least one extended feature, at least one mirror invariant feature, and at least one variance features.

68. The apparatus as recited in claim 55, wherein said logic is further operatively configured to perform in-plane estimation that detects an orientation of said face image data, and up-right face detection based on said in-plane estimation, wherein said up-right face detection identifies out-plane rotation variations of said face image data.

69. The apparatus as recited in claim 64, wherein as part of said SVM filter process said logic is configured to reduce redundancy in a feature space associated with at least one intermediate candidate portion based on wavelet transformation of said at least one intermediate candidate portion that produces a plurality of sub-bands portions.

70. The apparatus as recited in claim 69, wherein said logic is further configured to selectively crop at four sub-band portions.