Speedup of face detection in digital images

• US 7,190,829 B2
• Filed: 06/30/2003
• Issued: 03/13/2007
• Est. Priority Date: 06/30/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Zhang, Lei, Zhang, Hong-Jiang
• Primary Examiner(s): Miriam; Daniel
• Application Number:
  US10/610,245
• Publication Number:
  US 20040264744A1
• Time in Patent Office: 1,352 Days
• Field of Search: 382/118, 382/165, 382/181, 382/190, 382/260
• US Class Current: 382/165
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

Improved methods and apparatuses are provided for use in face detection. The methods and apparatuses significantly reduce the number of candidate windows within a digital image that need to be processed using more complex and/or time consuming face detection algorithms. The improved methods and apparatuses include a skin color filter and an adaptive non-face skipping scheme.

Claims

1. A method for use in face detection, the method comprising:

establishing a plurality of initial candidate windows within a digital image, said digital image having a plurality of pixels represented by color space information;

for each initial candidate window, examining said color space information for each of at least a portion of said pixels within said initial candidate window and classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel; and

establishing a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows;

wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes classifying “

gray”

pixels as skin color pixels; and

wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

2. The method as recited in claim 1, wherein said color space information includes at least chrominance information.

3. The method as recited in claim 2, wherein said color space information includes luminance information.

4. The method as recited in claim 3, wherein said color space information includes Y, Cr and Cb information.

5. The method as recited in claim 1, wherein said color space information includes red (R), green (G) and blue (B) color information.

6. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

$\mathrm{skin}=\{\begin{array}{cc}\mathrm{true},& \mathrm{if}\end{array}$

⁢

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) false otherwise ,

wherein Cr and Gb represent chrominance information associated with said pixel being classified.

7. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

$\mathrm{non}$

- ⁢

skin = { false , if ⁢

⁢

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) true otherwise ,

wherein Cr and Cb represent chrominance information associated with said pixel being classified.

8. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes converting RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r−

0.3313*g=0.5000*b ,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

9. The method as recited in claim 8, wherein convening said RGB color space information further includes converting said RGB color space information to corresponding luminance information Y as follows:

Y=0.2990*r0.5870*g=0.1140*b.

10. The method as recited in claim 1, wherein establishing said is plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

11. The method as recited in claim 10, wherein said threshold value is determined as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

12. The method as recited in claim 11, wherein α

between 0.3 and 0.7.

13. The method as recited in claim 12, wherein α

is 0.5.

14. The method as recited in claim 1, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

15. The method as recited in claim 14, wherein said threshold value is determined as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

16. The method as recited in claim 15, wherein α

between 0.3 and 0.7.

17. The method as recited in claim 16, wherein a is 0.5.

18. The method as recited in claim 1, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

19. The method as recited in claim 18, wherein said intermediate representation includes an integral image.

20. The method as recited in claim 1, further comprising:

processing said subsequent candidate windows using a face detector.

21. The method as recited in claim 20, wherein processing said subsequent candidate windows using a face detector further includes:

applying an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.

22. A computer-readable medium having computer-implementable instructions for causing one or more processing units to perform acts comprising:

establishing a plurality of initial candidate windows within a digital image, said digital image having a plurality of pixels represented by color space information;

for each initial candidate window, examining said color space information for each of at least a portion of said pixels within said initial candidate window;

classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel; and

establishing a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows;

wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes classifying “

gray”

pixels as skin color pixels; and

wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

23. The computer-readable medium as recited in claim 22, wherein said color space information includes at least chrominance information.

24. The computer-readable medium as recited in claim 23, wherein said color space information includes luminance information.

25. The computer-readable medium as recited in claim 24, wherein said color space information includes Y, Cr and Cb information.

26. The computer-readable medium as recited in claim 22, wherein said color space information includes red (R), green (G) and blue (B) color information.

27. The computer-readable medium as recited in claim 22, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

$\mathrm{skin}=\{\begin{array}{cc}\mathrm{true},& \mathrm{if}\end{array}$

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) false otherwise ,

wherein Cr and Cb represent chrominance information associated with said pixel being classified.

28. The computer-readable medium as recited in claim 22, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

$\mathrm{non}$

- ⁢

skin = { false , if ⁢

⁢

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) true otherwise ,

wherein Cr and Cb represent chrominance information associated with said pixel being classified.

29. The computer-readable medium as recited in claim 22, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes converting RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r 0.3313*g=0.5000*b,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

30. The computer-readable medium as recited in claim 29, wherein converting said RUB color space information further includes converting said RUB color space information to corresponding luminance information Y as follows:

Y=0.2990*r=0.5870*g=0.1140*b.

31. The computer-readable medium as recited in claim 22, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

32. The computer-readable medium as recited in claim 31, wherein said threshold value is determined as T=α

*W*H wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said is initial candidate window, and α

is a weight parameter greater than zero.

33. The computer-readable medium as recited in claim 32, wherein a between 0.3 and 0.7.

34. The computer-readable medium as recited in claim 33, wherein α

is 0.5.

35. The computer-readable medium as recited in claim 22, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

36. The computer-readable medium as recited in claim 35, wherein said threshold value is determined as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

37. The computer-readable medium as recited in claim 36, wherein α

between 0.3 and 0.7.

38. The computer-readable medium as recited in claim 37, wherein α

0.5.

39. The computer-readable medium as recited in claim 22, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

40. The computer-readable medium as recited in claim 39, wherein said intermediate representation includes an integral image.

41. The computer-readable medium as recited in claim 22, further comprising processing said subsequent candidate windows using a face detector.

42. The computer-readable medium as recited in claim 41, wherein processing said subsequent candidate windows using a face detector further includes:

applying an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.

43. An apparatus for use in face detection, the apparatus comprising:

memory suitable for storing a digital image having a plurality of image pixels represented by color space information; and

logic operatively coupled to said memory and configured to;

establish a plurality of initial candidate windows using said digital image, wherein each initial candidate window including pixels associated with at least a portion of said image pixels,

for each initial candidate window, examine said color space information for each of at feast a portion of said pixels within said initial candidate window and classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel, and

establish a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows;

wherein said logic is further configured to classify “

gray”

pixels as skin color pixels; and

wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

44. The apparatus as recited in claim 43, wherein said color space information includes at least chrominance information and luminance information.

45. The apparatus as recited in claim 43, wherein said color space information includes red (R), green (G) and blue (B) color information.

46. The apparatus as recited in claim 43, wherein logic is configured to classify each of said examined pixels as either said skin color pixel or said non-skin color pixel based on

$\mathrm{skin}=\{\begin{array}{cc}\mathrm{true},& \mathrm{if}\end{array}$

⁢

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) false otherwise ,

wherein Cr and Cb represent chrominance information associated with said pixel being classified.

47. The apparatus as recited in claim 43, wherein logic is configured to classify each of said examined pixels as either said skin color pixel or said non-skin color pixel based on

$\mathrm{non}$

- ⁢

skin = { false , if ⁢

⁢

( Cr - 1.2 * Cb + 12 ≥

0 ) ⁢

⁢

and ⁢

⁢

( Cr + 0.675 * Cb + 2.5 ≥

0 ) true otherwise ,

wherein Cr and Cb represent chrominance information associated with said pixel being classified.

48. The apparatus as recited in claim 43, wherein said logic is further configured to convert RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r−

0.3313*g=0.5000*b,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

49. The apparatus as recited in claim 43, wherein said logic is further configured to:

for each of said initial candidate windows, compare a total number of said skin color classified pixels with a threshold value, and

based on said comparison, identify said initial candidate window as one of said subsequent candidate windows.

50. The apparatus as recited in claim 49, wherein said threshold value is determined by said logic as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero, and wherein α

is between about 0.3 and about 0.7.

51. The apparatus as recited in claim 43, wherein said logic is further configured to:

for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value, and

based on said comparison identify said initial candidate window as one of said subsequent candidate windows.

52. The apparatus as recited in claim 51, wherein said threshold value is determined by said logic as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero, and wherein α

is between about 0.3 and about 0.7.

53. The apparatus as recited in claim 43, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

54. The apparatus as recited in claim 53, wherein said intermediate representation includes an integral image.

55. The apparatus as recited in claim 43, further comprising:

face detector logic operatively coupled to said memory and configured to process said subsequent candidate windows.

56. The apparatus as recited in claim 55, wherein said face detector logic is configured to:

apply an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.