Speedup of face detection in digital images

• US 20040264744A1
• Filed: 06/30/2003
• Published: 12/30/2004
• Est. Priority Date: 06/30/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Mingjing, Zhang, Lei, Zhang, Hong-Jiang
• Granted Patent: US 7,190,829 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

Improved methods and apparatuses are provided for use in face detection. The methods and apparatuses significantly reduce the number of candidate windows within a digital image that need to be processed using more complex and/or time consuming face detection algorithms. The improved methods and apparatuses include a skin color filter and an adaptive non-face skipping scheme.

Claims

1. A method for use in face detection, the method comprising:

establishing a plurality of initial candidate windows within a digital image, said digital image having a plurality of pixels represented by color space information;

for each initial candidate window, examining said color space information for each of at least a portion of said pixels within said initial candidate window and classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel; and

establishing a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows.

2. The method as recited in claim 1, wherein said color space information includes at least chrominance information.

3. The method as recited in claim 2, wherein said color space information includes luminance information.

4. The method as recited in claim 3, wherein said color space information includes Y, Cr and Cb information.

5. The method as recited in claim 1, wherein said color space information includes red (R), green (G) and blue (B) color information.

6. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

7. The method as recited in claim 1, wherein classifying each of said 1 examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

8. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes converting RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r−

0.3313*g+0.5000*b,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

9. The method as recited in claim 8, wherein converting said RGB color space information further includes converting said RGB color space information to corresponding luminance information Y as follows:

Y=0.2990*r+0.5870*g+0.1140*b.

10. The method as recited in claim 1, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes classifying “

gray”

pixels as skin color pixels.

11. The method as recited in claim 10, wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

12. The method as recited in claim 11, wherein said associated chrominance information Cr and Cb are equal to zero.

13. The method as recited in claim 1, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

14. The method as recited in claim 13, wherein said threshold value is determined as T=α

*W*H wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

15. The method as recited in claim 14, wherein α

between 0.3 and 0.7.

16. The method as recited in claim 15, wherein α

is 0.5.

17. The method as recited in claim 1, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

18. The method as recited in claim 17, wherein said threshold value is determined as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

19. The method as recited in claim 18, wherein α

between 0.3 and 0.7.

20. The method as recited in claim 19, wherein α

is 0.5.

21. The method as recited in claim 1, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

22. The method as recited in claim 21, wherein said intermediate representation includes an integral image.

23. The method as recited in claim 1, further comprising:

processing said subsequent candidate windows using a face detector.

24. The method as recited in claim 23, wherein processing said subsequent candidate windows using a face detector further includes:

applying an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.

25. A computer-readable medium having computer-implementable instructions for causing one or more processing units to perform acts comprising:

establishing a plurality of initial candidate windows within a digital image, said digital image having a plurality of pixels represented by color space information;

for each initial candidate window, examining said color space information for each of at least a portion of said pixels within said initial candidate window;

classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel; and

establishing a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows.

26. The computer-readable medium as recited in claim 25, wherein said color space information includes at least chrominance information.

27. The computer-readable medium as recited in claim 26, wherein said color space information includes luminance information.

28. The computer-readable medium as recited in claim 27, wherein said color space information includes Y, Cr and Cb information.

29. The computer-readable medium as recited in claim 25, wherein said color space information includes red (R), green (G) and blue (B) color information.

30. The computer-readable medium as recited in claim 25, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

31. The computer-readable medium as recited in claim 25, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes applying logic based on

32. The computer-readable medium as recited in claim 25, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes converting RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r−

0.3313*g+0.5000*b,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

33. The computer-readable medium as recited in claim 32, wherein converting said RGB color space information further includes converting said RGB color space information to corresponding luminance information Y as follows:

Y=0.2990*r+0.5870*g+0.1140*b.

34. The computer-readable medium as recited in claim 25, wherein classifying each of said examined pixels as either said skin color pixel or said non-skin color pixel further includes classifying “

gray”

pixels as skin color pixels.

35. The computer-readable medium as recited in claim 34, wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

36. The computer-readable medium as recited in claim 35, wherein said associated chrominance information Cr and Cb are equal to zero.

37. The computer-readable medium as recited in claim 25, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

38. The computer-readable medium as recited in claim 37, wherein said threshold value is determined as T=α

*W*H wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

39. The computer-readable medium as recited in claim 38, wherein α

between 0.3 and 0.7.

40. The computer-readable medium as recited in claim 39, wherein α

is 0.5.

41. The computer-readable medium as recited in claim 25, wherein establishing said plurality of subsequent candidate windows further includes, for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value and based on said comparison identifying said initial candidate window as one of said subsequent candidate windows.

42. The computer-readable medium as recited in claim 41, wherein said threshold value is determined as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero.

43. The computer-readable medium as recited in claim 42, wherein α

between 0.3 and 0.7.

44. The computer-readable medium as recited in claim 43, wherein α

is 0.5.

45. The computer-readable medium as recited in claim 25, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

46. The computer-readable medium as recited in claim 45, wherein said intermediate representation includes an integral image.

47. The computer-readable medium as recited in claim 25, further comprising processing said subsequent candidate windows using a face detector.

48. The computer-readable medium as recited in claim 47, wherein processing said subsequent candidate windows using a face detector further includes:

applying an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.

49. An apparatus for use in face detection, the apparatus comprising:

memory suitable for storing a digital image having a plurality of image pixels represented by color space information; and

logic operatively coupled to said memory and configured to;

establish a plurality of initial candidate windows using said digital image, wherein each initial candidate window including pixels associated with at least a portion of said image pixels,

for each initial candidate window, examine said color space information for each of at least a portion of said pixels within said initial candidate window and classifying each of said examined pixels as being either a skin color pixel or a non-skin color pixel, and

establish a plurality of subsequent candidate windows that includes at least a portion of said initial candidate windows based on said classified pixels within each of said initial candidate windows.

50. The apparatus as recited in claim 49, wherein said color space information includes at least chrominance information and luminance information.

51. The apparatus as recited in claim 49, wherein said color space information includes red (R), green (G) and blue (B) color information.

52. The apparatus as recited in claim 49, wherein logic is configured to classify each of said examined pixels as either said skin color pixel or said non-skin color pixel based on

53. The apparatus as recited in claim 49, wherein logic is configured to classify each of said examined pixels as either said skin color pixel or said non-skin color pixel based on

54. The apparatus as recited in claim 49, wherein said logic is further configured to convert RGB color space information to at least corresponding chrominance information Cr and Cb as follows:

Cr=0.5000*r−

0.4187*g−

0.0813*b and Cb=−

0.1687*r−

0.3313*g+0.5000*b,

wherein r is red color information, g is green color information, and b is blue color information for said pixel being examined.

55. The apparatus as recited in claim 49, wherein said logic is further configured to classify “

gray”

pixels as skin color pixels.

56. The apparatus as recited in claim 55, wherein said “

gray”

pixels have associated chrominance information Cr and Cb substantially equal to zero.

57. The apparatus as recited in claim 56, wherein said associated chrominance information Cr and Cb are equal to zero.

58. The apparatus as recited in claim 49, wherein said logic is further configured to:

for each of said initial candidate windows, compare a total number of said skin color classified pixels with a threshold value, and

based on said comparison, identify said initial candidate window as one of said subsequent candidate windows.

59. The apparatus as recited in claim 58, wherein said threshold value is determined by said logic as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and a is a weight parameter greater than zero, and wherein α

is between about 0.3 and about 0.7.

60. The apparatus as recited in claim 49, wherein said logic is further configured to:

for each of said initial candidate windows, comparing a total number of said non-skin color classified pixels with a threshold value, and

based on said comparison identify said initial candidate window as one of said subsequent candidate windows.

61. The apparatus as recited in claim 60, wherein said threshold value is determined by said logic as T=α

*W*H, wherein T is said threshold value, W is a width of said initial candidate window, H is a height of said initial candidate window, and α

is a weight parameter greater than zero, and wherein α

is between about 0.3 and about 0.7.

62. The apparatus as recited in claim 49, wherein at least one of said initial candidate windows includes an intermediate representation of at least a portion of said digital image.

63. The apparatus as recited in claim 62, wherein said intermediate representation includes an integral image.

64. The apparatus as recited in claim 49, further comprising:

face detector logic operatively coupled to said memory and configured to process said subsequent candidate windows.

65. The apparatus as recited in claim 64, wherein said face detector logic is configured to:

apply an adaptive non-face skipping scheme to establish a plurality of further processed candidate windows that includes a subset of said plurality of subsequent candidate windows.

66. A method for use in face detection, the method comprising:

establishing a plurality of candidate windows within a digital image;

classifying at least one of said candidate windows as a non-face window;

determining a confidence score for said classified candidate window; and

based on said confidence score, selectively skipping further classification of at least one spatially neighboring candidate window.

67. The method as recited in claim 66, wherein classifying said at least one candidate window as said non-face window further includes processing said at least one candidate window using a boosting cascade.

68. The method as recited in claim 67, wherein said confidence score for said classified candidate window corresponds to at least one classifier layer within said boosting cascade.

69. The method as recited in claim 66, wherein at least a portion of said candidate windows are associated with different scales.

70. The method as recited in claim 66, wherein said at least one spatially neighboring candidate window includes an adjacent candidate window.

71. The method as recited in claim 66, wherein said at least one spatially neighboring candidate window includes an overlapping candidate window.

72. The method as recited in claim 66, wherein establishing said plurality of candidate windows further includes applying a skin color filter to each of a larger plurality of initial candidate windows.

73. A computer-readable medium having computer-implementable instructions for causing at least on processing unit to perform acts comprising:

establishing a plurality of candidate windows within a digital image;

classifying at least one of said candidate windows as a non-face window;

determining a confidence score for said classified candidate window; and

based on said confidence score, selectively skipping further classification of at least one spatially neighboring candidate window.

74. The computer-readable medium as recited in claim 73, wherein classifying said at least one candidate window as said non-face window further includes processing said at least one candidate window using a boosting cascade.

75. The computer-readable medium as recited in claim 74, wherein said confidence score for said classified candidate window corresponds to at least one classifier layer within said boosting cascade.

76. The computer-readable medium as recited in claim 73, wherein at least a portion of said candidate windows are associated with different scales.

77. The computer-readable medium as recited in claim 73, wherein said at least one spatially neighboring candidate window includes an adjacent candidate window.

78. The computer-readable medium as recited in claim 73, wherein said at least one spatially neighboring candidate window includes an overlapping candidate window.

79. The computer-readable medium as recited in claim 73, wherein establishing said plurality of candidate windows further includes applying a skin color filter to each of a larger plurality of initial candidate windows.

80. An apparatus for use in face detection, the apparatus comprising:

memory suitable for storing a digital image; and

logic operatively coupled to said memory and configured to establish a plurality of candidate windows within said digital image, classify at least one of said candidate windows as a non-face window, determine a confidence score for said classified candidate window, and based on said confidence score selectively skip further classification of at least one spatially neighboring candidate window.

81. The apparatus as recited in claim 80, wherein said logic is further configured to process said at least one candidate window using a boosting cascade.

82. The apparatus as recited in claim 81, wherein said confidence score for said classified candidate window corresponds to at least one classifier layer within said boosting cascade.

83. The apparatus as recited in claim 80, wherein at least a portion of said candidate windows are associated with different scales.

84. The apparatus as recited in claim 80, wherein said at least one spatially neighboring candidate window includes an adjacent candidate window.

85. The apparatus as recited in claim 80, wherein said at least one spatially neighboring candidate window includes an overlapping candidate window.

86. The apparatus as recited in claim 80, wherein said logic is further configured to apply a skin color filter to each of a larger plurality of initial candidate windows associated with said digital image prior to establishing said plurality of candidate windows.