Face detection in digital images

• US 7,218,759 B1
• Filed: 12/08/1999
• Issued: 05/15/2007
• Est. Priority Date: 06/10/1998
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Lennon, Alison Joan, Ho, Edwin, Bradley, Andrew Peter
• Primary Examiner(s): DESIRE, GREGORY M
• Application Number:
  US09/457,003
• Time in Patent Office: 2,715 Days
• Field of Search: 348/64, 348/135, 348169-174, 348/369, 382/103, 382/118, 382/164, 382/165, 382/190, 382/199, 382/203, 396/153, 396/165, 396/166
• US Class Current: 382/118
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A method and apparatus for the detection of faces in a digital image captured under a variety of lighting conditions are provided. Rather than subjecting an entire image to computationally intensive face detection analysis, the image is subjected to computationally simply analysis to identify candidate pixels likely to be of skin colour. Only those pixels having such color are subject to computationally intensive face detection analysis. The pixels are selected by first selecting a color distribution model based on an image capture condition provided with the color digial image. Then, the color of the pixels is tested using the distribution model to determine those pixels having predominantly skin color. Preferably, the distribution model is selected from among a number of different distribution models with each of the distribution models associated with a different set of lighting conditions. In this way, a face detection process may accommodate a variety of lighting conditions associated with the capture of an image using a computationally simple analysis.

Claims

1. A method of detecting a face in a color digital image formed of a plurality of pixels, said method comprising the steps of:

selecting a color distribution model from a plurality of color distribution models, the selection being dependent on at least one image capture condition provided with the color digital image;

testing the color of the plurality of pixels to determine those pixels having predominantly skin color, the testing utilizing the selected color distribution model; and

executing a facial feature analysis based on the pixels determined as having predominantly skin color in the testing step.

2. A method according to claim 1, wherein the at least one image capture condition is acquired at a time the color digital image is captured.

3. A method according to claim 2, wherein the color digital image is encoded according to a predetermined format and the at least one image capture condition is represented as meta-data associated with the predetermined format.

4. A method according to claim 1 wherein the at least one image capture condition comprises lighting conditions at a time the color digital image was captured.

5. A method according to claim 1, wherein said testing step comprises the sub-step, preceding the testing, of dividing the color digital image into a plurality of regions, each region comprising a plurality of pixels,

wherein the testing is performed on pixels within each region to determine those ones of the plurality of regions that are predominantly skin color, and said executing step comprises performing the facial feature analysis on only those regions determined to be predominantly of skin color.

6. A method according to claim 5, wherein one of the plurality of regions is determined to be predominantly skin color if more than a predetermined percentage of the total number of pixels in the one region are classified as being skin color.

7. A method according to claim 5, wherein the plurality of regions are geometrically divided from the color digital image.

8. A method according to claim 5, wherein the plurality of regions are formed of pixels having substantially homogenous color.

9. A method according to claim 8, wherein the plurality of regions are formed using a region growing method based upon color differences.

10. A method according to claim 5, wherein said executing step is independent of face color.

11. A method according to claim 1, wherein the selected color distribution model has been generated using previously sampled facial image data.

12. A method according to claim 11, wherein the color distribution model is generated for a particular image capture device.

13. A method according to claim 11, wherein separate color distribution models are generated for different image capture conditions.

14. A method according to claim 13, wherein the at least one image capture condition comprises lighting conditions at a time the color digital image was captured and separate color models are generated for different lighting conditions at a time the previously sampled facial image data was captured.

15. A method according to claim 14, wherein separate color distribution models are generated for groups of images taken with a flash and images taken without a flash.

16. A method according to claim 14, wherein separate color distribution models are generated for groups of images taken indoors and images taken outdoors.

17. A method according to claim 11, wherein each color distribution model is represented as a frequency histogram of color representation vectors.

18. A method according to claim 17, wherein each of the plurality of pixels is classified as being skin color if the frequency of the color representation vector corresponding thereto exceeds a predetermined threshold frequency.

19. A method according to claim 11, wherein each color distribution model is represented as a probability distribution of color representation vectors.

20. A method according to claim 19, wherein each of the plurality of pixels is classified as being skin color if the probability of the color representation vector corresponding thereto exceeds a predetermined probability threshold.

21. A method according to claim 11, wherein each color distribution model is represented as a binary map of color representation vectors.

22. A method according to claim 17, 19 or 21, wherein the color representation vectors are derived from perceptual color space values of predetermined skin-color pixels in the previously sampled facial image data.

23. A method according to claim 17, 19 or 21, wherein color representation vectors contain chromatic color values derived from those RGB values of predetermined skin-color pixels in the previously sampled facial image data.

24. A method according to claim 21, wherein the binary map comprises a percentage of the skin color pixels that were identified in the previously sampled facial image data.

25. A method according to claim 24, wherein one of the plurality of pixels is classified as being skin color if the color representation vector corresponding thereto occurs within the binary map.

26. A method according to claim 1, wherein said testing step for determining whether a pixel has predominantly skin color, the threshold selected using the at least one image capture condition.

27. An apparatus for detecting a face in a color digital image formed of a plurality of pixels, said apparatus comprising:

means for selecting a color distribution model from a plurality of color distribution models, the selection being dependent on at least one image capture condition provided with the color digital image;

means for testing the color of the plurality of pixels to determine those pixels having predominantly skin color, said testing means utilizing the selected color distribution model; and

means for executing a facial feature analysis based on the pixels determined as having predominantly skin color by the means for testing.

28. An apparatus according to claim 27, wherein the at least one image capture condition is acquired at a time the color digital image is captured.

29. An apparatus according to claim 28, wherein the color digital image is encoded according to a predetermined format and the at least one image capture condition is represented as meta-data associated with the predetermined format.

30. An apparatus according to claim 27, wherein the at least one image capture condition comprises lighting conditions at a time the color digital image was captured.

31. An apparatus according to claim 27, wherein said means for testing comprises means for dividing the color digital image into a plurality of regions, each region comprising a plurality of pixels,

wherein said means for testing operates on pixels within each region to determine those ones of the plurality of regions that are predominantly skin color and said means for executing causes the further facial feature analysis to be performed on only those regions determined to be predominantly of skin color.

32. A computer readable medium incorporating a computer program product for detecting a face in a color digital image formed of a plurality of pixels, said computer program product comprising:

code for selecting a color distribution model from a plurality of color distribution models, the selection being dependent on at least one image capture condition provided with the color digital image;

code for testing the color of the plurality of pixels to determine those pixels having predominantly skin color, the testing utilizing the selected color distribution model; and

code for executing facial feature analysis based on the pixels determined as having predominantly skin color by the code for testing.

33. A computer readable medium according to claim 32, wherein the at least one image capture condition is acquired at a time the color digital image is captured.

34. A computer readable medium according to claim 33, wherein the color digital image is encoded according to a predetermined format and the at least one image capture condition is represented as meta-data associated with the predetermined format.

35. A computer readable medium according to claim 32, wherein the at least one image capture condition comprises lighting conditions at a time the color digital image was captured.

36. A computer readable medium according to claim 32, wherein said code for testing comprises code for dividing the color digital image into a plurality of regions, each region comprising a plurality of pixels,

wherein said code for testing operates on pixels within each region to determine those ones of the plurality of regions that are predominantly skin color, and said code for executing causes the facial feature analysis to be performed on only those regions determined to be predominantly of skin color.

37. A method of detecting a face in a color digital image formed of a plurality of pixels, and method comprising the step of:

testing the color of the plurality of pixels to determine those pixels having predominantly skin color, the testing utilizing at least one threshold dependent on the least one image capture condition provided with the color digital image, said image capture condition being expressed as metadata and not depending on a brightness of the digital image data; and

executing a facial feature analysis based on the pixels determined as having predominantly skin color in the testing step.

38. A method of detecting a face in a color digital image formed of a plurality of pixels, said method comprising the step of:

selecting a threshold from a plurality of thresholds, the selection being dependent on at least one image capture condition provided with the color digital image, said image capture condition being expressed as metadata;

testing the color of the plurality of pixels to determine those pixels having predominantly skin color, the testing utilizing the selected threshold; and

executing a facial feature analysis based on the pixels determined as having predominantly skin color in the testing step.

39. A method according to claim 38, wherein the at least one image capture condition comprises lighting conditions at a time the color digital image was captured.

40. A method according to claim 38, wherein said threshold comprises a color distribution model selected from a plurality of color distribution, models, the selected color distribution model having been generated using previously sampled facial image data.

41. A computer readable medium having a computer program recorded thereon and executable to make a computer detect a face in a color digital image formed of a plurality of pixels, said program comprising:

code for selecting a threshold from a plurality of thresholds, the selection being dependent on at least one image capture condition provided with the color digital image, said image capture condition being expressed as metadata;

code for testing the color of the plurality of pixels to determine those pixels having predominantly skin color, the testing utilizing the selected threshold; and

code for executing a fascial feature analysis based on the pixels determined as having predominantly skin color in the testing step.