System and method for automatic skin tone detection in images

• US 8,406,482 B1
• Filed: 08/28/2008
• Issued: 03/26/2013
• Est. Priority Date: 08/28/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Chien, Jen-Chan, Wang, Jue
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/200,476
• Time in Patent Office: 1,671 Days
• Field of Search: 382/118, 382/115, 382/159, 382/162, 382/165, 382/283
• US Class Current: 382/118
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

Systems, methods, and computer-readable storage media for automatically detecting skin tones in an input image are disclosed. An initial skin tone mask for the image may be created dependent on a general skin tone model. An upper threshold and/or a lower threshold may be applied to the initial skin tone mask to identify pixels most likely to be skin pixels and least likely to be skin pixels, respectively. These pixels may be used to produce an image-specific skin tone model, including one or more truncated Gaussian models for skin pixels and/or non-skin pixels defined in a three-dimensional color space. The image-specific skin tone model may be applied to the image to generate a final skin tone mask. Skin tone detection may be automatically or selectively performed in conjunction with image editing or image feature identification operations to target or exclude skin pixels or non-skin pixels during execution of the operations.

Claims

1. A computer-implemented method, comprising:

accessing data representing an image comprising a plurality of pixels;

creating an initial skin tone mask for the image dependent on a general skin tone model, wherein the general skin tone model is based on a plurality of training samples, and wherein the initial skin tone mask comprises a respective value for each of the plurality of pixels indicating a probability that the pixel is a skin pixel;

applying an upper threshold to the initial skin tone mask to identify one or more of the plurality of pixels of the image most likely to be skin pixels;

applying a lower threshold to the initial skin tone mask to identify one or more pixels of the image least likely to be skin pixels;

producing an image-specific skin tone model dependent on the one or more pixels most likely to be skin pixels, and producing an image-specific non-skin model dependent on the one or more pixels least likely to be skin pixels; and

applying the image-specific skin tone model and the image-specific non-skin model to the image to generate a final skin tone mask for the image, wherein the final skin tone mask comprises a respective value indicating a degree of membership in the final skin tone mask for each of the plurality of pixels in the image, and wherein the degree of membership in the final skin tone mask is a function of a similarity in color to the one or more pixels most likely to be skin pixels and the one or more pixels least likely to be skin pixels.

2. The method of claim 1, further comprising:

performing an image editing operation on the image targeting one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels targeted are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

3. The method of claim 2, wherein the image editing operation comprises one of:

a filtering operation, a color correction operation, or a resizing operation.

4. The method of claim 1, further comprising:

performing an image editing operation on the image excluding one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels excluded are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

5. The method of claim 4, wherein the image editing operation comprises one of:

a filtering operation, a color correction operation, or a resizing operation.

6. The method of claim 1, further comprising:

performing an image identification operation identifying in the image one or more of;

a face, a human being, or an adult image;

wherein said identifying is dependent on the final skin tone mask.

7. The method of claim 1, further comprising:

modifying the general skin tone model to compensate for an inherent color bias in the general skin tone model.

8. The method of claim 1, further comprising:

modifying the image-specific skin tone model to compensate for an inherent color bias in the general skin tone model.

9. The method of claim 1, wherein colors of at least two of the one or more pixels most likely to be skin pixels belong to different clusters of probable skin tone colors.

10. The method of claim 9, wherein said applying an upper threshold comprises applying different threshold values to the different clusters of probable skin tone colors.

11. The method of claim 1, further comprising:

receiving input specifying one of a plurality of clusters of probable skin tone colors;

wherein said applying an upper threshold comprises applying a respective threshold value dependent on the specified one of the plurality of clusters.

12. The method of claim 1, wherein said creating an initial skin tone mask, said applying an upper threshold, said producing an image-specific skin tone model, and said applying the image-specific skin tone model are performed in response to receiving input indicating that a skin detection mechanism should be applied in conjunction with an image processing operation on the image.

13. The method of claim 1, wherein the image-specific skin tone model comprises one or more truncated Gaussian functions.

14. The method as recited in claim 13, wherein each of the one or more truncated Gaussian functions is defined in a three-dimensional color space that separates luminance from chrominance.

15. The method of claim 13, further comprising:

receiving input comprising a color tolerance;

adjusting a color tolerance of at least one of the one or more truncated Gaussian functions dependent on the color tolerance; and

modifying the final skin tone mask in response to adjusting the color tolerance.

16. The method of claim 1,

wherein said producing an image-specific skin tone model and said producing an image-specific non-skin model dependent on comprise determining a skin tone cluster and a non-skin cluster, respectively;

wherein the skin tone cluster comprises colors of the one or more most likely skin pixels;

and wherein the non-skin cluster comprises colors of one or more least likely skin pixels.

17. The method of claim 16, wherein each of the skin tone cluster and the non-skin cluster comprises at least one truncated Gaussian function.

18. A non-transitory computer-readable storage medium, storing program instructions computer-executable to implement:

accessing data representing an image comprising a plurality of pixels;

creating an initial skin tone mask for the image dependent on a general skin tone model, wherein the general skin tone model is based on a plurality of training samples, and wherein the initial skin tone mask comprises a respective value for each of the plurality of pixels indicating a probability that the pixel is a skin pixel;

applying an upper threshold to the initial skin tone mask to identify one or more of the plurality of pixels of the image most likely to be skin pixels;

applying a lower threshold to the initial skin tone mask to identify one or more pixels of the image least likely to be skin pixels;

producing an image-specific skin tone model dependent on the one or more pixels most likely to be skin pixels, and producing an image-specific non-skin model dependent on the one or more pixels least likely to be skin pixels; and

applying the image-specific skin tone model and the image-specific non-skin model to the image to generate a final skin tone mask for the image, wherein the final skin tone mask comprises a respective value indicating a degree of membership in the final skin tone mask for each of the plurality of pixels in the image, and wherein the degree of membership in the final skin tone mask is a function of a similarity in color to the one or more pixels most likely to be skin pixels and the one or more pixels least likely to be skin pixels.

19. The storage medium of claim 18, wherein the program instructions are further executable to implement:

performing an image editing operation on the image targeting one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels targeted are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

20. The storage medium of claim 18, wherein the program instructions are further executable to implement:

performing an image editing operation on the image excluding one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels excluded are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

21. The storage medium of claim 18, wherein the program instructions are further executable to implement:

performing an image identification operation identifying in the image one or more of;

a face, a human being, or an adult image;

wherein said identifying is dependent on the final skin tone mask.

22. The storage medium of claim 18, wherein the program instructions are further executable to implement:

modifying the general skin tone model or the image-specific skin tone model to compensate for an inherent color bias in the general skin tone model.

23. The storage medium of claim 18,

wherein colors of at least two of the one or more pixels most likely to be skin pixels belong to different clusters of probable skin tone colors;

and

wherein said applying an upper threshold comprises applying different threshold values to the different clusters of probable skin tone colors.

24. The storage medium of claim 18, wherein the image-specific skin tone model comprises one or more truncated Gaussian functions defined in a three-dimensional color space that separates luminance from chrominance.

25. The storage medium of claim 18,

wherein said producing an image-specific skin tone model and said producing an image-specific non-skin model dependent on comprise determining a skin tone cluster and a non-skin cluster, respectively, each comprising at least one truncated Gaussian function;

wherein the skin tone cluster comprises colors of the one or more most likely skin pixels; and

wherein the non-skin cluster comprises colors of one or more least likely skin pixels.

26. A system, comprising:

at least one processor; and

a memory coupled to the at least one processor, wherein the memory stores program instructions executable by the at least one processor to implement;

accessing data representing an image comprising a plurality of pixels;

creating an initial skin tone mask for the image dependent on a general skin tone model, wherein the general skin tone model is based on a plurality of training samples, and wherein the initial skin tone mask comprises a respective value for each of the plurality of pixels indicating a probability that the pixel is a skin pixel;

applying an upper threshold to the initial skin tone mask to identify one or more of the plurality of pixels of the image most likely to be skin pixels;

applying a lower threshold to the initial skin tone mask to identify one or more pixels of the image least likely to be skin pixels;

producing an image-specific skin tone model dependent on the one or more pixels most likely to be skin pixels, and producing an image-specific non-skin model dependent on the one or more pixels least likely to be skin pixels; and

applying the image-specific skin tone model and the image-specific non-skin model to the image to generate a final skin tone mask for the image, wherein the final skin tone mask comprises a respective value indicating a degree of membership in the final skin tone mask for each of the plurality of pixels in the image, and wherein the degree of membership in the final skin tone mask is a function of a similarity in color to the one or more pixels most likely to be skin pixels and the one or more pixels least likely to be skin pixels.

27. The system of claim 26, wherein the program instructions are further executable to implement:

performing an image editing operation on the image targeting one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels targeted are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

28. The system of claim 26, wherein the program instructions are further executable to implement:

performing an image editing operation on the image excluding one of skin pixels or non-skin pixels to produce a modified version of the image, wherein the skin pixels or non-skin pixels excluded are dependent on the final skin tone mask; and

rendering the modified version of the image to a display device.

29. The system of claim 26, wherein the program instructions are further executable to implement:

performing an image identification operation identifying in the image one or more of;

a face, a human being, or an adult image;

wherein said identifying is dependent on the final skin tone mask.

30. The system of claim 26, wherein the program instructions are further executable to implement:

modifying the general skin tone model or the image-specific skin tone model to compensate for an inherent color bias in the general skin tone model.

31. The system of claim 26,

wherein colors of at least two of the one or more pixels most likely to be skin pixels belong to different clusters of probable skin tone colors;

and

wherein said applying an upper threshold comprises applying different threshold values to the different clusters of probable skin tone colors.

32. The system of claim 26, wherein the image-specific skin tone model comprises one or more truncated Gaussian functions defined in a three-dimensional color space that separates luminance from chrominance.

33. The system of claim 26,

wherein said producing an image-specific skin tone model and said producing an image-specific non-skin model dependent on comprise determining a skin tone cluster and a non-skin cluster, respectively, each comprising at least one truncated Gaussian function;

wherein the skin tone cluster comprises colors of the one or more most likely skin pixels; and

wherein the non-skin cluster comprises colors of one or more least likely skin pixels.