Skin-tone Filtering

• US 20110317917A1
• Filed: 06/29/2010
• Published: 12/29/2011
• Est. Priority Date: 06/29/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Free, Robert Mikio
• Granted Patent: US 8,824,747 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, are disclosed relating to skin-tone filtering for reducing the impact of lighting conditions, while providing a low-computation solution for effective face detection. In one aspect, methods include sampling a digital image frame from among a series of digital image frames. The methods further include analyzing pixels within the sampled digital image frame to determine whether pixels in the sampled digital image frame have a hue, independent of lightness, that is within a range of hues corresponding to human skin tone. Further, the methods include deciding whether the sampled digital image frame includes a depiction of human skin based on a result of the analyzing.

Claims

1. A method performed by an image processor, the method comprising:

sampling a digital image frame from among a series of digital image frames;

analyzing pixels within the sampled digital image frame to determine whether pixels in the sampled digital image frame have a hue, independent of lightness, that is within a range of hues corresponding to human skin tone; and

deciding whether the sampled digital image frame includes a depiction of human skin based on a result of the analyzing.

2. The method of claim 1, wherein the range of hues corresponding to human skin tone is determined in a hue space where a red hue has a value of 0, a green hue has a value of 120 degrees and a blue hue has a value of 240 degrees, the method further comprising:

centering the range of hues corresponding to human skin tone on a value of 25 degrees associated with skin tone orange.

3. The method of claim 2 further comprising extending the range of hues corresponding to human skin tone over 40 degrees.

4. The method of claim 1, wherein said analyzing includes:

determining respective distances from hues of pixels in the sampled digital image frame to a hue corresponding to skin-tone orange; and

generating an orange-distance filtered frame including the determined distances and corresponding to the sampled digital image frame.

5. The method of claim 4, wherein if said analyzing is performed in a hue space defined by an angular coordinate such that red corresponds to zero, green corresponds to 120 degrees and blue is 240 degrees, then said determining the respective distances includes subtracting from hue-values of pixels a value of 25 degrees corresponding to skin-tone orange.

6. The method of claim 4, wherein if said analyzing is performed in a two-dimensional (2D) hue space, then said determining the respective distances includes calculating 2D Cartesian distances from hue-coordinates of pixels to a location in 2D color space corresponding to skin-tone orange.

7. The method of claim 4, wherein said deciding comprises segmenting a skin-tone orange portion of the orange-distance filtered frame having distances to skin-tone orange less than a threshold distance to represent the depiction of human skin.

8. The method of claim 7 further comprising:

obtaining the series of digital image frames from an image capture device associated with the image processor;

determining a change in area of the skin-tone orange portion corresponding to the sampled frame relative to a previously sampled frame to represent likely back-and-forth movement of a face in front of the image capture device; and

determining a change in location of the skin-tone orange portion corresponding to the sampled frame relative to a previously sampled frame to represent likely lateral movement of the face in front of the image capture device.

9. The method of claim 8 further comprising detecting a face within the skin-tone orange portion corresponding to the sampled frame when the determined changes in area and location are less than respective predetermined variations.

10. The method of claim 7 further comprising:

accessing shape classifiers generated for detecting shapes corresponding to faces within skin-tone orange portions of orange-distance filtered images; and

detecting a face within the skin-tone orange portion of the orange-distance filtered frame corresponding to the sampled digital image frame by using the accessed shape classifiers.

11. The method of claim 10 further comprising:

generating the shape classifiers by a classification application, said generating comprising;

retrieving, from a shape library, images of shapes and shape combinations corresponding to a face depicted in a given color space, wherein the shapes corresponding to the face include at least an eye, a nose a mouth, and wherein the shape combinations corresponding to the face include at least eyes, eyes and a nose, a nose and a mouth;

producing orange-distance filtered versions of the retrieved images of shapes and shape combinations; and

assembling the orange-distance filtered images of shapes and shape combinations into the generated shape classifiers; and

storing the generated shape classifiers by the classification application to be accessed by image processors for detecting faces within skin-tone orange portions of orange-distance filtered images.

12. An appliance comprising:

a data storage device configured to store rules for acknowledging a detected face;

an image capture device configured to acquire color frames;

one or more data processors configured to perform operations comprising;

apply an orange-distance filter to a frame acquired by the image capture device for segmenting a skin-tone orange portion;

determine respective changes in an area and a location of the skin-tone orange portion of the acquired frame relative to a previously acquired frame;

infer, based on the determined changes, a presence of a face substantially at rest when the frame was acquired;

detect a face corresponding to the skin-tone orange portion of the acquired frame in response to the inference; and

acknowledge the detected face in accordance with a predetermined rule stored on the data storage device of the appliance.

13. The appliance of claim 12, wherein to apply the orange-distance filter the one or more processors are further configured to perform operations comprising:

determine respective distances from hues of pixels in the acquired frame to a hue corresponding to skin-tone orange;

generate an orange-distance filtered frame including the determined distances and corresponding to the acquired frame; and

segment a portion of the orange-distance filtered frame having distances to skin-tone orange less than a threshold distance to represent a depiction of a face.

14. The appliance of claim 13, wherein if the application of the orange distance filter is performed in a hue space defined by an angular coordinate such that red corresponds to zero, green corresponds to 120 degrees and blue is 240 degrees, then to determine the distances the one or more processors are further configured to perform operations comprising:

subtract from hue-values of pixels a value of 25 degrees corresponding to skin-tone orange.

15. The appliance of claim 13, wherein if the application of the orange distance filter is performed in a two-dimensional (2D) hue space, then to determine the distances the one or more processors are further configured to perform operations comprising:

calculate 2D Cartesian distances from hue-coordinates of pixels to a location in 2D color space corresponding to skin-tone orange.

16. The appliance of claim 12, wherein to acknowledge the detected face the one or more processors are further configured to perform operations comprising:

turn on a display of the appliance located on a same side as the image capture device, wherein the display had been off prior to the face detection.

17. The appliance of claim 12, wherein to acknowledge the detected face the one or more processors are further configured to perform operations comprising:

turn on a transceiver of the appliance which was turned off prior to the face detection.

18. A system comprising:

a storage system configured to store shape classifiers used for face detection;

one or more data processors configured to perform operations comprising;

retrieve, from a shape library stored on the storage system, images of shapes and shape combinations corresponding to a face depicted in a given color space, wherein the shapes corresponding to the face include at least an eye, a nose a mouth, and wherein the shape combinations corresponding to the face include at least eyes, eyes and a nose, a nose and a mouth;

produce orange-distance filtered versions of the retrieved images of shapes and shape combinations; and

generate shape classifiers for detecting shapes corresponding to faces within skin-tone orange portions of orange-distance filtered images of shapes and shape combinations; and

the storage system further configured to store the generated shape classifiers to be provided upon requests of image processing devices that connect to the system.

19. The system of claim 18, wherein to produce the orange-distance filtered versions of the retrieved images of shapes and shape combinations the one or more processors are further configured to perform operations comprising:

determine respective distances from hues of pixels in each retrieved image to a hue corresponding to skin-tone orange; and

generate an orange-distance filtered image including the determined distances for each corresponding retrieved image.

20. The system of claim 18, wherein the one or more processors are further configured to perform operations comprising:

maintain rules for using the generated shape classifiers for detecting faces within skin-tone orange portions of orange-distance filtered images; and

updating the maintained rules based on feedback collected from the image processing devices that have used the generated shape classifiers and maintained rules.