Digital image processing using face detection and skin tone information

• US 9,712,743 B2
• Filed: 12/05/2016
• Issued: 07/18/2017
• Est. Priority Date: 06/26/2003
• Status: Expired due to Term

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Steinberg, Eran, Barcovschi, Igor, Prilutsky, Yury, Bigioi, Petronel
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/369,698
• Publication Number:
  US 20170085785A1
• Time in Patent Office: 225 Days
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/675 comprising setting of focusing regions
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A technique for processing a digital image uses face detection to achieve one or more desired image processing parameters. A group of pixels is identified that corresponds to a face image within the digital image. A skin tone is detected for the face image by determining one or more default color or tonal values, or combinations thereof, for the group of pixels. Values of one or more parameters are adjusted for the group of pixels that correspond to the face image based on the detected skin tone.

Claims

1. A method of processing a digital image using face detection, the method comprising:

identifying a first group of pixels that correspond to a first face region within a digital image;

determining one or more first initial color values for pixels in the first group of pixels;

based on the one or more first initial color values for the pixels in the first group of pixels, determining a first skin tone for the first face region;

identifying a second group of pixels that correspond to a second face region within the digital image;

determining one or more second initial color values for pixels in the second group of pixels;

based on the one or more second initial color values for the pixels in the second group of pixels, determining a second skin tone for the second face region;

determining whether the first skin tone is lighter than the second skin tone; and

in response to determining that the first skin tone is lighter than the second skin tone;

based on, at least in part, the first skin tone and the second skin tone, determining an initial fill-flash and applying the initial fill-flash to the first face region;

based on, at least in part, the first skin tone and the second skin tone, increasing the fill-flash and applying the increased fill-flash to the second face region.

2. The method of claim 1, wherein the applying the initial fill-flash to the first face region comprises applying the initial fill-flash to the first face region in an amount depending on the first skin tone.

3. The method of claim 1, wherein applying the increased fill-flash to the second face region comprises applying the increased fill-flash to the second face region in an amount depending on the second skin tone.

4. The method of claim 1, wherein applying the initial fill-flash to the first face region comprises adjusting a luminance of the first face region depending on the first skin tone and the initial fill-flash.

5. The method of claim 1, wherein applying the increased fill-flash to the second face region comprises adjusting a luminance of the second face region depending on the second skin tone and the increased fill-flash.

6. The method of claim 1, wherein the initial fill-flash is further determined based on, at least in part, a background color surrounding the first face region.

7. The method of claim 1, wherein the increased fill-flash is further determined based on, at least in part, a background color surrounding the second face region.

8. An image acquisition apparatus including an imaging optical system, a digital image detector and storage medium, and having stored therein program instructions which, when executed by one or more processors cause the one or more processors, to perform:

identifying a first group of pixels that correspond to a first face region within a digital image;

determining one or more first initial color values for pixels in the first group of pixels;

based on the one or more first initial color values for the pixels in the first group of pixels, determining a first skin tone for the first face region;

identifying a second group of pixels that correspond to a second face region within the digital image;

determining one or more second initial color values for pixels in the second group of pixels;

based on the one or more second initial color values for the pixels in the second group of pixels, determining a second skin tone for the second face region;

determining whether the first skin tone is lighter than the second skin tone; and

in response to determining that the first skin tone is lighter than the second skin tone;

based on, at least in part, the first skin tone and the second skin tone, determining an initial fill-flash and applying the initial fill-flash to the first face region;

based on, at least in part, the first skin tone and the second skin tone, increasing the fill-flash and applying the increased fill-flash to the second face region.

9. The image acquisition apparatus of claim 8, wherein the applying the initial fill-flash for the first face region comprises applying the initial fill-flash to the first face region in an amount depending on the first skin tone.

10. The image acquisition apparatus of claim 8, wherein applying the increased fill-flash to the second face region comprises applying the increased fill-flash to the second face region in an amount depending on the second skin tone.

11. The image acquisition apparatus of claim 8, wherein applying the initial fill-flash to the first face region comprises adjusting a luminance of the first face region depending on the first skin tone and the initial fill-flash.

12. The image acquisition apparatus of claim 8, wherein applying the increased fill-flash to the second face region comprises adjusting a luminance of the second face region depending on the second skin tone and the increased fill-flash.

13. The image acquisition apparatus of claim 8, wherein the initial fill-flash is further determined based on, at least in part, a background color surrounding the first face region.

14. The image acquisition apparatus of claim 8, wherein the increased fill-flash is further determined based on, at least in part, a background color surrounding the second face region.

15. One or more non-transitory computer-readable storage media storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform:

identifying a first group of pixels that correspond to a first face region within a digital image;

determining one or more first initial color values for pixels in the first group of pixels;

based on the one or more first initial color values for the pixels in the first group of pixels, determining a first skin tone for the first face region;

identifying a second group of pixels that correspond to a second face region within the digital image;

determining one or more second initial color values for pixels in the second group of pixels;

based on the one or more second initial color values for the pixels in the second group of pixels, determining a second skin tone for the second face region;

determining whether the first skin tone is lighter than the second skin tone; and

in response to determining that the first skin tone is lighter than the second skin tone;

based on, at least in part, the first skin tone and the second skin tone, determining an initial fill-flash and applying the initial fill-flash to the first face region;

based on, at least in part, the first skin tone and the second skin tone, increasing the fill-flash and applying the increased fill-flash to the second face region.

16. The one or more non-transitory computer-readable storage media of claim 15, wherein the applying the initial fill-flash to the first face region comprises applying the initial fill-flash to the first face region in an amount depending on the first skin tone.

17. The one or more non-transitory computer-readable storage media of claim 15, wherein applying the increased fill-flash to the second face region comprises applying the increased fill-flash to the second face region in an amount depending on the second skin tone.

18. The one or more non-transitory computer-readable storage media of claim 15, wherein applying the initial fill-flash to the first face region comprises adjusting a luminance of the first face region depending on the first skin tone and the initial fill-flash.

19. The one or more non-transitory computer-readable storage media of claim 15, wherein the initial fill-flash is further determined based on, at least in part, a background color surrounding the first face region.

20. The one or more non-transitory computer-readable storage media of claim 15, wherein the increased fill-flash is further determined based on, at least in part, a background color surrounding the second face region.