System, method, and computer program for capturing an image with correct skin tone exposure

• US 10,558,848 B2
• Filed: 05/10/2018
• Issued: 02/11/2020
• Est. Priority Date: 10/05/2017
• Status: Expired due to Fees

• Current Assignee: Duelight LLC
• Original Assignee: Duelight LLC
• Inventors: Rivard, William Guie, Kindle, Brian J., Feder, Adam Barry
• Primary Examiner(s): Osifade, Idowu O
• Application Number:
  US15/976,756
• Publication Number:
  US 20190108387A1
• Time in Patent Office: 642 Days
• Field of Search: None
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06V 10/22 by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

A system and method are provided for capturing an image with correct skin tone exposure. In use, one or more faces are detected having threshold skin tone within a scene. Next, based on the detected one or more faces, the scene is segmented into one or more face regions and one or more non-face regions. Next, one or more images of the scene are captured. Further, in response to the capture, the one or more face regions are processed to generate a final image.

Claims

1. A device, comprising:

a non-transitory memory storing instructions;

a camera module; and

one or more processors in communication with the non-transitory memory and the camera module, wherein the one or more processors execute the instructions to;

detect one or more faces having threshold skin tone within a scene, wherein the threshold skin tone includes at least one of a predefined low intensity threshold and a predefined high intensity threshold;

based on the detected one or more faces, segment the scene into one or more face regions and one or more non-face regions;

measure an ambient lighting color balance;

correct a skin tone associated with each of the one or more faces based on the measurement;

create a texture map which selectively modifies contrast of the scene to adjust the skin tone associated with each of the one or more faces;

cause the camera module to capture one or more images of the scene; and

in response to the capture, process the one or more face regions to generate a final image.

2. The device of claim 1, wherein the threshold skin tone is a predefined average intensity or histogram median.

3. The device of claim 2, wherein the predefined average intensity or histogram median is based on the one or more face regions.

4. The device of claim 1, wherein the device is operable such that the one or more images include at least one high dynamic range (HDR) image.

5. The device of claim 1, wherein the device is operable such that the one or more images include two or more images such that a first image of the two or more images is exposed so that a median intensity of the one or more face regions defines a mid-point intensity of the first image, and a second image of the two or images is exposed so that a median intensity of the one or more non-face regions defines a mid-point intensity of the second image.

6. The device of claim 5, wherein the device is operable such that the processing includes processing the one or more non-face regions and applying an HDR effect by blending, using a strength coefficient, the first image and the second image.

7. The device of claim 1, wherein the device is operable such that the capturing is of a video stream.

8. The device of claim 1, wherein the device is operable such that the processing includes applying a local equalization on pixels within the one or more face regions.

9. The device of claim 1, wherein the device is operable such that a model is constructed of one or more individuals depicted within the scene based on a depth map and at least one visual image.

10. The device of claim 9, wherein the depth map is used to construct a three-dimensional (3D) model of the one or more faces.

11. The device of claim 9, wherein the at least one visual image includes a surface texture.

12. The device of claim 11, wherein the surface texture includes one or more features associated with each of the one or more faces.

13. The device of claim 1, wherein the device is operable such that the segmenting includes a transition region between the one or more face regions and the one or more non-face regions.

14. The device of claim 1, wherein the device is operable such that the one or more face regions includes at least one of a depth map, and a texture map.

15. The device of claim 14, wherein the device is operable such that a lighting gradient is used to modify post-processing lighting conditions on the one or more faces.

16. The device of claim 1, wherein the device is operable such that a histogram associated with the final image shows correct exposure data for the scene.

19. The device of claim 9, wherein the model is constructed based, at least in part, on the texture map.

17. A computer-implemented method, comprising:

detecting, using a processor, one or more faces having threshold skin tone within a scene, wherein the threshold skin tone includes at least one of a predefined low intensity threshold and a predefined high intensity threshold;

based on the detected one or more faces, segmenting, using the processor, the scene into one or more face regions and one or more non-face regions;

measuring, using the processor, an ambient lighting color balance;

correcting, using the processor, a skin tone associated with each of the one or more faces based on the measurement;

creating, using the processor, a texture map which selectively modifies contrast of the scene to adjust the skin tone associated with each of the one or more faces;

capturing, using a camera, one or more images of the scene; and

in response to the capturing, processing, using the processor, the one or more face regions to generate a final image.

18. A computer program product comprising computer executable instructions stored on a non-transitory computer readable medium that when executed by a processor instruct the processor to:

detect one or more faces having threshold skin tone within a scene, wherein the threshold skin tone includes at least one of a predefined low intensity threshold and a predefined high intensity threshold;

based on the detected one or more faces, segment the scene into one or more face regions and one or more non-face regions;

measure an ambient lighting color balance;

correct a skin tone associated with each of the one or more faces based on the measurement;

create a texture map which selectively modifies contrast of the scene to adjust the skin tone associated with each of the one or more faces;

cause a camera module to capture one or more images of the scene; and

in response to the capture, process the one or more face regions to generate a final image.