Automated selective application of fill light filter

• US 9,519,957 B1
• Filed: 07/03/2014
• Issued: 12/13/2016
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kulewski, Krzysztof, Krishnaswamy, Aravind, Babacan, Sevket Derin
• Primary Examiner(s): Yeh, Eueng-Nan
• Application Number:
  US14/323,911
• Time in Patent Office: 894 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

Aspects of the subject technology relate to automatically and selectively applying a fill light filter to an image. A process includes determining an edge-preserved, smoothed version of the image, and determining a grayscale version of the image. The process also includes comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image. The process also includes applying the fill light filter to the image based on the comparison. The fill light filter is automatically adjusted based on identifying regions in the image. Selectively applying the fill light filter can reduce artifacts and noise from forming or being amplified as a result of the fill light filter.

Claims

1. A method for image processing, the method comprising:

determining an edge-preserved, smoothed version of an image;

determining a grayscale version of the image;

comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image to determine a comparison value for each pixel;

determining a scaling value for each pixel based on the respective comparison value; and

selectively applying a fill light filter to the image by scaling the applied fill light filter for each pixel based on the respective scaling value.

2. The method of claim 1, wherein determining the edge-preserved, smoothed version of the image comprises:

blurring non-edge areas of the image; and

determining a grayscale version of the at least partially blurred image.

3. The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises determining a difference between color values of each pixel of the edge-preserved, smoothed version of the image and color values of each corresponding pixel of the grayscale version of the image.

4. The method of claim 3, wherein applying the fill light filter comprises reducing the applied fill light filter to a pixel when the determined difference for the pixel is negative.

5. The method of claim 1, wherein determining the edge-preserved, smoothed version of the image comprises applying a Local Statistical Convolution (LSC) function to each pixel of the image.

6. The method of claim 1, wherein determining the grayscale version of the image comprises applying a luminance function to each pixel of the image.

7. The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises determining one or more characteristics of the image.

8. The method of claim 1, wherein comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises calculating a linear combination of the edge-preserved, smoothed version of the image and the grayscale version of the image.

9. A non-transitory machine-readable medium including instructions stored therein, which when executed by a machine, cause the machine to perform operations comprising:

determining an edge-preserved, smoothed version of an image;

determining a grayscale version of the image;

comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image to determine a comparison value for each pixel;

determining a scaling value for each pixel based on the respective comparison value; and

selectively applying a fill light filter to the image by scaling the applied fill light filter for each pixel based on the respective scaling value.

10. The non-transitory machine-readable medium of claim 9, wherein the operation of determining the edge-preserved, smoothed version of the image comprises operations for:

blurring non-edge areas of the image; and

determining a grayscale version of the at least partially blurred image.

11. The non-transitory machine-readable medium of claim 9, wherein the operation of comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for determining a difference between color values of each pixel of the edge-preserved, smoothed version of the image and color values of each corresponding pixel of the grayscale version of the image.

12. The non-transitory machine-readable medium of claim 11, wherein the operation for applying the fill light filter comprises operations for reducing the applied fill light filter to a pixel when the determined difference for the pixel is negative.

13. The non-transitory machine-readable medium of claim 9, wherein the operation for determining the edge-preserved, smoothed version of the image comprises operations for applying a Local Statistical Convolution (LSC) function to each pixel of the image.

14. The non-transitory machine-readable medium of claim 9, wherein the operation for determining the grayscale version of the image comprises operations for applying a luminance function to each pixel of the image.

15. The non-transitory machine-readable medium of claim 9, wherein the operation for comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for determining one or more characteristics of the image.

16. The non-transitory machine-readable medium of claim 9, wherein the operation for comparing each pixel of the edge-preserved, smoothed version of the image to each corresponding pixel of the grayscale version of the image comprises operations for calculating a linear combination of the edge-preserved, smoothed version of the image and the grayscale version of the image.

17. A system comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions stored therein, which, when executed by the one or more processors, cause the one or more processors to perform operation comprising;

determining a first modified version of each pixel of an image;

determining a second modified version of each pixel of the image;

determining a comparison value for each pixel of the image by comparing at least the first modified version of each pixel to the second modified version of each pixel, respectively;

determining a magnitude adjustment value for each pixel based on the respective comparison value; and

selectively applying an image filter to each pixel based on the respective magnitude adjustment value for each pixel.

18. The system of claim 17, wherein determining the first modified version of each pixel of the image cause the one or more processors to perform operations comprising applying a first image filter function to each pixel of the image, and wherein determining the second modified version of each pixel of the image comprises applying a second image filter function to each pixel of the image.

19. The system of claim 18, wherein determining a comparison value for each pixel of the image cause the one or more processors to perform operations comprising calculating a linear combination of the first image filter function and the second image filter function for each pixel of the image.

20. The system of claim 19, wherein determining the magnitude adjustment value for each pixel cause the one or more processors to perform operations comprising calculating a sigmoid function based on the linear combination.