ENHANCING A DIGITAL IMAGE

• US 20170178299A1
• Filed: 12/16/2015
• Published: 06/22/2017
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Baek, Jongmin
• Granted Patent: US 9,799,106 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/168 involving transform domain methods

Abstract

One or more embodiments of an image enhancement system enable a computing device to generate an enhanced digital image. In particular, a computing device can enhance a digital image including, for example, a photograph of a whiteboard, document, chalkboard, or other object having a uniform background. The computing device can determine modifications to apply to the digital image by minimizing an energy heuristic that both causes pixels of the digital image to change to a uniform color (e.g., white) and preserves gradients from the digital image. The computing device can further generate an enhanced digital image by applying the determined modifications to the digital image.

Claims

1. A method for enhancing digital images, comprising:

accessing an input image comprising a background and a foreground;

determining, by at least one processor, modifications to enhance the input image by minimizing an energy heuristic that both causes pixels of the input image to change to a uniform color and preserves gradients from the input image; and

generating, by the at least one processor, an output image with an enhanced foreground by applying the determined modifications to the input image.

2. The method as recited in claim 1, wherein the input image comprises a digital photograph of a whiteboard with written text.

3. The method as recited in claim 1, wherein generating the output image with the enhanced foreground by applying the determined modifications to the input image comprises applying white color to non-gradient portions of the input image.

4. The method as recited in claim 1, wherein the energy heuristic comprises:

a background parameter that causes the background of the input image to change to the uniform color; and

one or more foreground parameters that preserve gradients from the input image.

5. The method as recited in claim 4, wherein the one or more foreground parameters that preserve gradients from the input image comprise:

a first foreground parameter that preserves horizontal gradients from the input image; and

a second foreground parameter that preserves vertical gradients from the input image.

6. The method as recited in claim 1, further comprising converting the input image to grayscale.

7. The method as recited in claim 6, wherein converting the input image to grayscale comprises:

identifying one or more minimum red green blue (RGB) pixel values across the input image; and

normalizing the one or more minimum RGB pixel values to zero.

8. The method as recited in claim 1, wherein generating the output image with the enhanced foreground further comprises blending color from the input image to the output image.

9. The method as recited in claim 8, wherein blending color from the original color image to the output image comprises adding color to the enhanced foreground based on fractional values of gradients from the input image.

10. The method as recited in claim 1, wherein determining the modifications to enhance the input image comprises:

converting the energy heuristic to a frequency domain; and

minimizing the energy heuristic by solving for one or more zero values for the converted energy heuristic.

11. The method as recited in claim 10, wherein converting the energy heuristic to the frequency domain comprises performing a Fast Fourier Transform (FFT).

12. The method as recited in claim 1, further comprising filtering the output image to fill in hollow portions of the enhanced foreground.

15. The method as recited in claim 1, wherein the energy heuristic comprises a screened Poisson equation.

13. The method as recited in claim 13, wherein filtering the output image to fill in hollow portions of the enhanced foreground comprises applying a cross-bilateral filter to the output image with respect to the input image.

14. The method as recited in claim 13, further comprising:

generating, based on the determined modifications to enhance the input image, a binarization map comprising pixel values to apply to the input image; and

wherein generating the output image with the enhanced foreground by applying the determined modifications to the input image comprises masking the binarization map to the input image.

16. A method for enhancing digital images, comprising:

accessing a digital image comprising a photograph of a whiteboard and textual content;

determining, by at least one processor, modifications to enhance the digital image by causing pixels of the digital image to change to a white pixel value while simultaneously preserving gradients based on color changes within the digital image;

generating, by the at least one processor, an enhanced digital image having enhanced textual content by applying the determined modifications to the digital image.

17. The method as recited in claim 16, wherein determining the modification to enhance the digital image comprises:

calculating a minimum value of a sum of squared parameters, the squared parameters comprising;

a background parameter comprising an indication that the enhanced digital image be entirely white; and

a foreground parameter comprising a difference between gradient values of the digital image and the enhanced digital image.

18. The method as recited in claim 17, further comprising:

converting the sum of squared parameters to a frequency domain; and

wherein calculating the minimum value of the sum of squared parameters comprises solving for one or more zero values for the converted sum of squared parameters.

19. A computing device comprising:

at least one processor; and

a non-transitory computer-readable medium storing instructions thereon that, when executed by the at least one processor, cause the computing device to;

access an input image comprising a background and a foreground;

determine modifications to enhance the input image by minimizing an energy heuristic that both causes pixels of the input image to change to a uniform color and preserves gradients from the input image; and

generate an output image with an enhanced foreground by applying the determined modifications to the input image.

20. The computing device of claim 19, wherein the instructions further cause the computing device to:

generate a trimap based on an output of the energy heuristic, the trimap classifying portions of the output image as foreground, background, and unknown;

for each pixel within the output image, determine a pixel value based on neighboring foreground and background portions of the output image; and

generate, based on the determined pixel value for each pixel within the output image, a binarization map;

wherein generating the output image with the enhanced foreground by applying the determined modifications to the input image comprises masking the generated binarization map against the input image.