ENHANCING A DIGITAL IMAGE

• US 20190220959A1
• Filed: 03/28/2019
• Published: 07/18/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Baek, Jongmin
• Granted Patent: US 10,643,314 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/168 involving transform domain methods

Abstract

One or more embodiments of an image enhancement system enable a computing device to generate an enhanced digital image. In particular, a computing device can enhance a digital image including, for example, a photograph of a whiteboard, document, chalkboard, or other object having a uniform background. The computing device can determine modifications to apply to the digital image by minimizing an energy heuristic that both causes pixels of the digital image to change to a uniform color (e.g., white) and preserves gradients from the digital image. The computing device can further generate an enhanced digital image by applying the determined modifications to the digital image.

Claims

1. A method comprising:

accessing an input image depicting an object comprising foreground content and a background, the background comprising a non-shadow portion and a shadow portion; and

generating, by at least one processor, an enhanced output image based on modifying the input image, wherein modifying the input image comprises;

identifying the non-shadow portion and the shadow portion of the background;

modifying the non-shadow portion and the shadow portion of the background to be a uniform color to create a uniform background; and

maintaining the foreground content on the uniform background.

2. The method of claim 1, wherein identifying the shadow portion and the non-shadow portion of the background comprises identifying gradient values for pixels within the input image.

3. The method of claim 2, wherein:

identifying the shadow portion of the background comprises identifying at least a first group of pixels having gradient values within a first range of gradient values; and

identifying the non-shadow portion of the background comprises identifying at least a second group of pixels having gradient values within a second range of gradient values.

4. The method of claim 3, wherein modifying the non-shadow portion and the shadow portion of the background to be the uniform color to create the uniform background comprises:

changing the first group of pixels having gradient values within the first range to the uniform color; and

changing the second group of pixels having gradient values within the second range to the uniform color.

5. The method of claim 3, wherein:

the first range of gradient values indicates a first non-gradient portion of the input image;

the second range of gradient values indicates a second non-gradient portion of the input image; and

the first range of gradient values is different from the second range of gradient values.

6. The method of claim 3, wherein the first range of gradient values and the second range of gradient values are based on a background parameter within an energy heuristic that defines a difference between gradient values corresponding to non-gradient portions.

7. The method of claim 1, wherein the uniform color used to create the uniform background is white.

8. The method of claim 1, wherein generating an enhanced output image further comprises preserving the foreground content of the input image without modification.

9. The method of claim 1, wherein preserving the foreground content of the input image comprises:

minimizing an energy heuristic corresponding to the background and the foreground content; and

maintaining gradients of foreground pixels of the foreground content based on minimizing the energy heuristic.

10. A system comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to;

access an input image depicting an object comprising foreground content and a background, the background comprising a non-shadow portion and a shadow portion; and

generating an enhanced output image based on modifying the input image, wherein modifying the input image comprises;

identifying the non-shadow portion and the shadow portion of the background;

modifying the non-shadow portion and the shadow portion of the background to be a uniform color to create a uniform background; and

maintaining the foreground content on the uniform background.

11. The system of claim 10, wherein:

identifying the shadow portion of the background comprises identifying a first group of pixels having gradient values within a first range of gradient values; and

identifying the non-shadow portion of the background comprises identifying a second group of pixels having gradient values within a second range of gradient values.

12. The system of claim 11, wherein modifying the non-shadow portion and the shadow portion of the background to be the uniform color to create the uniform background comprises:

changing the first group of pixels having gradient values within the first range to the uniform color; and

changing the second group of pixels having gradient values within the second range to the uniform color.

13. The system of claim 11, wherein:

the first range of gradient values indicates a first non-gradient portion of the input image;

the second range of gradient values indicates a second non-gradient portion of the input image; and

the first range of gradient values is different from the second range of gradient values.

14. The system of claim 11, wherein the first range of gradient values and the second range of gradient values are based on a background parameter within an energy heuristic that defines a difference between gradient values corresponding to non-gradient portions.

15. The system of claim 10, wherein generating an enhanced output image further comprises preserving the foreground content of the input image without modification.

16. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computer system to:

access an input image depicting an object comprising foreground content and a background, the background comprising a non-shadow portion and a shadow portion; and

generating an enhanced output image based on modifying the input image, wherein modifying the input image comprises;

identifying the non-shadow portion and the shadow portion of the background;

modifying the non-shadow portion and the shadow portion of the background to be a uniform color to create a uniform background; and

maintaining the foreground content on the uniform background.

17. The non-transitory computer readable medium of claim 16, wherein:

identifying the shadow portion of the background comprises identifying a first group of pixels having gradient values within a first range of gradient values; and

identifying the non-shadow portion of the background comprises identifying a second group of pixels having gradient values within a second range of gradient values.

18. The non-transitory computer readable medium of claim 17, wherein modifying the non-shadow portion and the shadow portion of the background to be the uniform color to create the uniform background comprises:

changing the first group of pixels having gradient values within the first range to the uniform color; and

changing the second group of pixels having gradient values within the second range to the uniform color.

19. The non-transitory computer readable medium of claim 18, wherein:

the first range of gradient values indicates a first non-gradient portion of the input image;

the second range of gradient values indicates a second non-gradient portion of the input image; and

the first range of gradient values is different from the second range of gradient values.

20. The non-transitory computer readable medium of claim 16, wherein generating an enhanced output image further comprises preserving the foreground content of the input image without modification, wherein preserving the foreground content of the input image comprises:

minimizing an energy heuristic corresponding to the background and the foreground content; and

maintaining gradients of foreground pixels of the foreground content based on minimizing the energy heuristic.