ENHANCING A DIGITAL IMAGE

• US 20180122054A1
• Filed: 10/02/2017
• Published: 05/03/2018
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Baek, Jongmin
• Granted Patent: US 10,290,086 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20028 Bilateral filtering
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/168 involving transform domain methods

Abstract

One or more embodiments of an image enhancement system enable a computing device to generate an enhanced digital image. In particular, a computing device can enhance a digital image including, for example, a photograph of a whiteboard, document, chalkboard, or other object having a uniform background. The computing device can determine modifications to apply to the digital image by minimizing an energy heuristic that both causes pixels of the digital image to change to a uniform color (e.g., white) and preserves gradients from the digital image. The computing device can further generate an enhanced digital image by applying the determined modifications to the digital image.

Claims

1. A method comprising:

accessing an input image comprising a first background associated with a first foreground and a second background associated with a second foreground, wherein the second background overlaps an area of the first background; and

creating an enhanced output image based on modifying the input image, wherein modifying the input image comprises;

changing pixels of the first background and pixels of the second background to cause the first background and the second background to be a uniform color; and

generating an outline of the second background based on identifying a gradient between the first background and the second background.

2. The method of claim 1, wherein:

the first background of the input image corresponds to a whiteboard within the input image; and

the second background of the input image corresponds to an object that overlaps the whiteboard.

3. The method of claim 2, wherein:

the object corresponding to the second background is a paper object; and

the outline of the second background corresponds to a perimeter of the paper object.

4. The method of claim 1, further comprising preserving first content within the first foreground of the input image.

5. The method of claim 4, wherein preserving the first content within the first foreground of the input image comprises:

minimizing a first energy heuristic corresponding to the first background and the first foreground; and

maintaining gradients of foreground pixels of the first foreground based on minimizing the first energy heuristic.

6. The method of claim 5, further comprising preserving second content within the second foreground of the input image.

7. The method of claim 6, wherein preserving the second content within the second foreground of the input image comprises:

minimizing a second energy heuristic corresponding to the second background and the second foreground; and

maintaining gradients of foreground pixels of the second foreground based on minimizing the second energy heuristic.

8. The method of claim 1, wherein identifying the gradient between the first background and the second background is based on the first background being a different color than the second background.

9. The method of claim 1, wherein generating the outline of the second background comprises generating a line along the gradient between the first background and the second background.

10. A system comprising:

at least one processor; and

at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to;

access an input image comprising a first background associated with a first foreground and a second background associated with a second foreground, wherein the second background overlaps an area of the first background; and

create an enhanced output image based on modifying the input image, wherein modifying the input image further causes the system to;

change pixels of the first background and pixels of the second background to cause the first background and the second background to be a uniform color; and

generate an outline of the second background based on identifying a gradient between the first background and the second background.

11. The system of claim 10, wherein generating the outline of the second background comprises generating a line along the gradient between the first background and the second background.

12. The system of claim 11, wherein identifying the gradient between the first background and the second background is based on the first background being a different color than the second background.

13. The system of claim 12, wherein changing the pixels of the first background comprises minimizing a first energy heuristic with respect to the first background and the first foreground.

14. The system of claim 13, wherein changing the pixels of the second background comprises minimizing a second energy heuristic with respect to the second background and the second foreground.

15. The system of claim 14, wherein the first energy heuristic and the second energy heuristic comprise a background parameter that causes the pixels of the first background and the pixels of the second background to change to the uniform color.

16. The system of claim 15, wherein the uniform color of the first background and the second background is white.

17. A non-transitory computer readable medium storing instructions thereon that, when executed by at least one processor, cause a computer system to:

access an input image comprising a first background associated with a first foreground and a second background associated with a second foreground, wherein the second background overlaps an area of the first background; and

create an enhanced output image based on modifying the input image, wherein modifying the input image further causes the computer system to;

change pixels of the first background and pixels of the second background to cause the first background and the second background to be a uniform color; and

generate an outline of the second background based on identifying a gradient between the first background and the second background.

18. The non-transitory computer readable medium of claim 17, wherein:

the first background of the input image corresponds to a whiteboard within the input image;

the second background of the input image corresponds to an object that overlaps the whiteboard; and

the outline of the second background corresponds to a perimeter of the paper object.

19. The non-transitory computer readable medium of claim 17, further comprising instructions that, when executed by the at least one processor, cause the computer system to preserve first content within the first foreground of the input image.

20. The non-transitory computer readable medium of claim 19, further comprising instructions that, when executed by the at least one processor, cause the computer system to preserve second content within the second foreground of the input image.