METHOD AND APPARATUS FOR ENHANCING A DIGITAL PHOTOGRAPHIC IMAGE

• US 20140010448A1
• Filed: 07/08/2013
• Published: 01/09/2014
• Est. Priority Date: 07/06/2012
• Status: Active Grant

• Current Assignee: Jolly Seven, Series 70 of Allied Security Trust I (Allied Security Trust)
• Original Assignee: Yissum Research Development Company of the Hebrew University of Jerusalem Ltd. (Hebrew University of Jerusalem)
• Inventors: Lischinski, Daniel, Werman, Michael, Kaufman, Liad
• Granted Patent: US 9,256,927 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/008 Local, e.g. shadow enhancement
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

A method and apparatus for enhancing a digital photographic image uses a combination of global and content-specific operations. Image data is analyzed to detect one or more content items in the image to be separately enhanced. Then a content-specific enhancement operation is performed on data representing at least one detected content item, separately from the remainder of the image data. Several such content-specific operations may be performed for different types of content item. The content-specific enhancement operation comprises manipulating data, and the manipulation applies different adjustments to different pixels in the detected content item and the determination of the adjustments is based solely on analysis of the image data. Then a different enhancement operation is performed on at least the remaining image data excluding said one or more detected content items.

Claims

1. A method for enhancing a digital photographic image, the method comprising processing data representing the image, the processing comprising:

analysing the data to detect one or more content items in the image to be separately enhanced;

performing a content-specific enhancement operation on data representing at least one detected content item separately from the remainder of the image data, wherein the content-specific enhancement operation comprises manipulating data, and wherein the manipulation applies different adjustments to different pixels in the detected content item and the determination of the adjustments is based solely on analysis of the image data; and

performing a different enhancement operation on at least the remaining image data excluding said one or more detected content items.

2. The method of claim 1 wherein the adjustment is calculated to depend on the probability of the pixel being part of the content item.

3. The method of claim 1 wherein the analysing comprises using at least two different detection processes each of which is configured to detect a different type of content item in the data representing the image.

4. The method of claim 3 wherein at least two different types of content item are detected during said analysing and wherein said performing a content-specific comprising performing a different content-specific enhancement operation on each of said different types of content item.

5. The method of claim 1 wherein during the content-specific enhancement operation adjustments are applied to only certain pixels forming part of a content item.

6. The method of claim 3 wherein the types of content item include one or more of:

side lit objects, sky, water, vegetation, and under exposed objects.

7. The method of claim 1 wherein the analysing comprises detecting under-exposed objects in the image and the content-specific enhancement operation comprises adjusting the luminance of pixels by an amount which depends on the saliency of the object.

8. The method of claim 1 wherein the manipulation is based on analysis of only the data representing the detected content item.

9. The method of claim 1 wherein the content-specific enhancement operation comprises:

determining a luminance histogram for the pixels in the detected content item,

identifying two maxima meeting predetermined criteria are present in the histogram, and

adjusting the intensities of pixels in the region of the darker of the two maxima towards the brighter of the two maxima.

10. The method of claim 9 comprising decomposing luminance data for the content item into a base layer and a detail layer and wherein the adjustment is applied to the luminance values of the detail layer.

11. The method of claim 1 wherein the content-specific enhancement operation comprises:

multiplying the luminance of each pixel in the content item by a factor calculated to shift the luminance towards a selected percentile.

12. The method of claim 1 in which the analysing comprises determining the probability that pixels in the image data are part of an image of sky and performing the content-specific enhancement operation on pixels having a sky probability above a predetermined threshold.

13. The method of claim 12 in which the image enhancement operation comprises identifying one or both of blue sky components and cloud components among the pixels having a sky probability above a predetermined threshold and performing one or both of:

adjusting pixels of blue sky components towards a target blue color, and

adjusting pixels of cloud components towards a target white colour.

14. The method of claim 1 wherein the analysing comprises determining the saliency of content items in the image and detecting content items in the image that are under-exposed, and the performing a content-specific enhancement operation comprises adjusting the luminance of pixels in under-exposed content items by an amount dependent on the saliency of the content item.

15. The method of claim 1 further comprising performing a detail enhancement operation on the data representing the image, the detail enhancement operation comprising:

determining the probability of the pixels not being the detected one or more content items and

adjusting the luminance of each pixel by an amount that depends on the probability.

16. A method for enhancing a digital photographic image, the method comprising processing data representing the image, the processing comprising:

using at least two different detection processes to detect one or more content items in the image to be separately enhanced, each detection process being configured to detect a different type of content item in the data representing the image;

for each detected content item performing a content-specific enhancement operation on data representing the content item separately from the remainder of the image data, wherein the content-specific enhancement operation comprises manipulating data, and wherein the manipulation applies different adjustments to different pixels in the detected content item and the determination of the adjustments is based solely on analysis of the image data.

17. The method of claim 16 further comprising performing a different enhancement operation on at least the remaining image data excluding said one or more detected content items.

18. Apparatus for enhancing a digital photographic image comprising data store and a processor, the processor being configured to:

receive data representing the image from the data store,

analyze the data to detect one or more content items in the image to be separately enhanced;

perform a content-specific enhancement operation on data representing at least one detected content item separately from the remainder of the image data, wherein the content-specific enhancement operation comprises manipulating data, and wherein the manipulation applies different adjustments to different pixels in the detected content item and the determination of the adjustments is based solely on analysis of the image data; and

perform a different enhancement operation on at least the remaining image data excluding said one or more detected content items.

19. A computer program product comprising a computer readable storage medium having computer readable program embodied therewith, the computer readable program comprising:

computer readable program configured to receive data representing the image from the data store;

computer readable program configured to analyse the data to detect one or more content items in the image to be separately enhanced;

computer readable program configured to perform a content-specific enhancement operation on data representing at least one detected content item separately from the remainder of the image data, wherein the content-specific enhancement operation comprises manipulating data, and wherein the manipulation applies different adjustments to different pixels in the detected content item and the determination of the adjustments is based solely on analysis of the image data; and

computer readable program configured to perform a different enhancement operation on at least the remaining image data excluding said one or more detected content items.