HIGH DYNAMIC RANGE AND TONE MAPPING IMAGING TECHNIQUES

• US 20150078661A1
• Filed: 08/26/2014
• Published: 03/19/2015
• Est. Priority Date: 08/26/2013
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: MATTHEWS, IAIN, GRANADOS, MIGUEL, TENA, JOSE RAFAEL, AYDIN, TUNC OZAN, LALONDE, JEAN FRANCOIS, THEOBALT, CHRISTIAN
• Granted Patent: US 9,275,445 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

Algorithms for improving the performance of conventional tone mapping operators (TMO) by calculating both a contrast waste score and a contrast loss score for a first tone-mapped image produced by the TMO. The two contrast scores can be used to optimize the performance of the TMO by reducing noise and improving contrast. Algorithms for generating an HDR image by converting non-linear color space images into linear color space format, aligning the images to a reference, de-ghosting the aligned images if necessary, and merging the aligned (and potentially de-ghosted) images to create an HDR image. The merging can be performed with exposure fusion, HDR reconstruction, or other suitable techniques.

Claims

1. A computer-implemented method for improving the quality of an image, comprising:

receiving an input image;

utilizing a tone mapping operator (TMO) having one or more operational parameters to tone map the input image and create a first tone-mapped image;

generating a contrast waste score for the first tone-mapped image;

generating a contrast loss score for the first tone-mapped image;

with a computer, automatically utilizing the contrast waste score and the contrast loss score to tune one or more of the operational parameters of the TMO; and

utilizing the TMO with the one or more tuned operational parameters to generate a second tone-mapped image from the input image.

2. A method as defined in claim 1, wherein the contrast waste score is representative of the amount of contrast in the first tone-mapped image that is used to represent noise.

3. A method as defined in claim 2, wherein the contrast waste score is generated by calculating a luminance difference between two adjacent pixels in the first tone-mapped image and multiplying that difference by the probability that the two adjacent pixels measure the same luminance.

4. A method as defined in claim 3, wherein the luminance difference is a normalized perceivable luminance difference.

5. A method as defined in claim 3, wherein the probability is determined from a noise model of a camera used to capture the input image.

6. A method as defined in claim 1, wherein the contrast loss score is representative of the amount of contrast in the input image that is not mapped into the first tone-mapped image.

7. A method as defined in claim 6, wherein the contrast loss score is generated by comparing the loss of perceivable luminance differences with respect to a standard tone mapping procedure.

8. A method as defined in claim 1, wherein the contrast waste score and the contrast loss score are tuned by generating a confidence parameter from those two scores.

9. A method as defined in claim 8, wherein the confidence parameter can be automatically adjusted.

10. A method as defined in claim 8, wherein the confidence parameter can be manually adjusted by a human operator.

11. A computer-implemented method for generating an HDR image from a plurality of input images, comprising:

if one or more of the input images are not in a linear color space format, converting those to linear color space format;

aligning the linear color space images to a reference; and

merging the aligned images to create an HDR image.

12. A method as defined in claim 11, wherein the images are merged by fusing the images.

13. A method as defined in claim 11, wherein the images are merged by reconstructing the images.

14. A method as defined in claim 13, wherein the reconstructing includes transforming the images to a common scale by applying the inverse of the camera response.

15. A method as defined in claim 14, wherein the images on the common scale can be averaged on a pixel-by-pixel basis, with the under-exposed and over-exposed pixel values excluded from the average.

16. A method as defined in claim 13, further including tone mapping the reconstructed image.

17. A method as defined in claim 16, further including, for the case where the input images included RAW images, performing a sharpening operation.

18. A method as defined in claim 16, further including preserving skin tone of human subjects in the images while tone mapping.

19. A method as defined in claim 11, further including de-ghosting the aligned images before the images are merged.

20. A method as defined in claim 11, further including, for the case of HDR images to be prepared of subjects that include objects that are moving faster than humans can move, capturing the input images at points in time separated sufficiently to include at least one input image with the fast-moving object in the scene and at least one input image with the fast-moving object not in the scene.

21. A computer-implemented method for generating an HDR image from a plurality of input images, comprising:

if one or more of the input images are not in a linear color space format, converting those to linear color space format;

for the case where the input images included linear color space images, performing a sharpening operation;

aligning the linear color space images to a reference; and

merging the aligned images to create an HDR image, the merging including reconstructing the images by transforming the images to a common scale by applying the inverse of the camera response.

22. A method as defined in claim 21, wherein the images on the common scale can be averaged on a pixel-by-pixel basis, with the under-exposed and over-exposed pixel values excluded from the average.