High dynamic range image generation and rendering

• US 8,606,009 B2
• Filed: 02/04/2010
• Issued: 12/10/2013
• Est. Priority Date: 02/04/2010
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Shijun
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/700,292
• Publication Number:
  US 20110188744A1
• Time in Patent Office: 1,405 Days
• Field of Search: 382/169, 382/168, 382/274, 345/589, 345/590, 345/690
• US Class Current: 382/169
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

Techniques and tools for high dynamic range (HDR) image rendering and generation. An HDR image generating system performs motion analysis on a set of lower dynamic range (LDR) images and derives relative exposure levels for the images based on information obtained in the motion analysis. These relative exposure levels are used when integrating the LDR images to form an HDR image. An HDR image rendering system tone maps sample values in an HDR image to a respective lower dynamic range value, and calculates local contrast values. Residual signals are derived based on local contrast, and sample values for an LDR image are calculated based on the tone-mapped sample values and the residual signals. User preference information can be used during various stages of HDR image generation or rendering.

Claims

1. One or more computer-readable memory or storage devices storing instructions that, when executed by a computing device, perform a method of rendering a high dynamic range digital image as a lower dynamic range digital image, the method comprising:

tone mapping each of plural sample values in a high dynamic range digital image to a respective lower dynamic range value;

calculating a local contrast value for each of the tone-mapped sample values;

deriving a residual signal for each of the tone-mapped sample values, the deriving based at least in part on the respective local contrast value; and

calculating sample values for a lower dynamic range digital image based at least in part on the tone-mapped sample values and the respective residual signals.

2. The memory or storage devices of claim 1, wherein the method further comprises:

preprocessing the high dynamic range digital image prior to the tone mapping.

3. The memory or storage devices of claim 1 wherein the tone-mapping comprises:

generating a global tone mapping lookup table; and

identifying a lower dynamic range value for each of plural sample values in the high dynamic range digital image;

wherein the identifying comprises looking up each of the plural sample values in the global tone mapping lookup table.

4. The memory or storage devices of claim 3 wherein the global tone mapping lookup table is generated based on an adaptive gamma function.

5. The memory or storage devices of claim 1 wherein the deriving the residual signal for each of the tone-mapped sample values comprises multiplying each tone-mapped sample value by the respective local contrast value.

6. The memory or storage devices of claim 1, wherein the method further comprises:

applying filtering to one or more of the residual signals.

7. The memory or storage devices of claim 1, wherein the method further comprises:

applying scaling to one or more of the residual signals wherein the scaling comprises applying a scaling factor that is a function of the tone-mapped sample value that corresponds to the respective residual signal.

8. The memory or storage devices of claim 1 wherein the sample values for the lower dynamic range digital image are each calculated by combining a tone-mapped sample value, a residual signal and a high-frequency dithering signal.

9. The memory or storage devices of claim 1, wherein the method further comprises:

identifying a lower dynamic range value for each of plural sample values in a lower resolution preview version of the high dynamic range digital image;

tone mapping the plural sample values in the lower resolution preview version to the respective identified lower dynamic range value;

calculating a local contrast value for each of the tone-mapped sample values in the lower resolution preview version;

deriving a residual signal for each of the tone-mapped sample values in the lower resolution preview version, the deriving based at least in part on the respective local contrast value;

calculating sample values for a preview image based at least in part on the tone-mapped sample values in the lower resolution preview version and the respective residual signals.

13. The memory or storage devices of claim 1, wherein the method further comprises:

applying one or more modified tone mapping parameters to one or more copies of the high dynamic range digital image to obtain modified versions of the lower dynamic range digital image; and

producing an animation effect by displaying the lower dynamic range digital image along with the modified versions of the lower dynamic range digital image in a sequence.

10. In a computer system, a method of rendering a high dynamic range image based on user preference information, the method comprising:

providing one or more user preference controls for setting user preferences for high dynamic range image rendering parameters;

receiving user preference information associated with a user from the user preference controls, the user preference information corresponding to high dynamic range image rendering parameters for a high dynamic range digital image file containing a high dynamic range digital image; and

rendering the high dynamic range digital image based at least in part on the user preference information;

wherein the rendering comprises;

calculating a local contrast value for each of plural tone-mapped sample values;

deriving a residual signal for each of the plural tone-mapped sample values, the deriving being based at least in part on the respective local contrast value; and

calculating sample values for a lower dynamic range digital image based at least in part on the plural tone-mapped sample values and the respective residual signals.

11. The method of claim 10 further comprising:

storing the user preference information associated with the user as metadata in the high dynamic range digital image file.

12. The method of claim 11 wherein storing the user preference information as metadata facilitates preserving original high dynamic range digital image information in the file.

14. The method of claim 10 further comprising:

receiving user preference information associated with a second user from the user preference controls; and

storing the user preference information associated with the second user as metadata in the high dynamic range digital image file along with the user preference information associated with the first user.

15. A high dynamic range digital image rendering system comprising:

a lookup table containing a plurality of high dynamic range (HDR) digital image sample values and corresponding lower dynamic range (LDR) sample values;

a tone mapper that obtains LDR sample values from the lookup table that correspond to sample values of an HDR digital image being rendered to create tone-mapped sample values;

a residual signal extractor that;

calculates local contrast values for the tone-mapped sample values, and

derives residual signals for the tone-mapped sample values by multiplying the tone-mapped sample values by the corresponding local contrast values;

an LDR image generator that generates samples for an output LDR image based on the tone mapped sample values and the residual signals; and

at least one processor that communicates with the tone mapper, residual signal extractor, and LDR image generator to perform the respective functions.

16. The system of claim 15, further comprising a preprocessor that performs digital signal processing operations on the HDR digital image being rendered.

17. The system of claim 15, wherein the LDR sample values obtained by the tone mapper are stored in an LDR image buffer.

18. The system of claim 15, wherein the lookup table LDR sample values are generated using an adaptive gamma function.

19. The system of claim 15, further comprising a residual signal filterer that filters residual signals derived by the residual signal extractor.

20. The system of claim 19, wherein the residual signal filterer applies a 3×

3 boxcar filter to remove outlier samples.

21. A computer-implemented method of rendering a high dynamic range digital image as a lower dynamic range digital image, the method comprising:

tone mapping each of plural sample values in a high dynamic range digital image to a respective lower dynamic range value;

calculating a local contrast value for each of the tone-mapped sample values;

deriving a residual signal for each of the tone-mapped sample values, the deriving based at least in part on the respective local contrast value; and

calculating sample values for a lower dynamic range digital image based at least in part on the tone-mapped sample values and the respective residual signals.

22. The method of claim 21 wherein the tone-mapping comprises:

generating a global tone mapping lookup table; and

identifying a lower dynamic range value for each of plural sample values in the high dynamic range digital image;

wherein the identifying comprises looking up each of the plural sample values in the global tone mapping lookup table.

23. The method of claim 21, wherein the deriving the residual signal for each of the tone-mapped sample values comprises multiplying each tone-mapped sample value by the respective local contrast value.

24. The method of claim 21, wherein the method further comprises:

applying scaling to one or more of the residual signals wherein the scaling comprises applying a scaling factor that is a function of the tone-mapped sample value that corresponds to the respective residual signal.