REAL TIME TONE MAPPING OF HIGH DYNAMIC RANGE IMAGE DATA AT TIME OF PLAYBACK ON A LOWER DYNAMIC RANGE DISPLAY

• US 20200134792A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: MANDAL, Saswata, KIM, Ryan Y., Slattery, Kathleen A.
• Granted Patent: US 10,957,024 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: H04N 1/4074 using histograms
• CPC Class Codes: H04N 5/57 Control of contrast or brightness

Abstract

A converter can process image data from input HDR images in real time to compute new metadata about the brightness, contrast, color gamut and/or color volume for the image data to be displayed from each frame. Existing metadata can be ignored. The converter can combine the metadata for a current HDR frame with metadata for a plurality of immediately previous sequential frames to provide parameters for tone mapping. The converter uses these parameters, and characteristics about a lower dynamic range display which will receive output image data, to define a transfer function for converting the input HDR image data into output image data for display. The converter analyzes and tone maps HDR frames at a rate sufficient to allow output video to be generated at a desired frame rate while receiving the image data from frames of the input HDR video at an input frame rate.

Claims

1. A computer comprising:

a converter application for converting image data from a high dynamic range image to a lower dynamic range for display, the converter application comprising computer program instructions executing on the computer and configuring the computer to perform a process, comprising;

receiving image data to be displayed from a high dynamic range image;

computing metadata for the image data to be displayed, the metadata describing a luminance range of the image data;

computing tone mapping parameters for the image data based on at least the computed metadata and characteristics of a display having a lower dynamic range than the high dynamic range image;

tone mapping the received image data using the tone mapping parameters to generate output lower dynamic range image data; and

presenting the lower dynamic range image data on the display.

2. The computer of claim 1, wherein computing the metadata comprises generating a luminance histogram based on the image data to be displayed.

3. The computer of claim 2, wherein computing the metadata comprises determining reference luminance levels describing the luminance range based on the histogram.

4. The computer of claim 1, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

comparing the computed tone mapping parameters to previously computed tone mapping parameters based on metadata for a plurality of immediately previous sequential frames; and

in response to the computed tone mapping parameters being within a range of the previously computed tone mapping parameters, using the computed tone mapping parameters for the current frame as the tone mapping parameters for the image data to be displayed from the current frame.

5. The computer of claim 4, wherein the plurality of immediately previous sequential frames comprises between eight and one hundred twenty frames.

6. The computer of claim 1, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

averaging a history of previously computed metadata for a plurality of immediately previous sequential frames and the current frame; and

computing the tone mapping parameters for the current frame based on the averaged history.

7. The computer of claim 6, wherein the plurality of immediately previous sequential frames comprises between eight and one hundred twenty frames.

8. A computer-implemented process for converting image data from a high dynamic range image to a lower dynamic range for display, comprising:

receiving image data to be displayed from a high dynamic range image;

computing metadata for the image data to be displayed, the metadata describing a luminance range of the image data;

computing tone mapping parameters for the image data based on at least the computed metadata and characteristics of a display having a lower dynamic range than the high dynamic range image;

tone mapping the received image data using the tone mapping parameters to generate output lower dynamic range image data; and

presenting the lower dynamic range image data on the display.

9. The computer-implemented process of claim 8, wherein computing the metadata comprises generating a luminance histogram based on the image data to be displayed.

10. The computer-implemented process of claim 9, wherein computing the metadata comprises determining reference luminance levels describing the luminance range based on the histogram.

11. The computer-implemented process of claim 8, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

comparing the computed tone mapping parameters to previously computed tone mapping parameters based on metadata for a plurality of immediately previous sequential frames; and

in response to the computed tone mapping parameters being within a range of the previously computed tone mapping parameters, using the computed tone mapping parameters for the current frame as the tone mapping parameters for the image data to be displayed from the current frame.

12. The computer-implemented process of claim 11, wherein the plurality of immediately previous sequential frames comprises between eight and one hundred twenty frames.

13. The computer-implemented process of claim 8, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

averaging a history of previously computed metadata for a plurality of immediately previous sequential frames and the current frame; and

computing the tone mapping parameters for the current frame based on the averaged history.

14. The computer-implemented process of claim 13, wherein the plurality of immediately previous sequential frames comprises between eight and one hundred twenty frames.

15. An article of manufacture comprising:

a computer storage medium;

computer program instructions stored on the computer storage medium which, when processed by a processing device, instruct the processing device to perform a process comprising;

receiving image data to be displayed from a high dynamic range image;

computing metadata for the image data to be displayed, the metadata describing a luminance range of the image data;

computing tone mapping parameters for the image data based on at least the computed metadata and characteristics of a display having a lower dynamic range than the high dynamic range image;

tone mapping the received image data using the tone mapping parameters to generate output lower dynamic range image data; and

presenting the lower dynamic range image data on the display.

16. The article of manufacture of claim 15, wherein computing the metadata comprises generating a luminance histogram based on the image data to be displayed.

17. The article of manufacture of claim 16, wherein computing the metadata comprises determining reference luminance levels describing the luminance range based on the histogram.

18. The article of manufacture of claim 15, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

comparing the computed tone mapping parameters to previously computed tone mapping parameters based on metadata for a plurality of immediately previous sequential frames; and

in response to the computed tone mapping parameters being within a range of the previously computed tone mapping parameters, using the computed tone mapping parameters for the current frame as the tone mapping parameters for the image data to be displayed from the current frame.

19. The article of manufacture of claim 18, wherein the plurality of immediately previous sequential frames comprises between eight and one hundred twenty frames.

20. The article of manufacture of claim 15, wherein the high dynamic range image comprises high dynamic range video including a plurality of frames, and computing tone mapping parameters for the image data comprises:

computing tone mapping parameters based on the computed metadata for the image data to be displayed, the image data to be displayed originating from a current frame of the high dynamic range video;

averaging a history of previously computed metadata for a plurality of immediately previous sequential frames and the current frame; and

computing the tone mapping parameters for the current frame based on the averaged history.