Method and apparatus for converting low dynamic range video to high dynamic range video

• US 10,445,865 B1
• Filed: 03/27/2018
• Issued: 10/15/2019
• Est. Priority Date: 03/27/2018
• Status: Active Grant

• Current Assignee: TFI Digital Media Limited
• Original Assignee: TFI Digital Media Limited
• Inventors: Wong, Hon Wah, Luk, Hon Tung, Fong, Chi Keung, Sze, Yin, Cheung, Hok Kwan, Yuen, Yiu Fai
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/936,459
• Publication Number:
  US 20190304072A1
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/92 based on global image properties

Abstract

A method and apparatus for converting a standard dynamic range (SDR) video to a high dynamic range (HDR) video. The conversion is adaptive and takes both spatial and temporal information of a current frame and previous frames into consideration such that the majority of pixels falls into the most sensitive regions of human eyes in the target dynamic range, while at the same time, the continuity of luminance is maintained in temporal domain to prevent flickering.

Claims

1. A method for converting a first dynamic range video to a second dynamic range video, comprising providing image processing electronics configured to:

receive the first dynamic range video;

generate one or more breathing parameters of a current frame from temporal information and spatial information of the current frame and one or more previous frames of the first dynamic range video;

determining one or more mapping functions according to the generated breathing parameters of the current frame and one or more spatial criteria;

determining one or more weights for each pixel of the current frame corresponding to the determined mapping functions;

converting a pixel of the current frame to a converted pixel according to the determined mapping functions and corresponding weights; and

repeating the conversion for all pixels of the current frame to form a converted frame in the second dynamic range video.

2. The method of claim 1, wherein the spatial information of the current frame includes one or more of distributions of luminance, chrominance, red, green, and blue (RGB) channels, and combinations thereof, post-processing results based on one or more of the RGB channels and combinations thereof of the current frame.

3. The method of claim 1, wherein the mapping functions are non-deceasing convex functions.

4. The method of claim 1, wherein the generation of the breathing parameters of the current frame comprises:

retrieving one or more previous breathing parameters of the previous frames;

creating a histogram or cumulative frequency curve for the spatial information of the current frame;

transforming the histogram or cumulative frequency curve into temporary breathing parameters of the current frame; and

generating the current breathing parameters of the current frame by combining the previous breathing parameters of the previous frames and the temporary breathing parameters of the current frame with a temporal function so as to control the difference between the breathing parameters of the current frame and the previous frames to below a threshold.

5. The method of claim 4, wherein the temporal function is a linear function along a temporal domain.

6. The method of claim 1, wherein the determination of the mapping functions comprises:

applying one or more spatial criteria on the current breathing parameters of the current frame to estimate one or more percentages of which the spatial criteria are satisfied by the current frame;

clustering the pixels of the current frame into one or more pixel-groups depending on whether any one of the spatial criteria and which spatial criterion is satisfied by each pixel; and

determining the mapping functions for each of the pixel-groups according to the satisfied spatial criteria.

7. The method of claim 6, wherein the pixels satisfying one of the spatial criteria are pixels of a skin tone color.

8. The method of claim 6, wherein the pixels satisfying one of the spatial criteria are pixels of a high saturation color.

9. The method of claim 1, wherein the determination of the weights comprises:

applying the spatial criteria to estimate the suitability of each of the pixels to each of the mapping functions; and

determining the respective weights for each pixel corresponding to each of the mapping functions, according to the estimated suitability.

10. The method of claim 1, wherein the conversion of a pixel of the current frame to a converted pixel comprises:

mapping a pixel of the current frame to one or more temporary converted pixel values with the mapping functions; and

calculating a weighted sum of the temporary converted pixel values with the weights corresponding to the mapping functions to obtain a converted pixel in the converted frame in the second dynamic range video.

11. The method of claim 10, wherein the conversion of the pixel-groups comprises multiplying a constant instead of using the mapping functions, wherein the constant is an effective gain of applying the mapping functions on the maximum value of the pixels.

12. An apparatus for converting a first dynamic range video to a second dynamic range video, configured to:

receive the first dynamic range video;

generate one or more breathing parameters of a current frame from temporal information and spatial information of the current frame and one or more previous frames of the first dynamic range video;

determining one or more mapping functions according to the generated breathing parameters of the current frame and one or more spatial criteria;

determining one or more weights for each pixel of the current frame corresponding to the determined mapping functions;

converting a pixel of the current frame to a converted pixel according to the determined mapping functions and corresponding weights; and

repeating the conversion for all pixels of the current frame to form a converted frame in the second dynamic range video.