Guided color grading for extended dynamic range

• US 9,679,366 B2
• Filed: 10/21/2014
• Issued: 06/13/2017
• Est. Priority Date: 10/22/2013
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Xu, Ning, Chen, Tao, Crenshaw, James E., Hulyalkar, Samir N.
• Primary Examiner(s): Zhai, Kyle
• Application Number:
  US15/030,346
• Publication Number:
  US 20160253792A1
• Time in Patent Office: 966 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: H04N 1/40006 Compensating for the effects of ageing, i.e. changes over time
• CPC Class Codes: H04N 1/6052 Matching two or more picture signal generators or two or more picture reproducers
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 5/57 Control of contrast or brightness
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85

Abstract

Novel methods and systems for color grading are disclosed. The color grading process for a visual dynamic range image can be guided by information relating to the color grading of other images such as the standard dynamic range image.

Claims

1. A method to perform color grading for obtaining a higher dynamic range color graded image, the method comprising the following computer-implemented steps:

providing an ungraded image;

providing a color graded lower dynamic range image representing the same material as the ungraded image;

spatially and temporally aligning the ungraded image and the color graded lower dynamic range image thereby obtaining an aligned ungraded image; and

color grading the aligned ungraded image based on the color graded lower dynamic range image, thereby obtaining the higher dynamic range color graded image,

wherein;

(a) the spatially and temporally aligning comprises;

detecting feature points in the ungraded image and in the color graded lower dynamic range image;

matching the feature points; and

estimating a transformation between the ungraded image and the color graded lower dynamic range image, based on the matching of the feature points, and

(b) the detecting further comprises extracting feature descriptors in at least one local region of the ungraded image and at least one local region of the color graded lower dynamic range image, and wherein the matching further comprises matching the feature descriptors.

2. The method of claim 1, wherein the spatially and temporally aligning comprises applying a hand trim alignment pass.

3. The method of claim 1, wherein the color grading comprises applying a hand trim color grading pass.

4. The method of claim 1, wherein the image is a frame in a video.

5. The method of claim 1, wherein the spatially and temporally aligning comprises applying a hand trim alignment pass.

6. The method of claim 1, wherein the color grading comprises applying a hand trim color grading pass.

7. A method to perform color grading for obtaining a higher dynamic range master image, the method comprising the following computer-implemented steps:

providing an ungraded image;

providing a color graded lower dynamic range image representing the same material as the ungraded image;

applying an inverse tone and gamut mapping transformation defined based on the dynamic range and color gamut of the higher dynamic range master image to the color graded lower dynamic range image, thereby obtaining an inversely mapped lower dynamic range image;

providing a higher dynamic range editing, based on the ungraded image; and

applying the higher dynamic range editing to the ungraded image to generate an edited ungraded image, and a color transformation to the edited ungraded image, wherein the color transformation is based on the ungraded image and the inversely mapped lower dynamic range image, thereby obtaining the higher dynamic range master image.

8. The method of claim 7, wherein the higher dynamic range editing comprises spatially and temporally aligning the ungraded image and the inversely mapped lower dynamic range image, thereby obtaining an aligned image, and wherein the step of applying a color transformation is applied to the aligned image, thereby obtaining the higher dynamic range master image.

9. The method of claim 8, further comprising the computer-implemented step of determining corresponding colors between the ungraded image and the inversely mapped lower dynamic range image, and wherein the applying a color transformation is further based on the corresponding colors.

10. The method of claim 9, wherein the determining corresponding colors comprises applying adaptive sampling.

11. The method of claim 10, wherein the adaptive sampling is based on having a near-uniform distribution of intensity.

12. The method of claim 9, wherein the color transformation comprises three one-dimensional functions for three color components.

13. The method of claim 12, wherein the one-dimensional functions are a polynomial or are based on a look-up table.

14. The method of claim 9, wherein the color transformation comprises a three by three linear matrix.

15. The method of claim 9, wherein the color transformation comprises color grading outlier pixels at least one additional time.

16. The method of claim 9, wherein the image is a frame in a video.

17. The method of claim 7, wherein the applying an inverse tone and gamut mapping transformation comprises the following computer-implemented steps:

applying gamma decoding to the color graded lower dynamic range image in a first color space, thereby obtaining a gamma decoded first image;

converting the gamma decoded first image from the first color space to a second color space, thereby obtaining a second image;

applying a reverse tone mapping transformation to the second image, thereby obtaining a third image;

applying a reverse saturation correction to the third image, thereby obtaining a fourth image; and

converting the fourth image from the second color space to a third color space, thereby obtaining an inversely mapped lower dynamic range image.

18. The method of claim 7, wherein the image is a frame in a video.