Scalable systems for controlling color management comprising varying levels of metadata

• US 9,111,330 B2
• Filed: 05/17/2012
• Issued: 08/18/2015
• Est. Priority Date: 05/27/2011
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Messmer, Neil W., Atkins, Robin, Margerm, Steve, Longhurst, Peter W.
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): LEE, KWANG B
• Application Number:
  US14/003,097
• Publication Number:
  US 20140078165A1
• Time in Patent Office: 1,188 Days
• Field of Search: 345/589
• US Class Current: 1/1
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: H04N 1/603 controlled by characteristics of the picture signal generator or the picture reproducer
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/440227 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 9/69 for modifying the colour signals by gamma correction controlling camera response for colour signals H04N23/83

Abstract

Several embodiments of scalable image processing systems and methods are disclosed herein whereby color management processing of source image data to be displayed on a target display is changed according to varying levels of metadata.

Claims

1. A method for processing image data on a target display device, the method comprising:

obtaining multiple sections of image data associated with an input data stream produced using a reference display device having one or more reference display device parameters, wherein each section of the multiple sections of the image data is associated with respective metadata arranged in one or more variable levels selected from multiple predetermined levels, wherein a higher level of metadata provides additional information to a lower level of metadata for calculating color management parameters for processing a corresponding section of the image data, and wherein a particular level of the multiple predetermined levels includes metadata representing the one or more reference display device parameters;

determining that metadata associated with a particular section of the multiple sections of the image data is arranged to include the particular level of metadata representing the one or more reference display device parameters of the reference display device; and

in response to determining that the metadata associated with the particular section of the image data is arranged to include the particular level of metadata representing the one or more reference display device parameters of the reference display device, calculating color management algorithm parameters for processing the particular section of the image data on the target display device based on the one or more reference display device parameters of the reference display device and one or more other parameters represented by a level of metadata different than the particular level of metadata that is associated with the particular section of the image data.

2. The method of claim 1, further comprising:

decoding the input data stream to recover the multiple sections of the image data and associated metadata.

3. The method of claim 2, wherein the step of decoding the input data stream further comprises:

splitting the input data stream into an image data stream and a metadata stream that includes the metadata associated with the multiple sections of the image data.

4. The method of claim 1, wherein calculating color management algorithm parameters further comprises predicting the particular section of the image data based upon at least one of video encoding parameters, source content gamut parameters and environmental parameters.

5. The method of claim 4, wherein said video encoding parameters further comprise primary coordinates used to encode source content, white point coordinates used to encode source content, minimum code value, maximum code value, or response curve for at least one primary.

6. The method of claim 4, wherein said source content gamut parameters further comprises a color gamut of the reference display device used to color grade the image data.

7. The method of claim 6, wherein said source content gamut parameters further comprise:

primary chromaticity coordinates, white tristimulus values, black tristimulus values, or coordinated values for maximum and minimum luminance levels.

8. The method of claim 4, wherein said environmental parameters further comprise ambient light conditions at a time of color grading the source content, a reference monitor surround color, an absolute black value for the reference monitor, or a color temperature of the ambient light during the time of color grading.

9. The method of claim 4, wherein predicting the particular section of the image data further comprises applying a substantially best fit curve to the particular section of the image data.

10. The method of claim 1, wherein calculating color management algorithm parameters further comprises dividing a luminance range of the particular section of the image data into luminance regions.

11. The method of claim 10, wherein dividing the luminance range of the particular section of the image data into luminance regions further comprises:

calculating a histogram plot of at least one image of the particular section of the image data to determine minimum, footroom, midpoint, headroom and maximum image data values of the at least one image; and

dividing the luminance range of the particular section of the image data according the minimum, footroom, midpoint, headroom and maximum image data values.

12. The method of claim 11, wherein calculating a histogram plot further comprises calculating a histogram plot of a video scene.

13. The method of claim 1, wherein calculating color management algorithm parameters further comprises:

determining if the particular section of the image data has been color graded using a first reference display device and using a second reference display device; and

changing image data mapping using the parameters of the color grading using associated with the second reference display device.

14. The method of claim 13, wherein changing image data mapping further comprises:

determining that the target display'"'"'s capabilities substantially match the capabilities of the second reference display device; and

in response to determining that the target display'"'"'s capabilities substantially match the capabilities of the second reference display, providing a linear mapping.

15. The method of claim 1, further comprising:

determining that the metadata associated with the particular section of the image data is arranged to include target display metadata; and

in response to determining that the metadata associated with the particular section of the image data is arranged to include the target display metadata, changing the particular section of the image data to be sent to the target display based upon said target display metadata.

16. The method of claim 15, wherein said target display metadata comprises target display screen characteristics data, target display ambient environment data, or user preference data.

17. The method of claim 1, wherein calculating color management algorithm parameters further comprises:

identifying regions having protected colors within the particular section of the image data; and

passing the regions having protected colors to be rendered on the target display without alteration.

18. The method of claim 1, wherein the step calculating color management algorithm parameters further comprises:

identifying regions having salient highlights within the particular section of the image data; and

clipping the regions having salient highlights.

19. The method of claim 1, wherein calculating color management algorithm parameters further comprises:

identifying regions having out of gamut color within the particular section of the image data; and

scaling the regions having out of gamut color back into gamut.

21. The method of claim 1, wherein a section of the multiple sections of image data includes one or more images or one or more video frames.

22. The method of claim 1, wherein the one or more reference display device parameters include reference display gamut parameters, a maximum luminance of the reference display device, a Tristimulus value for white and black, or an absolute XYZ value for a black level of the reference display device in a normal color grading environment.

23. The method of claim 1, wherein calculating the color management algorithm parameters comprises calculating the color management algorithm parameters based on parameters represented by one or more levels of metadata that the reference display device is capable to process.

24. The method of claim 1, wherein a count of levels of metadata is constant for the image data.

25. The method of claim 1, wherein the one or more reference display device parameters represented by the particular level of metadata are constant for the image data.

20. A system for decoding image data on a target display comprising:

a video decoder configured to obtain multiple sections of image data associated with an input data stream produced using a reference display device having one or more reference display device parameters, wherein each section of the multiple sections of the image data is associated with respective metadata arranged in one or more variable levels selected from multiple predetermined levels, wherein a higher level of metadata provides additional information to a lower level of metadata for calculating color management parameters for processing a corresponding section of the image data, and wherein a particular level of the multiple predetermined levels includes metadata representing the one or more reference display device parameters

a metadata decoder configured to determine that metadata associated with a particular section of the multiple sections of the image data is arranged to include the particular level of metadata representing the one or more reference display device parameters of the reference display device;

a color management module configured to;

receive an indication from the metadata decoder indicating that the metadata associated with the particular section of the image data is arranged to include the particular level of metadata representing the one or more reference display device parameters of the reference display device; and

calculate color management algorithm parameters for processing the particular section of the image data on the target display device based on the one or more reference display device parameters of the reference display device and one or more other parameters represented by a level of metadata different than the particular level of metadata that is associated with the particular section of the image data; and

a target display configured to render and display the particular section of the image data based on the color management algorithm parameters received from the color management module.