Content-adaptive adjustment of display device brightness levels when rendering high dynamic range content

• US 10,104,334 B2
• Filed: 01/27/2017
• Issued: 10/16/2018
• Est. Priority Date: 01/27/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Evans, Glenn F., Kucukgoz, Mehmet, Wozniak, Matthew
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US15/418,522
• Publication Number:
  US 20180220101A1
• Time in Patent Office: 627 Days
• Field of Search: 382162, 382168, 348 22, 348231, 348239, 345589
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G09G 2320/0271 Adjustment of the gradation levels within the range of the gradation scale, e.g. by redistribution or clipping
• CPC Class Codes: G09G 2320/0626 for control of overall brightness
• CPC Class Codes: G09G 2340/06 Colour space transformation
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2360/16 Calculation or use of calculated indices related to luminance levels in display data
• CPC Class Codes: G09G 3/2092 Details of a display terminals using a flat panel, the details relating to the control arrangement of the display terminal and to the interfaces thereto suitable for both CRT and flat panel G09G5/003; specific for a CRT G09G1/165
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 7/0117 involving conversion of the spatial resolution of the incoming video signal for graphics images G09G2340/0407
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Innovations in rendering of high dynamic range (“HDR”) content are described. A playback system can dynamically and proactively adjust the brightness level of a target display device. In some cases, this can reduce power consumption when rendering HDR content, without any noticeable degradation in quality. For example, the playback system includes decision logic. The decision logic is configured to receive summary information for sample values of a picture of HDR content and determine (based at least in part on the summary information) a brightness level of the target display device. The playback system can send, to the target display device, a control signal that indicates the brightness level of the target display device. A tone mapper can be configured to apply tone mapping to input values for the sample values of the picture of HDR content, according to a tone mapping function, and thereby produce output values.

Claims

1. A computer system comprising one or more processing units and memory, wherein the computer system implements a playback system that is to be connected to a target display device having a brightness level that is dynamically controllable by the playback system, and wherein the playback system includes:

decision logic implemented by software executable on one or more central processing units, among the one or more processing units, by software controlling special-purpose hardware, or by special-purpose hardware, wherein the decision logic is configured to;

receive histogram information for sample values of a picture of high dynamic range (“

HDR”

) content, wherein the histogram information for the sample values of the picture of HDR content is organized as multiple bins, each of the multiple bins having a count of the sample values of the picture of HDR content within that bin, and wherein each of the multiple bins represents a range of possible sample values between a minimum possible sample value for the bin and a maximum possible sample value for the bin;

determine, based at least in part on the histogram information for the sample values of the picture of HDR content, a brightness level of the target display device for display of the picture of HDR content;

set a tone mapping function to, at least in part, compensate for a change, if any, in the brightness level of the target display device by scaling at least some input values to the tone mapping function;

send, to the target display device, a control signal that indicates the brightness level of the target display device for display of the picture of HDR content; and

after tone mapping, send, to the target display device, the sample values of the picture of HDR content; and

a tone mapper configured to apply tone mapping to input values for the sample values of the picture of HDR content, according to the tone mapping function, and thereby produce output values for the sample values of the picture of HDR content.

2. The computer system of claim 1, wherein, to determine the brightness level of the target display device for display of the picture of HDR content, the decision logic is configured to:

select a highest bin, among the multiple bins, that satisfies a threshold count; and

set the brightness level of the target display device based at least in part on the selected bin.

3. The computer system of claim 2, wherein the brightness level of the target display device for display of the picture of HDR content is also based at least in part on counts of sample values in one or more other bins, among the multiple bins, that are lower than the selected bin.

4. The computer system of claim 1, wherein the multiple bins have variable size in terms of range of possible sample values in the multiple bins, respectively.

5. The computer system of claim 4, wherein sizes of the multiple bins are perceptually weighted and define steps that are at least approximately uniform.

6. The computer system of claim 1, wherein the decision logic is further configured to buffer an indication of the brightness level of the target display device, and wherein the indication of the brightness level is paired with the sample values of the picture of HDR content and/or with encoded data for the picture of HDR content.

7. The computer system of claim 1, wherein the playback system further includes one of:

a metadata extractor configured to extract, from encoded data for the picture of HDR content, metadata indicating the histogram information and send the histogram information to the decision logic;

a digital signal processor or module configured to compute the histogram information from the sample values of the picture of HDR content and send the histogram information to the decision logic; and

a video decoder configured to decode encoded data for the picture of HDR content, and thereby produce the sample values of the picture of HDR content, compute the histogram information from the sample values of the picture of HDR content, and send the histogram information to the decision logic.

8. In a computer system that implements a playback system connected to a target display device having a brightness level that is dynamically controllable by the playback system, a method comprising:

producing sample values of a picture of high dynamic range (“

HDR”

) content;

generating, from the sample values of the picture of HDR content, histogram information for the sample values of the picture of HDR content, wherein the histogram information for the sample values of the picture of HDR content is organized as multiple bins, each of the multiple bins having a count of the sample values of the picture of HDR content within that bin, and wherein each of the multiple bins represents a range of possible sample values between a minimum possible sample value for the bin and a maximum possible sample value for the bin;

determining, based at least in part on the histogram information for the sample values of the picture of HDR content, a brightness level of the target display device for display of the picture of HDR content;

setting a tone mapping function to, at least in part, compensate for a change, if any, in the brightness level of the target display device by scaling at least some input values to the tone mapping function; and

with a tone mapper, applying tone mapping to input values for the sample values of the picture of HDR content, according to the tone mapping function, thereby producing output values for the sample values of the picture of HDR content;

sending, to the target display device, a control signal that indicates the brightness level of the target display device for display of the picture of HDR content; and

after the tone mapping, sending, to the target display device, the sample values of the picture of HDR content.

9. The method of claim 8, wherein the generating the histogram information includes one of:

with a video decoder, computing the histogram information from the sample values of the picture of HDR content; and

with a digital signal processor or other module outside of the video decoder, computing the histogram information from the sample values of the picture of HDR content.

10. The method of claim 8, wherein:

the target display device has a single backlight, and the control signal indicates the brightness level of the single backlight of the target display device;

or the target display device has multiple backlights, the histogram information is organized by regions that correspond to zones associated with the multiple backlights, respectively, and the control signal indicates the brightness level of one of the multiple backlights of the target display device.

11. The method of claim 8, further comprising:

buffering an indication of the brightness level of the target display device, wherein the indication of the brightness level is paired with the sample values of the picture of HDR content and/or with encoded data for the picture of HDR content.

12. The method of claim 8, wherein the target display device has a screen area, and wherein the method further comprises determining a proportion of the screen area that is used for rendering of the HDR content.

13. The method of claim 12, wherein, when a tile that includes only a portion less than all of the screen area is used for rendering of the HDR content, the method further comprises:

receiving histogram information for sample values of each of one or more other tiles of the screen area; and

merging the histogram information for the sample values of the picture of HDR content and the histogram information for the sample values of each of the one or more other tiles of the screen area, wherein the brightness level of the target display device is also determined based at least in part on the histogram information for the one or more other tiles of the screen area.

16. The method of claim 8, wherein the determining the brightness level of the target display device for the display of the picture of HDR content includes:

determining, from the histogram information, a maximum sample value among the sample values of the picture of HDR content; and

based at least in part on the maximum sample value, setting the brightness level of the target display device for display of the picture of HDR content.

17. The method of claim 8, wherein the determining the brightness level of the target display device for display of the picture of HDR content includes:

selecting a highest bin, among the multiple bins, that satisfies a threshold count; and

setting the brightness level of the target display device based at least in part on the selected bin.

18. The method of claim 8, wherein the multiple bins have variable size in terms of range of possible sample values in the multiple bins, respectively.

19. The method of claim 8, wherein sizes of the multiple bins are perceptually weighted and define steps that are at least approximately uniform.

14. One or more computer-readable media storing computer-executable instructions for causing a computer system, when programmed thereby, to perform operations, the computer system including a target display device having a brightness level that is dynamically controllable by the playback system, the operations comprising, for each of multiple pictures of high dynamic range (“

HDR”

) content in a video sequence;

decoding encoded data for the picture of HDR content, thereby producing sample values of the picture of HDR content;

generating histogram information for the sample values of the picture of HDR content, wherein the histogram information for the sample values of the picture of HDR content is organized as multiple bins, each of the multiple bins having a count of the sample values of the picture of HDR content within that bin, and wherein each of the multiple bins represents a range of possible sample values between a minimum possible sample value for the bin and a maximum possible sample value for the bin;

determining, based at least in part on the histogram information for the sample values of the picture of HDR content, a brightness level of the target display device for display of the picture of HDR content;

setting a tone mapping function to, at least in part, compensate for a change, if any, in the brightness level of the target display device by scaling at least some input values to the tone mapping function;

with a tone mapper, applying tone mapping to input values for the sample values of the picture of HDR content, according to the tone mapping function, thereby producing output values for the sample values of the picture of HDR content;

sending, to the target display device, a control signal that indicates the brightness level of the target display device for display of the picture of HDR content; and

after the tone mapping, sending, to the target display device, the sample values of the picture of HDR content.

15. The one or more computer-readable media of claim 14, wherein the generating the histogram information includes one of:

extracting, from the encoded data for the picture of HDR content, metadata indicating the histogram information;

with a video decoder, computing the histogram information from the sample values of the picture of HDR content; and

with a digital signal processor or module outside the video decoder, computing the histogram information from the sample values of the picture of HDR content.

20. The one or more computer-readable media of claim 14, wherein the multiple bins have variable size in terms of range of possible sample values in the multiple bins, respectively.