METADATA DESCRIBING NOMINAL LIGHTING CONDITIONS OF A REFERENCE VIEWING ENVIRONMENT FOR VIDEO PLAYBACK

• US 20160366444A1
• Filed: 05/04/2016
• Published: 12/15/2016
• Est. Priority Date: 06/09/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Sullivan, Gary J.
• Granted Patent: US 10,368,105 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

Innovations in the areas of generating, parsing, and using metadata that describes nominal lighting conditions of a reference viewing environment for video playback are presented herein. In various examples described herein, metadata includes parameters that describe the nominal lighting conditions (e.g., level of ambient light, color characteristics of ambient light) of a reference viewing environment. By conveying a representation of the nominal lighting conditions of the reference viewing environment (e.g., one assumed when mastering image content), a transmitter system can enable a receiver system to adapt its local display of the image content. Upon receiving image content and the metadata, the receiver system can identify characteristics of the actual viewing environment, use the metadata to determine whether the actual viewing environment matches the reference viewing environment, and, if not, adjust sample values of the image content, adjust a display device, or adjust lighting conditions of the actual viewing environment.

Claims

1. A computing system comprising:

a metadata generator configured to generate metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of;

one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and

one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space; and

a buffer configured to store the metadata for output with encoded data for the video.

2. The computing system of claim 1, further comprising:

a buffer configured to receive input video for the video;

a pre-processor configured to selectively adjust at least some characteristics of the input video based on one or more of (a) characteristics of a reference display device, and (b) the one or more nominal lighting conditions of the reference viewing environment; and

a video encoder configured to encode the selectively adjusted input video to produce the encoded data for the video.

3. The computing system of claim 1, wherein the indicator of light per unit of area is a value in units of lux, and wherein the coordinates in the multi-dimensional mapping of values in the color space are normalized x and y chromaticity coordinates of the ambient light in the reference viewing environment.

4. The computing system of claim 1, wherein the one or more parameters that specify the nominal color characteristic of the ambient light further include a value in a color temperature scale.

5. The computing system of claim 1, wherein the metadata further includes one or more parameters that specify one or more of:

a nominal level of light reflected by a reference display device in the reference viewing environment; and

one or more color characteristics of the light reflected by the reference display device in the reference viewing environment.

6. The computing system of claim 1, wherein the metadata further includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from an initial lighting state of a hypothetical viewing environment.

7. The computing system of claim 1, wherein the metadata further includes multiple sets of parameters, and wherein each of the multiple sets of parameters:

is associated with a different hypothetical viewing environment having an initial lighting state; and

includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from the initial lighting state of the hypothetical viewing environment for that set of parameters.

8. The computing system of claim 1, wherein the reference viewing environment includes multiple regions, and wherein the metadata includes, for each of the multiple regions, one or more parameters that specify those of the one or more nominal lighting conditions that apply for the region, the multiple regions including a first region and a second region, the first region being behind an expected position of a viewer in the reference viewing environment, the second region being in front of the expected position of the viewer in the reference viewing environment.

9. The computing system of claim 1, wherein the metadata is organized as part of:

a supplemental enhancement information (“

SEI”

) message, wherein the SEI message is part of a video elementary bitstream for the video;

a video usability information (“

VUI”

) message, wherein the VUI message is part of the video elementary bitstream for the video;

a field of a container format;

or

a message according to a transport protocol.

10. In a computing system, a method comprising:

receiving metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of;

one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and

one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space; and

parsing the metadata.

11. The method of claim 10, further comprising:

determining one or more actual lighting conditions of an actual viewing environment; and

to compensate for differences between the one or more actual lighting conditions and the one or more nominal lighting conditions, adjusting one or more of;

a characteristic of at least some sample values of the video;

a characteristic of an actual display device in the actual viewing environment; and

at least one of the one or more actual lighting conditions of the actual viewing environment.

12. The method of claim 11, wherein the one or more actual lighting conditions include an actual level of ambient light in the actual viewing environment, and wherein the adjusting comprises:

if the nominal level of ambient light is brighter than the actual level of ambient light, decreasing brightness of the at least some sample values of the video; and

if the nominal level of ambient light is darker than the actual level of ambient light, increasing brightness of the at least some sample values of the video.

13. The method of claim 11, wherein the one or more actual lighting conditions include an actual color characteristic of ambient light in the actual viewing environment, and wherein the adjusting comprises scaling or attenuating the at least some sample values of the video to compensate for differences between the actual color characteristic and the nominal color characteristic.

14. The method of claim 11, wherein the metadata includes multiple sets of parameters, and wherein each of the multiple sets of parameters:

is associated with a different hypothetical viewing environment having an initial lighting state; and

includes one or more parameters that specify, for the one or more nominal lighting conditions of the reference viewing environment, adjustments, if any, to reach a final lighting state from the initial lighting state of the hypothetical viewing environment for that set of parameters.

15. The method of claim 14, further comprising:

based at least in part on the one or more actual lighting conditions of the actual viewing environment, selecting one of the multiple sets of parameters whose different hypothetical viewing environment is closest to the actual viewing environment; and

for the adjusting, performing adjustments, if any, for the selected set of parameters to reach the final lighting state.

16. The method of claim 11, wherein the determining the one or more actual lighting conditions includes measuring at least one of the one or more actual lighting conditions.

17. The method of claim 11, further comprising:

receiving user input that specifies a value for at least one of the one or more actual lighting conditions; and

storing the specified value in a storage area;

wherein the determining the one or more actual lighting conditions includes looking up the specified value in the storage area.

18. The method of claim 10, further comprising:

receiving encoded data for the video; and

decoding the encoded data for the video.

19. The method of claim 10, wherein the metadata is organized as part of a supplemental enhancement information (“

SEI”

) message, wherein the SEI message is part of a video elementary bitstream for the video, and wherein persistence scope for the SEI message is one of;

coded layer-wise video sequence (“

CLVS”

), such that the SEI message persists, for a current layer in decoding order, from a current picture until end of the CLVS;

coded video sequence, such that the SEI message persists from the current picture until end of the coded video sequence; and

some other portion of the video, such that the SEI message persists until end of the portion of the video.

20. A computing system comprising:

a buffer configured to store metadata that describes one or more nominal lighting conditions of a reference viewing environment for playback of video, wherein the metadata includes one or more of;

one or more parameters that specify a nominal level of ambient light in the reference viewing environment, the one or more parameters that specify the nominal level of ambient light including an indicator of light per unit of area; and

one or more parameters that specify a nominal color characteristic of the ambient light in the reference viewing environment, the one or more parameters that specify the nominal color characteristic of ambient light including coordinates in a multi-dimensional mapping of values in a color space;

a metadata parser configured to parse the metadata;

a buffer configured to store encoded data for video; and

a video decoder configured to decode the encoded data for the video.