Method and system of multi-dynamic range multi-layer video blending with alpha channel sideband for video playback

• US 10,638,105 B2
• Filed: 06/27/2016
• Issued: 04/28/2020
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Luo, Ning, Wang, Changliang
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US16/304,456
• Publication Number:
  US 20190149792A1
• Time in Patent Office: 1,401 Days
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: G09G 2340/0428 Gradation resolution change
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 5/10 Intensity circuits
• CPC Class Codes: H04N 9/76 for mixing of colour signals H04N9/75 takes precedence

Abstract

Methods, systems, and articles of multi-dynamic range multi-layer video blending with alpha channel sideband for video playback.

Claims

1. A computer-implemented method of video playback comprising:

obtaining first image pixel data of at least a portion of an image;

obtaining second image pixel data of the image having pixel data of multiple color channels from a main band used to transmit the second image pixel data;

obtaining transparency-related alpha data associated with the pixels of the second image pixel data, wherein the alpha data forms an alpha channel sideband used to transmit the alpha data separate from the main band; and

providing the first image pixel data, the second image pixel data from the main band, and the alpha data from the alpha channel sideband to be blended to form a composite layer to be used to display the image,

wherein the first image data comprises content of a video sequence, and the second image data comprises content of an application interface that forms at least one of;

a frame at one or more sides of the video sequence, and at least partially transparent activation buttons over the video sequence.

2. The method of claim 1 wherein the first image pixel data is in a first dynamic range, the method comprising converting the second image pixel data from a second dynamic range different from the first dynamic range, and to the first dynamic range before the first image pixel data, the second image pixel data, and the alpha data are blended.

3. The method of claim 2 wherein the first dynamic range is in high dynamic range (HDR) and the second dynamic range is in standard dynamic range (SDR).

4. The method of claim 1 wherein the alpha channel sideband provides a stream of words where each word has at least one 8-bit alpha value, and the word size is a multiple of eight bits.

5. The method of claim 4 wherein the individual words each have 32 bits and carry four 8-bit alpha data values.

6. The method of claim 1 wherein the second image data comprises 32-bit words each having data of red, green, and blue (RGB) channels using 10 bits for each channel and 2 bits extra.

7. The method of claim 1 wherein the alpha channel sideband is used to transfer the alpha data between a memory and a processor to perform the blending.

8. The method of claim 1 comprising converting the second image data from SDR to HDR formats by tone mapping while extracting the alpha data to be placed in the alpha channel sideband.

9. The method of claim 1 comprising padding a final alpha data word of an individual frame to align the last word of the frame to have the same bit size as the other words in the frame.

10. The method of claim 1 wherein the first image pixel data is transmitted to a device that decompresses the first image pixel data and has the second image pixel data in a non-compressed state;

wherein the first image pixel data is in a first dynamic range, the method comprising converting the second image pixel data from a second dynamic range different from the first dynamic range, and to the first dynamic range before the first image pixel data, the second image pixel data, and the alpha data are blended; and

wherein the alpha channel sideband is used to transfer the alpha data between a memory and a processor to perform the blending.

11. A computer-implemented system of video playback comprising:

at least one memory;

at least one display;

at least one processor communicatively connected to the at least one display and at least one memory; and

a dynamic range convertor arranged to operate by;

obtaining image pixel data of an image in a first dynamic range and having non-alpha pixel data and transparency-related alpha data,

converting the non-alpha pixel data to a second dynamic range, and

placing the converted non-alpha pixel data in a main band; and

an alpha data handling unit arranged to operate by placing the alpha data from the image pixel data in an alpha channel sideband separate from the non-alpha pixel data placed in the main band,

wherein the first image data comprises content of a video sequence, and the second image data comprises content of an application interface that forms at least one of;

a frame at one or more sides of the video sequence, and at least partially transparent activation buttons over the video sequence.

12. The system of claim 11 comprising a blending unit receiving other image data in the first dynamic range and of the image, the non-alpha pixel data in the first dynamic range from the main band, and the alpha data from the alpha channel sideband to blend the other image data, the non-alpha pixel data, and the alpha data to form a composite layer of the image to be used to display the image.

13. The system of claim 12 wherein the alpha channel sideband transfers the alpha data between a memory and a processor to perform the blending.

14. The system of claim 11 wherein the first dynamic range is in standard dynamic range (SDR), and the second dynamic range is in high dynamic range (HDR).

15. The system of claim 11 wherein the alpha channel sideband provides a stream of words where each word has at least one 8-bit alpha value, and the word size is a multiple of eight bits.

16. The system of claim 15 wherein the individual words each have 32 bits and carry four 8-bit alpha data values.

17. The system of claim 11 wherein the non-alpha pixel data comprises 32-bit words each having data of red, green, and blue (RGB) channels using 10 bits for each channel and 2 bits extra.

18. The system of claim 11 wherein the non-alpha pixel data is converted from SDR to HDR formats by tone mapping, and the alpha data is extracted from the image data to be placed in the alpha channel sideband.

19. The system of claim 11 wherein a final alpha data word of an individual frame is padded to align the last word of the frame to have the same bit size as the other words in the frame.

20. The system of claim 11 wherein the image pixel data is second image pixel data, and wherein first pixel image data comprises content of a video sequence, and the second image pixel data comprises content of an application interface that forms at least one of:

a frame at one or more sides of the video sequence, and transparent activation buttons over the video sequence;

wherein the first image pixel data is transmitted to a device that decompresses the first image pixel data and wherein the device has the second image pixel data in a non-compressed state;

wherein the first image pixel data is in the second dynamic range, the method comprising converting the non-alpha data of the second image pixel data from the first dynamic range to the second dynamic range before the first image pixel data, the second image pixel data, and the alpha data are blended; and

wherein the alpha channel sideband is used to transfer the alpha data between a memory and at least one processor to perform the blending.

21. At least one non-transitory computer-readable medium having stored thereon instructions that when executed cause a computing device to operate by:

obtaining first image pixel data of at least a portion of an image and in a first dynamic range;

obtaining second image pixel data of the image and having non-alpha pixel data in a second dynamic range and transparency-related alpha data;

converting the non-alpha pixel data from the second dynamic range to the first dynamic range;

placing the non-alpha pixel data into a main band and having pixel data of multiple color channels;

placing the transparency-related alpha data into an alpha channel sideband used to transmit the alpha data separate from the main band; and

providing the first image pixel data, the converted non-alpha pixel data from the main band, and the alpha data from the alpha channel sideband to be blended to form a composite layer to be used to display the image,

wherein the first image data comprises content of a video sequence, and the second image data comprises content of an application interface that forms at least one of;

a frame at one or more sides of the video sequence, and at least partially transparent activation buttons over the video sequence.

22. The medium of claim 21 wherein the alpha channel sideband provides a stream of words where each word has at least one 8-bit alpha value, and the word size is a multiple of eight bits, wherein the individual words each have 32 bits and carry four 8-bit alpha data values.

23. The medium of claim 21 wherein the second image data comprises 32-bit words each having data of red, green, and blue (RGB) channels using 10 bits for each channel and 2 bits extra.

24. The medium of claim 21 wherein the first dynamic range is in high dynamic range (HDR) and the second dynamic range is in standard dynamic range (SDR);

wherein the alpha channel sideband provides a stream of words where each word has at least one 8-bit alpha value, and the word size is a multiple of eight bits, wherein the individual words each have 32 bits and carry four 8-bit alpha data values;

wherein the second image data comprises 32-bit words each having data of red, green, and blue (RGB) channels using 10 bits for each channel and 2 bits extra;

wherein the alpha channel sideband is used to transfer the alpha data between a memory and a processor to perform the blending;

the instructions causing the computing device to operate by;

converting the second image data from SDR to HDR formats by tone mapping, and extracting the alpha data to be placed in the alpha channel sideband;

and padding a final alpha data word of an individual frame to align the last word of the frame to have the same bit size as the other words in the frame;

wherein the first image pixel data is transmitted to a device that decompresses the first image pixel data and the device already has the second image pixel data in a non-compressed state.