Advanced screen content coding solution

• US 10,291,827 B2
• Filed: 11/20/2014
• Issued: 05/14/2019
• Est. Priority Date: 11/22/2013
• Status: Active Grant

• Current Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Futurewei Technologies Incorporated (Huawei Investment & Holding Co., Ltd.)
• Inventors: Ma, Zhan, Wang, Wei, Yu, Haoping, Wang, Xian, Ye, Jing
• Primary Examiner(s): Moyer, Andrew M
• Application Number:
  US14/549,405
• Publication Number:
  US 20150146976A1
• Time in Patent Office: 1,636 Days
• Field of Search: None
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 1/646 Transmitting or storing colour television type signals, e.g. PAL, Lab; Their conversion into additive or subtractive colour signals or vice versa therefor H04N1/642, H04N1/644 take precedence
• CPC Class Codes: H04N 19/157 Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/93 Run-length coding
• CPC Class Codes: H04N 2201/0089 Image display device

Abstract

A method and device for coding screen content into a bitstream by selecting a color palette table for a coding unit (CU) of screen content, creating a color index map having indices for the coding unit (CU), and encoding the selected color palette table and the color index map for the CU into a bitstream.

Claims

1. A method for coding screen content into a bitstream, the method comprising:

dividing the screen content into a plurality of coding units (CUs), wherein each CU comprises a plurality of pixels and wherein each pixel has a numerical value;

creating a color palette table for a first coding unit (CU) of the plurality of CUs, wherein the color palette table comprises a plurality of entries, and wherein the plurality of entries indicate the most frequent numerical values of the plurality of pixels in the first CU;

creating a first color index map for the first CU, the first color index map comprising an entry for each pixel in the first CU, wherein each entry in the first color index map comprises an index into the color palette table;

creating a second color index map for a second CU of the plurality of CUs, the second color index map comprising an entry for each pixel in the second CU, wherein each entry in the second color index map comprises an index into the color palette table;

creating a first prediction residual table comprising an entry for each pixel in the first CU, wherein each entry in the first prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the first color index map entry for that pixel and the numerical value of that pixel;

creating a second prediction residual table comprising an entry for each pixel in the second CU, wherein each entry in the second prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the second color index map entry for that pixel and the numerical value of that pixel; and

encoding the color palette table, the first color index map, the second color index map, the first prediction residual table, and the second prediction residual table into the bitstream.

2. The method as specified in claim 1, wherein the method is processed using a planar color format.

3. The method as specified in claim 1, wherein the color palette table is derived from a neighboring CU using a reconstructed CU in a pixel domain.

4. The method as specified in claim 3 further comprising generating a color palette table of the neighboring CU based on available reconstructed pixels, wherein the neighboring CU has a different depth and size than the first CU.

5. The method as specified in claim 3 further comprising generating the color palette table of the neighboring CU based on available reconstructed pixels, wherein the neighboring CU has the same depth and size as the first CU.

6. The method as specified in claim 1, wherein generating the color palette table comprises creating a histogram of numerical values of the plurality of pixels in the first CU, sorting the histogram from most frequent numerical value to least frequent numerical value, and creating an entry in the color palette table for the most frequent numerical value.

7. The method of claim 6, wherein generating the first color index map comprises comparing the numerical value of each pixel in the first CU with each color palette table entry and creating an entry in the first color index map for each pixel in the first CU, wherein the entry in the first color index map corresponds to the index of the color palette table entry whose value is closest to the numerical value of that pixel, wherein generating the second color index map comprises comparing the numerical value of each pixel in the second CU with each color palette table entry and creating an entry in the second color index map for each pixel in the second CU, and wherein the entry in the second color index map corresponds to the index of the color palette table entry whose value is closest to the numerical value of that pixel.

8. The method as specified in claim 1 further comprising encoding a size of the color palette table into the bitstream.

9. The method as specified in claim 1 further comprising encoding an indicator of a spatial relationship between the first CU and the second CU within the screen content into the bitstream.

10. The method as specified in claim 1, wherein encoding the first color index map comprises performing a one-dimension string match operation, wherein the string match operation searches for matches in a single direction, and wherein a string match is signaled using matched pairs.

11. The method as specified in claim 10, wherein the string match operation is performed on a plurality of adjacent pixels using a running hash method.

12. The method as specified in claim 1, wherein the method is processed using an interleaved color format.

13. The method of claim 1, wherein generating the color palette table further comprises creating a color palette table entry for each unique numerical value found in the plurality of pixels in the first CU.

14. The method as specified in claim 1, wherein encoding the first color index map comprises performing a hybrid one-dimension string match operation, wherein encoding the second color index map comprises performing the hybrid one-dimension string match operation, wherein the hybrid one-dimension string match operation searches for matches in two directions per dimension, and wherein a string match is signaled using matched pairs.

15. The method as specified in claim 14, wherein the string match operation is performed on a rectangular block of adjacent pixels using a running hash method.

16. The method as specified in claim 1, wherein encoding the first color index map comprises performing a two-dimension string match operation, wherein encoding the second color index map comprises performing the two-dimension string match operation, wherein the two-dimension string match operation searches for matches in a single direction per dimension, and wherein a string match is signaled using matched pairs.

17. The method as specified in claim 16, wherein the string match operation is performed on a rectangular block of adjacent pixels using a running hash method.

18. The method of claim 1, wherein for a first entry in the first color index map for the first CU:

performing a first search in a vertical direction for a first string of entries in the first color index map, wherein each entry in the first string has the same index value as the first entry;

performing a second search in a horizontal direction for a second string of entries in the first color index map, wherein each entry in the second string has the same index value as the first entry; and

encoding a portion of the first color index map according to the results of the first search and the second search, wherein the encoding comprises insertion, into the bitstream, of a search direction flag, an indicator of the index value of the first entry, and an indicator of the length of the string of entries in the first color index map having the same index value as the first entry.

19. The method of claim 18, wherein encoding the portion of the color index map according to the results of the first search and the second search further comprises:

determining the rate distortion (RD) cost associated with encoding the first string; and

determining the RD cost associated with encoding the second string,

wherein, when the RD cost associated with encoding the first string is less than the RD cost associated with encoding the second string, the search direction flag indicates the vertical direction and the indicator of the length of the string of entries in the index map is an indicator of the length of the first string, and

wherein, when the RD cost associated with encoding the second string is less than the RD cost associated with encoding the first string, the search direction flag indicates the horizontal direction and the indicator of the length of the string of entries in the index map is an indicator of the length of the second string.

20. A system for coding screen content into a bitstream, the system comprising:

a memory comprising a plurality of coding units (CUs), wherein the plurality of CUs represents the screen content, wherein each CU comprises a plurality of pixels, and wherein each pixel has a numerical value; and

a processor coupled with the memory, wherein the processor is configured to;

create a color palette table for a first CU of the plurality of CUs, wherein the color palette table comprises a plurality of entries, wherein the plurality of entries indicate the most frequent numerical values of the plurality of pixels in the first CU;

create a first color index map for the first CU, the first color index map comprising an entry for each pixel in the first CU, wherein each entry in the first color index map comprises an index into the color palette table;

create a second color index map for a second CU of the plurality of CUs, the second color index map comprising an entry for each pixel in the second CU, wherein each entry in the second color index map comprises an index into the color palette table;

create a first prediction residual table, the first prediction residual table comprising an entry for each pixel in the first CU, wherein each entry in the first prediction residual table comprises the difference between the color palette table value corresponding to the first color index map entry for that pixel and the numerical value of that pixel;

create a second prediction residual table, the second prediction residual table comprising an entry for each pixel in the second CU, wherein each entry in the second prediction residual table comprises the difference between the color palette table value corresponding to the second color index map entry for that pixel and the numerical value of that pixel; and

encode the color palette table, the first color index map, the first prediction residual table, the second prediction residual table, and the second color index map into the bitstream.

21. The system of claim 20, wherein the CUs represent the screen content using a planar color format.

22. The system of claim 20, wherein the CUs represent the screen content using an interleaved color format.

23. The system of claim 20, wherein the processor is further configured to create the color palette table by creating a histogram of numerical values of the plurality of pixels in the first CU, sorting the histogram from most frequent numerical value to least frequent numerical value, and creating an entry in the color palette table for the most frequent numerical value.

24. The system of claim 23, wherein the processor is further configured to generate the first color index map by comparing the numerical value of each pixel in the first CU with each color palette table entry and creating an entry in the first color index map for each pixel in the first CU, wherein the entry in the first color index map corresponds to the index of the color palette table entry whose value is closest to the numerical value of that pixel, wherein the processor is further configured to generate the second color index map comprises comparing the numerical value of each pixel in the second CU with each color palette table entry and creating an entry in the second color index map for each pixel in the second CU, and wherein the entry in the second color index map corresponds to the index of the color palette table entry whose value is closest to the numerical value of that pixel.

25. A method for decoding screen content from a bitstream, comprising:

receiving the bitstream, wherein the bitstream comprises an encoded palette table, an encoded first color index map for a first coding unit (CU), an encoded second color index map for a second CU, an encoded first prediction residual table, and an encoded second prediction residual table;

creating a first color index map from the first encoded color index map, wherein the first color index map comprises a plurality of entries, wherein each entry in the first color index map comprises an index value into a color palette table created from the encoded color palette table, and wherein creating the first color index map comprises;

locating an encoded string within the bitstream, wherein the encoded string is associated with a starting entry in the color index map and wherein the encoded string comprises a direction flag, an indicator of an index value of an entry in the color index map, and an indicator of the length of the string of entries in the index map having the same index value as the first entry;

determining a reference index value according to the indicator of the index value in the encoded string;

determining a string length from the indicator of the length of the string in the encoded string; and

repeatedly inserting the reference index value into the color index map beginning at the starting entry and continuing according to the direction flag and the string length; and

creating a second color index map from the encoded second color index map, wherein the second color index map comprises a plurality of entries, wherein each entry in the second color index map comprises an index value into the color palette table;

creating a first prediction residual table from the encoded first prediction residual table, the first residual prediction table comprising an entry for each pixel in the first CU, wherein each entry in the first prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the first color index map entry for that pixel and the numerical value of that pixel;

creating a second prediction residual table from the encoded second prediction residual table, the second prediction residual table comprising an entry for each pixel in the second CU, wherein each entry in the second prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the second color index map entry for that pixel and the numerical value of that pixel; and

creating the first CU according to the first color index map and the color palette table and the second CU according to the second color index map and the color palette table.

26. An apparatus for decoding screen content from a bitstream, comprising:

a memory configured to store a bitstream comprising an encoded palette table, an encoded first color index map for a first coding unit (CU), an encoded second color index map for a second CU, an encoded first prediction residual table, and an encoded second prediction residual table, wherein the first CU and the second CU each comprise a plurality of pixels associated with screen content; and

a processor coupled with the memory, wherein the processor is configured to;

create a first color index map from the encoded first color index map, wherein the first color index map comprises a plurality of entries, wherein each entry in the first color index map comprises an index value into the color palette table;

create a second color index map from the encoded second color index map, wherein the second color index map comprises a plurality of entries, wherein each entry in the second color index map comprises an index value into the color palette table;

locate an encoded string within the bitstream, wherein the encoded string is associated with a starting entry in the color index map and wherein the encoded string comprises a direction flag, an indicator of an index value of an entry in the color index map, and an indicator of the length of the string of entries in the index map having the same index value as the first entry;

determine a reference index value according to the indicator of the index value in the encoded string;

determine a string length from the indicator of the length of the string in the encoded string;

repeatedly insert the reference index value into the color index map beginning at the starting entry and continuing according to the direction flag and the string length;

create a first prediction residual table from the encoded first prediction residual table, the first residual prediction table comprising an entry for each pixel in the first CU, wherein each entry in the first prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the first color index map entry for that pixel and the numerical value of that pixel;

create a second prediction residual table from the encoded second prediction residual table, the second prediction residual table comprising an entry for each pixel in a second CU, wherein each entry in the second prediction residual table comprises the difference between the numerical value of the color palette table entry corresponding to the second color index map entry for that pixel and the numerical value of that pixel; and

create the first CU according to the first color index map and the color palette table and the second CU according to the second color index map and the color palette table.