Video sequence encoding system and algorithms

• US 8,526,488 B2
• Filed: 02/09/2011
• Issued: 09/03/2013
• Est. Priority Date: 02/09/2010
• Status: Active Grant

• Current Assignee: Beamr Imaging Ltd.
• Original Assignee: Vanguard Software Solutions, Inc. (Dolby Laboratories Incorporated)
• Inventors: Zheludkov, Alexander, Martemyanov, Alexey, Minin, Ivan, Axenov, Michael, Terterov, Nickolay
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): BELAI, NAOD W
• Application Number:
  US13/024,117
• Publication Number:
  US 20110194615A1
• Time in Patent Office: 937 Days
• Field of Search: 375/240
• US Class Current: 375/240
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

The video sequence encoding system with high compression efficiency is based on the H.264/AVC international video coding standard and is implementing proposed algorithms for: macroblock motion estimation; simplified rate-distortion optimization for transform size decision-making; acceleration of optimal macroblock type decision; modulation of quantization parameter with look-ahead refinement; and film-grain parameters calculation. The video encoding system comprises a motion estimation unit based on a measurement function for motion search of the matching texture block; a macroblock decision making unit; a pipeline processing module configured to accelerate complex transform; and a quantization parameter modulator configured to enhance the picture quality.

Claims

1. A video codec with high compression efficiency, the video codec comprising:

a macroblock encoding module configured to receive a digitized picture divided into macroblocks, encode at least some of the macroblocks of the picture and output a compressed stream of pixels; and

a quantization parameter modulator operably coupled to the macroblock encoding module and configured to calculate quantization parameters for the macroblocks;

wherein the macroblock encoding module comprises;

a motion estimation unit configured to estimate macroblock motion based at least in part on a measurement function for motion search of a matching texture block; and

a macroblock decision-making unit that is responsible to the motion estimation unit and that is configured to optimize selection of macroblock type and transform size;

the measurement function for motion search is calculated as;

W(I,R,M,N,QP,mvx,mvy)=C(I)·

(SAD(I,R)+Cost(M,N,mvx,mvy,QP))+M·

N·

C(R),

where;

M, N—

width and height of a current texture block;

I—

two-dimensional M×

N array of initial texture luminance component values corresponding to the current texture block with elements I(x, y),

where xε

[0,M−

1], yε

[0,N−

1];

R—

two-dimensional M×

N array of texture prediction luminance component values corresponding to a reference texture block with elements R(x, y),

where xε

[0,M−

1], yε

[0,N−

1];

QP—

a quantization parameter used for video encoding;

mvx,mvy—

components of difference between a current motion offset and a current motion vector prediction;

SAD(I, R)—

is a Sum of Absolute Differences between corresponding elements of the arrays I and R;

Cost(M,N,mvx,mvy,QP)—

a motion vector cost function; and

C(A)—

a complexity function for two-dimensional array A, A corresponding to arrays I or R, respectively;

2. The video codec according to claim 1, further comprising a pre-processing module configured to perform input video preliminary processing, including:

dividing the digitized picture into groups of pixels to thereby provide the macroblocks;

performing format transforms;

performing filtering; and

providing quality improvement.

3. The video codec according to claim 2, wherein the pre-processing module is configured to provide pipelined video processing.

4. The video codec according to claim 1, further comprising a film grain calculation module configured to restore an original film-grain effect.

5. A method of video encoding using a codec with high compression efficiency, the method comprising:

dividing an input picture into macroblocks of pixels;

calculating quantization parameters for at least some of the macroblocks;

performing motion estimation based on a measurement function for motion search of a matching texture block;

calculating an optimal macroblock type; and

selecting an optimal transform size for corresponding macroblock transforms;

wherein the measurement function for motion estimation is calculated as;

W(I,R,M,N,QP,mvx,mvy)=C(I)·

(SAD(I,R)+Cost(M,N,mvx,mvy,QP))+M·

N·

C(R),

where;

M, N—

width and height of a current texture block;

I—

two-dimensional M×

N array of initial texture luminance component values corresponding to the current texture block with elements I(x, y), where xε

[0,M−

1], yε

[0,N−

1];

R—

two-dimensional M×

N array of texture prediction luminance component values corresponding to a reference texture block with elements R(x, y), where xε

[0,M−

1], yε

[0,N−

1];

QP—

a quantization parameter used for video encoding;

mvx,mvy—

components of difference between a current motion offset and a current motion vector prediction;

SAD(I,R)—

is a Sum of Absolute Differences between corresponding elements of the arrays I and R;

Cost(M,N,mvx,mvy,QP)—

a motion vector cost function; and

C(A)—

a complexity function for two-dimensional array A, A corresponding to arrays I or R, respectively;

6. The method of video encoding according to claim 5, comprising preliminary processing input video using pipelined video processing.

7. The method of pipelined video processing in a video encoding system of claim 6, comprising:

providing at least one elementary object, the object having one input data pipe and one output data pipe;

sending at least one line of pixels of an input image from an external memory to the input data pipe;

storing input pixel lines in an internal object memory;

when the number of input lines in the internal object memory equals a predefined number, calculating a line of output pixels;

forwarding at least one line of output pixels to the output data pipe;

repeating until processing of the input image is complete; and

combining lines of output pixels into an output image.

8. The method of claim 7, wherein, in case of complex transform, the method comprises:

providing a plurality of elementary objects corresponding to elementary transforms;

combining the plurality of elementary objects into a chain having one input data pipe and one output data pipe, each object being responsible for a respective transform;

sending the input image pixels into input data pipe of the chain; and

receiving the lines of output pixels from the chain output data pipe.

9. The method of video encoding according to claim 5, wherein the calculation of quantization parameters for at least some of the macroblocks is performed using a look-ahead approach.

10. The method of calculating quantization parameters in a video sequence encoder of claim 9, comprising:

creating a reduced resolution picture for each frame in the video sequence;

performing block-based motion estimation for each reduced resolution picture beginning with a second frame of the video sequence;

calculating Inter-complexity and Intra-complexity functions for each macroblock in the reduced resolution picture;

calculating importance as a minimal value of the Inter-complexity function and the Intra-complexity function;

creating an importance map;

filtering the importance map using a multi-pass non-linear filter; and

modulating a quantization parameter using the importance map.

11. The method of modulating the quantization parameter of claim 10, comprising reducing direct resolution twice in each direction and calculating each pixel in the reduced resolution picture as:

RRP[x][y]=(Luma_Orig[x*2][y*2]+Luma_Orig[x*2+1][y*2]+Luma_Orig[x*2][y*2+1]++Luma_Orig[x*2+1][y*2+1]+2)/4.

12. The method of claim 10, comprising refining an importance map of a previous reduced resolution picture using motion information from a current reduced resolution picture.

13. The method of claim 12, wherein an importance is calculated as:

NewImportance[x+mvx][y+mvy]=(NewImportance[x+mvx][y+mvy]+CurrentImportance[x][y])/4,

where;

mvx and mvy—

motion vectors found during block-based motion estimation for the current picture.

14. The method of claim 5, wherein the auxiliary function F(A, x, y, dx, dy) is calculated as:

15. The method of claim 5, wherein the step of selecting the optimal transform size for a current macroblock transform comprises:

defining a function for quantization of Discrete Cosine (DC) transform coefficients for a 4×

4 transform;

defining a function for quantization of Discrete Cosine (DC) transform coefficients for an 8×

8 transform;

defining a function for inverse quantization of Discrete Cosine (DC) transform coefficients for a 4×

4 transform;

defining a function for inverse quantization of Discrete Cosine (DC) transform coefficients for an 8×

8 transform;

calculating a cost function for a 4×

4 transform based on the functions for quantization and inverse quantization of DC transform coefficients for a 4×

4 transform and a quantization parameter for the current macroblock;

calculating a cost function for an 8×

8 transform based on the functions for quantization and inverse quantization of DC transform coefficients for an 8×

8 transform and the quantization parameter for the current macroblock;

comparing values of cost functions for a 4×

4 transform and for an 8×

8 transform; and

selecting an optimal transform size based on the cost function with a minimum value.

16. The method of claim 15, wherein the cost function for the 4×

4 transform is calculated as;

17. A method of accelerating selection of optimal macroblock type in a video sequence encoder with high compression efficiency, comprising:

defining a two-dimensional 16×

16 array of picture luma component values corresponding to a current macroblock MB;

choosing a best mode of Intra 16×

16 prediction;

choosing best motion vectors and reference frames of Inter prediction;

calculating two-dimensional 16×

16 arrays of texture prediction for luma component values corresponding to current macroblock for Intra 16×

16 macroblock type P<sub>INTRA—

16×

16</sub> and for Inter macroblock type P_INTER;

calculating two dimensional arrays of differences R<sub>INTRA—

16×

16</sub> between MB and P<sub>INTRA—

16×

16</sub>;

calculating two dimensional arrays of differences R_INTER between MB and P_INTER;

calculating a sum of absolute values SAD(R_INTER);

calculating a sum of absolute values SAD(R<sub>INTRA—

16×

16</sub>);

calculating an auxiliary complexity function C(MB) as;

18. The method of claim 17, wherein K=5.86, and values of M(QP), S(QP), and T(QP) are each selected from a corresponding array as follows:

M (QP)=

{12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 12.11, 13.13, 13.48, 13.87, 14.96, 15.66, 16.80, 17.30, 18.36, 19.73, 21.99, 23.67, 26.09, 28.52, 31.84, 34.77, 37.99, 41.02, 44.14, 46.37, 49.10, 52.73, 54.96, 59.61, 63.48, 68.36, 73.24, 78.13, 83.59, 89.06, 94.53, 100.00, 105.47, 114.26, 123.05, 131.84, 140.63, 153.13, 165.63, 178.13, 190.63, 203.13},

S(QP)=

{900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 900, 920, 950, 1000, 1080, 1190, 1350, 1536, 1690, 1870, 2080, 2340, 2600, 2840, 3060, 3280, 3500, 3840, 4230, 4580, 5020, 5500, 6170, 6920, 7620, 8500, 9390, 10310, 11280, 12110, 13000},

T (QP)=

{256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 256, 265, 273, 281, 289, 300, 313, 320, 329, 353, 369, 400, 425, 433, 457, 465, 500, 521, 545, 553, 568, 594, 610, 634, 658, 700, 778, 882, 1000, 1176, 1404, 1653, 1901, 2142, 2400}.