Real-time video coding/decoding

• US 7,336,720 B2
• Filed: 09/26/2003
• Issued: 02/26/2008
• Est. Priority Date: 09/27/2002
• Status: Active Grant

• Current Assignee: Beamr Imaging Ltd.
• Original Assignee: Vanguard Software Solutions, Inc. (Dolby Laboratories Incorporated)
• Inventors: Terterov, Irena, Terterov, Nick, Neimark, Eugene, Dolgoborodov, Alexey, Semenyuk, Vladimir, Zheludkov, Alexander, Martemyanov, Alexey
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US10/672,195
• Publication Number:
  US 20050276323A1
• Time in Patent Office: 1,614 Days
• Field of Search: 375/240.12, 375/240.13, 375/240.14, 375/240.05, 375/240.16, 375/240.15, 382/282, 382/243, 382/236
• US Class Current: 375/240.12
• CPC Class Codes: H03M 7/4006 Conversion to or from arithmetic code
• CPC Class Codes: H04N 19/10 using adaptive coding
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/62 : by frequency transforming in three dimensions H04N19/63 takes precedence
• CPC Class Codes: H04N 19/63 : using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/649 : the transform being applied to non rectangular image segments
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 : Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop

Abstract

A video codec for real-time encoding/decoding of digitized video data with high compression efficiency, comprising a frame encoder receiving input frame pixels; a codec setting unit for setting and storing coding setting parameters; a CPU load controller for controlling desired frame encoding time and CPU loading; a rate controller for controlling frame size; a coding statistics memory for storing frequency tables for arithmetic coding of bitstream parameters and a reference frame buffer for storing reference frames. The frame encoder comprises a motion estimation unit, a frame head coding unit, a coded frame reconstruction and storage unit and a macroblock encoding unit. The macroblock encoding unit provides calculation of texture prediction and prediction error, transforming texture prediction error and quantization of transform coefficient, calculation of motion vector prediction and prediction error and arithmetic context modeling for motion vectors, header parameters and transform coefficients. The codec also includes a deblocking unit for processing video data to eliminate blocking effect from restored data encoded at high distortion level, which may be a part of encoder or decoder, an internal resize unit, providing matching downscaling of a frame before encoding and upscaling of decoded frame according to the coding setting parameters, and a noise suppression unit.

Claims

1. A method of real-time encoding a digitized sequence of video frames using a codec with high compression efficiency, comprising steps of:

dividing a video frame into macroblocks of pixels;

performing texture prediction using reconstructed texture of previously encoded/decoded video data;

performing a texture prediction error transform; and

performing quantization and encoding of DCT transform coefficients;

wherein quantization of DCT transform coefficients is performed using the following formula;

q=(c·

A (Quantstep)+round_const)+round_const)/2²⁰;

where c—

coefficient value;

q—

quantized coefficient value;

A—

a constant depending on quantization parameter index;

round_const—

rounding control;

0.5 sign (c), if |c|<

20·

2²⁰/A (Quantstep) and 0.25 sign (c), if |c|≧

20·

2²⁰/A (Quantstep).

2. The method of claim 1, comprising a step of downscaling before encoding of the video frame using bilinear interpolation.

3. The method of claim 1, comprising a step of controlling parameters of encoded frames.

4. The method of claim 1, comprising a step of controlling frame encoding time and CPU load.

5. The method of claim 1, comprising a step of selecting best parameters and encoding mode for macroblock coding based on preset coding parameters and codec working parameters.

6. The method of claim 1, comprising a step of noise suppression.

7. The method of claim 5, wherein the encoding mode is a low-complexity 3-dimensional data coding.

8. The method of claim 5, wherein the encoding mode is motion compensation.

9. The method of claim 8, wherein frame encoding starts with choosing a best prediction mode.

10. The method of claim 9, wherein the prediction mode is inter prediction mode predicting block pixels using reconstructed texture of previously coded/decoded frames and specifying block motion vectors.

11. The method of claim 9, wherein the prediction mode is intra prediction mode predicting block pixels using reconstructed texture of previously coded/decoded blocks of current frame and specifying prediction method.

12. The method of claim 11, comprising wavelet transform, wherein resulting wavelet transform coefficients are compressed by context-based entropy coding.

13. The method of claim 12, wherein uniform quantization with constant step size is applied to all wavelet transform coefficients.

20. The method of claim 1, wherein encoding of DCT transform coefficients is performed by arithmetic coding based on two-dimensional contest/position-depending modeling.

21. A method of decoding of sequence of video frames encoded according to claim 13, comprising steps of:

arithmetic decoding;

decoding coded block pattern of macroblock mode and texture using arithmetic context-based modeling;

decoding texture prediction error using arithmetic context-based modeling;

calculating prediction for motion vectors; and

decoding motion vectors using context-based arithmetic modeling.

22. The method of decoding of claim 21, comprising internal bilinear upscaling correlated with bilinear downscaling provided at the time of encoding.

23. The method of decoding of claim 21, wherein the texture prediction error is provided by inverse transform and dequantization correlated with corresponding encoding procedures.

24. The method of decoding of claim 21, comprising step of deblocking of decoded video frame using at least one of horizontal and vertical deblocking passes for smoothing of sequence of video frame border points.

14. A method of real-time encoding a digitized sequence of video frames using a codec with high compression efficiency, comprising steps of:

dividing a video frame into macroblocks of pixels;

performing texture prediction using reconstructed texture of previously encoded/decoded video data;

selecting best parameters for macroblock encoding based on preset coding parameters and codec working parameters for macroblock coding;

selecting intra prediction mode predicting block pixels using reconstructed texture of previously coded/decoded blocks of current frame and specifying prediction method;

selecting motion compensation mode for macroblock encoding;

performing a texture prediction error transform; and

performing quantization and encoding DCT transform coefficients using wavelet transform,

wherein resulting wavelet transform coefficients are compressed by the context-based entropy coding is based on contexts including three neighboring coefficients and one root coefficient, the value of each coefficient being coded arithmetically, and the context-based entropy coding of absolute value of transform coefficients is determined in accordance with the following algorithm;

set a current value of coefficient=0;

construct bits of context for entropy-coded binary value;

bit 0=abs(n1)>

current value, where abs (n1) is absolute value of the first neighboring coefficient;

bit 1=abs(n2)>

current value, where abs (n2) is absolute value of the second neighboring coefficient;

bit 2=abs(n3)>

current value, where abs (n3) is absolute value of the third neighboring coefficient;

bit 3=0 (root coefficient=0);

bits 4,5=(abs(n3)*3+abs(n1)*3+abs(n2)*2+4)/8={0, 1, 2, 3 or greater};

using the context, send bit “

1”

if abs(coefficient)=current value, otherwise send bit “

0”

;

increment the current value;

if abs(coefficient)≠

current value, repeat the construct step;

if abs(coefficient)>

0, sent a sign,

wherein the bits of context number are;

Bit 0=(n1>

0);

Bit 1=(n3>

0).

15. A method of real-time encoding a digitized sequence of video frames using a codec with high compression efficiency, comprising steps of:

dividing a video frame into macroblocks of pixels;

performing texture prediction using reconstructed texture of previously encoded/decoded video data;

selecting best parameters for macroblock encoding based on preset coding parameters and codec working parameters;

selecting motion compensation mode for macroblock encoding;

selecting inter prediction mode for frame encoding predicting block pixels using reconstructed texture of previously coded/decoded blocks of current frame and specifying prediction method,

performing a texture prediction error transform;

encoding DCT transform coefficients using wavelet transform,

compressing resulting wavelet transform coefficients by context-based entropy coding, and

applying an uniform quantization with constant step size to all wavelet transform coefficients;

wherein the uniform quantization of transform coefficients is presented as follows;

q_Coeff=round (Coeff/Quantizer),

wherein Coeff—

wavelet transform coefficient;

q_Coeff—

quantized coefficient;

Quantizer—

quantization step size.

16. The method of claim 15, comprising step of motion estimation for calculating-components of motion vectors.

17. The method of claim 16, wherein the motion vectors are calculated with quarter-pel accuracy.

19. The method of claim 16, comprising step of arithmetic encoding of motion vector prediction difference.

18. A method of real-time encoding a digitized sequence of video frames using a codec with high compression efficiency, comprising steps of:

dividing a video frame into macroblocks of pixels;

performing texture prediction error transform;

selecting best parameters for macroblock coding based on preset coding parameters and codec working parameters;

selecting motion compensation mode for encoding;

performing a texture prediction error transform;

performing quantization and encoding of DCT transform coefficients; and

performing calculation of components of motion vectors using motion estimation, wherein the motion estimation comprises;

calculating motion vectors MV(wb, hb, CF, RF) with integer-pel accuracy using previously calculated motion data;

calculating motion vectors MV(wb, hb, CF, RF) [block_x][block_y]performing inverse logarithmic motion search with parameters block_x, block_y, current_range;

performing motion vector refinement choosing from sets of neighboring motion vectors MVNeiborhood (wb, hb, CF, RF)[block_x][block_y]elements (mvx, mvy) that provide minimum value of motion vector weight function Q(mvx, mvy, CF, RF, wb, hb, block_x, block_y);

performing motion vector estimation with quarter-pel accuracy based on results of motion vector estimation with integer-pel accuracy by changing components of the integer-pel accuracy motion vector MV(wb, hb, CF, RF) [block_x][block_y]in range [−

¾

;

+¾

] with a step ¼

; and

calculating motion vectors MV(wb, hb, CF, RF) with quarter-pel accuracy by sequentially applying the motion estimation steps with integer-pel accuracy and with quarter-pel accuracy, wherein CF—

current frame with horizontal coordinate x and vertical coordinate y;

RF—

reference frame with horizontal coordinate x and vertical coordinate y;

wb—

width of the blocks for which motion estimation is performed;

hb—

height of the blocks for which motion estimation is performed;

W—

a multiple of wb, current and reference frame width;

H—

a multiple of hb, current and reference frame height;

Q(mvx, mvy, CF, RF, wb, hb, block_x, block_y)—

motion vector weight calculation function;

MV(wb, hb, CF, RF)[block_x][block_y]—

motion vector (i.e. pair (mvx,mvy) of integers) corresponding to the frame CF and reference frame RF for a block of width wb, height hb, which left-top corner is located at a pixel with horizontal coordinate block_x and vertical coordinate block_y;

MV(wb, hb, CF, HF)—

a set of motion vectors MV(wb, hb, CF, RF)[block_x][block_y] for;

block_x=0, wb, 2·

wb, 3·

wb, . . . , block_x<

W, and block_y=0, hb, 2·

hb, 3·

hb, . . . , block_y<

H;

MVNeighborhood (wb, hb, CF, RF)[bloc_y][block_y]—

a set of neighboring motion vectors MV(wb, hb, CF, RF)[nx][ny], where nx may be equal to block_x—

wb, block_x block_x+wb and ny may be equal to block_y−

hb block_y, block_y+hb, and nx≧

0, ny≧

0, nx≦

W−

wb, ny≦

H−

hb.