Real-time video coding/decoding

• US 8,023,562 B2
• Filed: 09/05/2008
• Issued: 09/20/2011
• Est. Priority Date: 09/07/2007
• Status: Active Grant

• Current Assignee: Beamr Imaging Ltd.
• Original Assignee: Vanguard Software Solutions, Inc. (Dolby Laboratories Incorporated)
• Inventors: Terterov, Nikolay, Zheludkov, Alexander, Martemyanov, Alexey
• Primary Examiner(s): AN, SHAWN S
• Application Number:
  US12/205,482
• Publication Number:
  US 20090067504A1
• Time in Patent Office: 1,110 Days
• Field of Search: 375/240.16, 375/240.15, 375/240.12, 375/240.14, 375/240.13, 382/236, 382/238, 382/232
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A video codec having a modular structure for encoding/decoding a digitized sequence of video frames in a multi-core system is described. The video codec comprises a memory unit; a multithreading engine. and a plurality of control and task modules organized in a tree structure, each module corresponding to a coding operation. The modules communicate with each other by control messages and shared memory. The control modules control all coding logic and workflow, and lower level task modules perform tasks and provide calculations upon receiving messages from the control task modules. The multithreading engine maintains context of each task and assigns at least one core to each task for execution. The method of coding/decoding comprises denoising, core motion estimation, distributed motion estimation, weighted texture prediction and error resilient decoding.

Claims

1. A method of encoding, a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the core motion estimation comprises;

dividing a current video frame and a reference video frame into blocks of pixels;

defining in the current frame an initial block B(F, X_INIT, Y_INIT) with coordinates X_INIT, Y_INIT;

selecting in the reference frame a plurality of N candidate blocks with coordinates (X_j, Y_j) where j=0, 1 . . . N, using a predetermined criterion; and

performing core motion estimation for the current frame based on similarity between pixels of a block of the current frame and pixels of blocks of the reference frame;

wherein the motion estimation comprises;

calculating values of a function of distortion Q(F, R, X_j, Y_j) for all N candidate blocks; and

comparing minimum values of Q(F, R, X_j, Y_j) with predetermined thresholds to find coordinates for motion estimation X_j=X_MIN, Y_j=Y_MIN from the function of distortion having a lowest value;

Q_MIN=Q(F, R, X_MIN, Y_MIN).

2. The method of claim 1, wherein the motion estimation algorithm comprises:

calculating Q(F,R, X_MIN+j*Dx_MIN,Y_MIN+j*Dy_MIN) for increasing j until inequality (I) is true, or a boundary of the frame is reached;

Q(F,R,X_MIN+j*Dx_MIN,Y_MIN+j*Dy_MIN)<

=Q(F,R,X_MIN+(j+1)*Dx_MIN,Y_MIN+(j+1)*Dy_MIN) 

 

(I)

 

where j=0, 1, 2 . . . J, and J−

is minimum nonnegative integer value for which the inequality (I) is true;

Dx and Dy are steps in horizontal and vertical direction;

defining parameters X_MIN=X_MIN+J*Dx_MIN, Y_MIN=Y_MIN+J*Dy_MIN;

calculating Q_MIN=Q(F, R, X_MIN, Y_MIN);

finding one resulting coordinate pair X_MIN, Y_MIN for Q_MIN=Q(F,R, X_MIN,Y_MIN) from the following coordinate pairs;

(X_MIN+Dx_MIN,Y_MIN+Dy_MIN);

(X_MIN−

Dx_MIN,Y_MIN+Dy_MIN);

(X_MIN+Dx_MIN,Y_MIN−

Dy_MIN), and

(X_MIN−

Dx_MIN,Y_MIN−

Dy_MIN).

3. A method of encoding a digitized sequence of video frames in a multi-core system, comprising distributed motion estimation for encoding successive frames using one reference frame and the core motion estimation algorithm of claim 2, wherein the distributed motion estimation comprises:

reducing frame size using reduction coefficient in each spatial dimension;

calculating and storing a plurality of spatially reduced frames;

selecting reduced block size in reduced frames as 8×

8;

calculating and storing motion vector sets for reduced frame pairs using the core motion estimation algorithm;

up-scaling the motion vectors for each block according to the reduction coefficients in each dimension and defining a rough full-pel motion vector for each block of an up-scaled frame;

calculating a full-pel motion vector by applying the core motion estimation algorithm to the rough full-pel motion vector as an additional candidate, and reducing search range according to the reduction coefficients; and

calculating a quarter-pel motion vector for each block by applying the core motion estimation algorithm to the full-pel motion vector as a starting point and a search range of 2.

4. A method of encoding a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the weighted texture prediction comprises;

performing estimate for normalized density functions for histogram H_o of the original frame and histogram H_r of the reference frame;

comparing the difference between the density function for H_o and the density function H_r with a predetermined threshold, and determining that the weighted prediction is to be applied if the difference is less than the threshold; and

calculating weighted prediction parameters A and B for at least one block of pixels P(x, y);

A*L(x+mx,y+my)+B;

A=So/Sr; and

B=Mo−

A*Mr;

where L(x, y) is luminance of the reference frame;

(mx, my) is motion vector for the block P(x, y);

So is deviation for the original frame;

Sr is deviation for the reference frame;

Mo is mean value for the original frame; and

Mr is mean value for the reference frame.

5. A method of encoding a digitized sequence of video frames in a multi-core system, the method comprising:

performing core motion estimation;

determining weighted texture prediction use from a difference between normalized density functions for luminance histograms of an original frame and of a reference frame; and

calculating parameters for weighted texture prediction;

wherein the method further comprises the step of;

pre-processing temporal denoising, including;

providing motion estimation by creating a texture prediction frame for a current frame from a previous frame and a corresponding motion vector;

performing deblocking of the prediction frame; and

modifying the current frame based on at least one pixel of the current frame and at least one pixel of the prediction frame;

I=I+sign(I−

P)·

F(abs(I−

P)),

where;

I is the pixel of the current frame,

P is same position pixel of the prediction frame after deblocking,

abs is the absolute value function,

sign is a sign function (sign(x)=−

1, if x<

0, sign(x)=0, if x=0, and sign(x)=1, if x>

0), and

F is a discrete time function.

6. The method of claim 5, wherein the discrete time function has the following properties:

F(D) is sufficiently smooth inside interval [0;

T₂];

F(0)=0;

F(D)≧

D/2 and F(D)≦

D, if Dε

[0;

T₀);

F(D)=D/2, if Dε

[T₀;

T₁); and

F(D)≦

D/2 and F(D)≧

0, if Dε

[T₁;

T₂);

where T₀, T₁ and T₂ are predetermined parameters which control strength of noise suppression, 0≦

T₀<

T₁<

T₂.

7. The method according to claim 6, wherein the de-noising is not performed if a motion vector function mv_cost exceeds a pre-defined threshold.