Method and apparatus for perceptual model based video compression

• US 20040161034A1
• Filed: 02/14/2003
• Published: 08/19/2004
• Est. Priority Date: 02/14/2003
• Status: Abandoned Application

• Current Assignee: XVD Corp. (XVD Technology Holdings Ltd.)
• Original Assignee: XVD Corp. (XVD Technology Holdings Ltd.)
• Inventors: Asnis, Ilya, Morozov, Andrei
• Application Number:
  US10/366,863
• Publication Number:
  US 20040161034A1
• Time in Patent Office: Days
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and apparatus for perceptual model based video compression calculates a bitrate value that follows with stabilizing delay the actual bitrates of previous frames. A current quantization coefficient is determined with the calculated bitrate value and a perceptual model. The current quantization coefficient'"'"'s rate of change is limited based on a previous quantization coefficient. After the current quantization coefficient has been calculated and limited, a current frame is encoded with the limited current quantization coefficient.

Claims

1. A computer implemented method comprising:

calculating a bitrate value that follows with stabilizing delay the actual bitrates of previous frames;

determining a current quantization coefficient with the calculated bitrate value and a perceptual model;

limiting the current quantization coefficient'"'"'s rate of change based on a previous quantization coefficient; and

encoding a frame with the limited current quantization coefficient.

2. The computer implemented method of claim 1 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

3. The computer implemented method of claim 1 wherein the current quantization coefficient'"'"'s rate of change is limited within 0.5*Q_N−

1<

=Q_CALC<

=2*Q<sub>N−

1</sub>, wherein Q<sub>N−

1</sub> is the Q determined for a preceding frame.

4. The computer implemented method of claim 1 wherein the bitrate value =RNT_N +RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

5. A computer implemented method comprising:

determining an encoding complexity control scalar based on a perceptual model with a stabilized time weighed preceding encodings based bitrate;

bounding the determined encoding complexity control scalar based on a set of one or more previous encoding complexity control scalars used to encode a set of one or more preceding frames; and

encoding a current frame using the bounded encoding complexity control scalar.

6. The computer implemented method of claim 5 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

7. The computer implemented method of claim 5 wherein the encoding complexity control scalar is bounded by 0.5*Q_N−

1<

=Q_CALC<

=2*Q<sub>N−

1</sub>, wherein Q<sub>N−

1</sub> is the Q determined for a preceding frame.

8. The computer implemented method of claim 5 wherein the stabilized time weighed preceding encodings based bitrate=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RNN is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

9. A computer implemented method comprising:

establishing a target bit utilization range for a duration of a plurality of video frames based on information known about the plurality of video frames;

calculating a model reaction parameter within the target bit utilization range based on the remaining available bits for the plurality of video frames;

calculating a perceptual model correction parameters with the calculated current frame'"'"'s budget and the remaining available bits for the plurality of video frames; and

modifying a current perceptual model defining parameter in accordance with the calculated perceptual model correction parameters, a preceding frame'"'"'s bitrate, and the remaining available bits for the plurality of video frames.

10. The computer implemented method of claim 9 wherein the model reaction parameter is the quotient of the number of bits per frame and a local bit utilization range.

11. The computer implemented method of claim 9 wherein the perceptual model correction parameters include a bitrate oscillation damping variable (D_R) and a bit budget control variable (D_B), calculated according to the following equations:

D_R=Model reaction parameter/Bytes per frame (D_R being a bitrate oscillation damping variable), andD_B=(Model reaction parameter)²/Bytes per frame (D_B being bit budget control variable).

12. A computer implemented method comprising:

determining an encoding complexity control scalar with a perceptual model and a preceding encodings based bitrate to encode a set of one or more frames in a video;

updating the preceding encodings based bitrate after encoding each frame of the set of frames in the video; and

shifting the perceptual model in accordance with controlling bit utilization over the video'"'"'s duration.

13. The computer implemented method of claim 12 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

14. The computer implemented method of claim 12 wherein the stabilized time weighed preceding encodings based bitrate=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K₃ and K₄ are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

15. A computer implemented method comprising:

encoding a plurality of frames of a video for consistent perceived visual quality of the video with an encoding complexity control scalar calculated in accordance with a perceptual model and adjusted for each of the plurality of frames in accordance with an average bitrate of a set of one or more preceding encoded frames, the average bitrate being adjusted to compensate for preceding encoded frames with a bitrate exceeding a certain threshold; and

modifying the perceptual model to control bit utilization for encoding the video.

16. The computer implemented method of claim 15 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

17. The computer implemented of claim 15 wherein the average bitrate is=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

18. An apparatus comprising:

an encoding complexity control scalar generation unit including

a perceptual model parameter unit to host perceptual model parameters,

an input bitrate calculation unit to calculate an input bitrate based on previously encoded frames bitrates, and

an encoding complexity control scalar calculation unit coupled with the perceptual model parameter unit and the input bitrate calculation unit, the encoding complexity control scalar calculation unit to calculate an encoding complexity control scalar with perceptual model parameters from the perceptual model parameter unit and an input bitrate from the input bitrate calculation unit; and

a video compression unit coupled with the encoding complexity generation unit to receive an encoding complexity control scalar and to compress video, the video compression unit including

a quantization unit,

a motion compensation unit, and

an encoding unit.

19. The apparatus of claim 18 wherein the quantization unit is a DCT unit.

20. The apparatus of claim 18 further comprising an optical medium reading module coupled with the video compression unit.

21. A machine-readable medium having a set of instructions to cause a device to perform the following operations:

calculating a bitrate value that follows with stabilizing delay the actual bitrates of previous frames;

determining a current quantization coefficient with the calculated bitrate value and a perceptual model;

limiting the current quantization coefficient'"'"'s rate of change based on a previous quantization coefficient; and

encoding a frame with the limited current quantization coefficient.

22. The machine-readable medium of claim 21 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

23. The machine-readable medium of claim 21 wherein the current quantization coefficient'"'"'s rate of change is limited within 0.5*Q_N−

1<

=Q_CALC<

=2*Q<sub>N−

1</sub>, wherein Q<sub>N−

1</sub> is the Q determined for a preceding frame.

24. The machine-readable medium of claim 21 wherein the bitrate value=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

25. A machine-readable medium having a set of instructions to cause a device to perform the following operations:

determining an encoding complexity control scalar based on a perceptual model with a stabilized time weighed preceding encodings based bitrate;

bounding the determined encoding complexity control scalar based on a set of one or more previous encoding complexity control scalars used to encode a set of one or more preceding frames; and

encoding a current frame using the bounded encoding complexity control scalar.

26. The machine-readable medium of claim 25 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

27. The machine-readable medium of claim 25 wherein the encoding complexity control scalar is bounded by 0.5*Q_N−

1<

=Q_CALC<

=2*Q<sub>N−

1</sub>, wherein Q<sub>N−

1</sub> is the Q determined for a preceding frame.

28. The machine-readable medium of claim 25 wherein the stabilized time weighed preceding encodings based bitrate=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub> *K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

29. A machine-readable medium having a set of instructions to cause a device to perform the following operations:

establishing a target bit utilization range for a duration of a plurality of video frames based on information known about the plurality of video frames;

calculating a model reaction parameter within the target bit utilization range based on the remaining available bits for the plurality of video frames;

calculating a perceptual model correction parameters with the calculated current frame'"'"'s budget and the remaining available bits for the plurality of video frames; and

modifying a current perceptual model defining parameter in accordance with the calculated perceptual model correction parameters, a preceding frame'"'"'s bitrate, and the remaining available bits for the plurality of video frames.

30. The machine-readable medium of claim 29 wherein the model reaction parameter is the quotient of the number of bits per frame and a local bit utilization range.

31. The machine-readable medium of claim 29 wherein the perceptual model correction parameters include a bitrate oscillation damping variable (D_R) and a bit budget control variable (D_B), calculated according to the following equations:

D_R=Model reaction parameter/Bytes per frame (D_R being a bitrate oscillation damping variable), andD_B=(Model reaction parameter)²/Bytes per frame (D_B being bit budget control variable).

32. A machine-readable medium having a set of instructions to cause a device to perform the following operations:

determining an encoding complexity control scalar with a perceptual model and a preceding encodings based bitrate to encode a set of one or more frames in a video;

updating the preceding encodings based bitrate after encoding each frame of the set of frames in the video; and

shifting the perceptual model in accordance with controlling bit utilization over the video'"'"'s duration.

33. The machine-readable medium of claim 32 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

34. The machine-readable medium of claim 32 wherein the stabilized time weighed preceding encodings based bitrate=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

36. The machine-readable medium of claim 34 wherein the perceptual model is defined by the following equation:

Q_PM*(R_CALC/R_PM)^P.

37. The machine-readable medium of claim 34 wherein the average bitrate is=RNT_N+RL_N−

RNTL_N, wherein RNT_N=RNT<sub>N−

1</sub>*K1+RN_N*K2, where K1 and K2 are coefficients which define how fast a system reacts to sudden difficulty changes between frames and RN_N is equal to the last previously encoded non-transitional frame bitrate, RL_N=RL<sub>N−

1</sub>*K3+R_N*K4 where R_N is the previously encoded frame bitrate, K3 and K4 are coefficients which define a slow reaction infinite response filter, and RNTL_N=RNTL<sub>N−

1</sub>*K3+RN_N*K4.

35. A machine-readable medium having a set of instructions to cause a device to perform the following operations:

encoding a plurality of frames of a video for consistent perceived visual quality of the video with an encoding complexity control scalar calculated in accordance with a perceptual model and adjusted for each of the plurality of frames in accordance with an average bitrate of a set of one or more preceding encoded frames, the average bitrate being adjusted to compensate for preceding encoded frames with a bitrate exceeding a certain threshold; and

modifying the perceptual model to control bit utilization for encoding the video.