Scalable video transcoding

• US 20050147163A1
• Filed: 12/30/2003
• Published: 07/07/2005
• Est. Priority Date: 12/30/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chen, Qi, Li, Shipeng, Wu, Feng, Yuan, Lujun
• Granted Patent: US 7,391,809 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A scalable video transcoder with limited drifting error is disclosed. The transcoder accumulates re-quantization errors of I-frames or P-frames. The accumulated errors are only selectively introduced into the transcoding loop in every block to compensate for transcoding errors. A threshold algorithm adaptively utilizes the accumulated errors at block level to control drifting error at an acceptable limited level. Transcoder complexity is scalable because a DCT transform may be removed from the transcoder loop when the accumulated errors are not used for compensation. The re-quantization process can be simply implemented by look-up table(s), which significantly reduces the complexity of the re-quantization process.

Claims

1. A method for transcoding input video data encoded at an initial bit rate into output video data encoded at a target bit rate, the input video data and the output video data comprising one or more of an intra frame (I-frame), a predictive frame (P-frame), and a bidirectional frame (B-frame), the method implemented by a processor, the method comprising:

accumulating transcoding error associated with transcoding the input video into the output video data to generate accumulated transcoding error;

motion-compensating the accumulated transcoding error to generate

motion-compensated accumulated transcoding error;

error-compensating the input video data with the motion-compensated accumulated transcoding error to generate error-compensated video data;

selectively disabling one or more of the accumulating, motion-compensating, and error-compensating operations in response to detecting one or more conditions related to transcoding the input video data.

2. A method as recited in claim 1 wherein the selectively disabling operation comprises:

determining whether the input video data comprises bidirectional frame (B-frame) video data; and

if the input video data comprises B-frame video data,

applying a B-frame switching module operable to disable the accumulating, motion-compensating, and error-compensating operations from being applied to the B-frame video data, and

inserting B-frame bypass data into the output video data, the B-frame bypass data indicating that the B-frame video data is not transcoded.

3. A method as recited in claim 2 wherein the selectively disabling operation further comprises:

determining whether utilization of the processor is greater than a predetermined maximum utilization; and

applying the B-frame switching module only if the utilization of the processor is greater than the predetermined maximum and the input video data comprises B-frame video data.

4. A method as recited claim 1 wherein the selectively disabling operation comprises:

determining whether the motion-compensated accumulated transcoding error is less than a threshold value; and

if the motion-compensated accumulated transcoding error is less than the threshold value, applying a compensating switching module to prevent the error-compensating operation.

5. A method as recited in claim 4 wherein the determining whether the motion-compensated accumulated transcoding error is less than the threshold value comprises calculating an error metric, the error metric being a function of accumulated errors of pixels in a video block.

6. A method as recited in claim 5 wherein calculating the error metric comprises calculating an absolute error of an 8×

8 video block according to an equation, the equation being;

$S=\underset{i=0}{\overset{7}{\sum <br><br>}}<br><br>\underset{j=0}{\overset{7}{\sum <br><br>}}<br><br><br><br>e<br><br>\left(i,j\right)<br><br>,<br><br>\text{\hspace{1em}<br><br>}$

wherein s represents the absolute error, e(i, j) represents accumulated error of each pixel in the 8×

8 block, and i and j represent indices to pixels in the 8×

8 block.

7. A method as recited in claim 4 further comprising setting the threshold value equal to one of a plurality of predetermined threshold values based on one or more of frame-type of the input video data and whether the error-compensating operation is disabled.

8. A method as recited in claim 1 wherein the selectively disabling operation comprises:

determining whether the input video data comprises intra-frame (I-frame) video data; and

if the input video data comprises I-frame video data, applying a compensating switching module to disable the error-compensating operation.

9. A method as recited in claim 1 wherein the selectively disabling operation comprises applying an open-loop switching module to prevent the accumulating, motion-compensating, and error-compensating operations.

10. A method as recited in claim 1 further comprising determining an initial re-quantization parameter to apply to the error-compensated video data to achieve the target bit rate, the determining comprising applying an equation, the equation being:

$Q^{\mathrm{\prime <br><br>}}=\frac{\mathrm{RQ}}{R^{\mathrm{\prime <br><br>}}},$

wherein Q′

represents the initial re-quantization parameter, R represents the initial bit rate, Q represents a quantization parameter associated with the input video data encoded at the initial bit rate, and R′

represents the target bit rate.

11. A method as recited in claim 10 further comprising determining a secondary re-quantizing parameter according to an equation, the equation being:

$q_j=\frac{\sqrt[3]{x_j}\times <br><br>\underset{n=1}{\overset{N}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sqrt[3]{x_n^2}}{B^{\mathrm{\prime <br><br>}}},n=1,2,\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},N,$

wherein N represents a number of frames in the input video data, q_j represents the quantization parameter for a jth macroblock in a frame of the input video data, B′

represents a target number of bits in a frame of the output video data, n represents the total number of macroblocks in a frame, and x_n represents a complexity of the nth macroblock.

12. A method as recited in claim 11 further comprising calculating the target number of bits, B′

, according to an equation, the equation being;

B′

=α

B,

wherein α

represents a ratio of the target bit rate to the initial bit rate and B represents the number of bits of the current frame in the input video data.

13. A method as recited in claim 11 further comprising calculating the target number of bits, B′

, according to an equation, the equation being;

B′

=wα

B,

wherein α

represents a ratio of the target bit rate to the initial bit rate, w represents weighting factor, and B represents the number of bits of the current frame in the input video data.

14. A method as recited in claim 12 further comprising determining a difference between the target number of bits and an actual number of bits encoded in a frame of the output video data, the determining comprising applying an equation, the equation being:

$\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>B=B^{\mathrm{\prime <br><br>}}-\underset{n=1}{\overset{j-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>r_n<br><br>\left(q_n\right),$

wherein Δ

B represents the difference between the target number of bits and the actual number of bits encoded in the frame of output video data, B′

represents the target number of bits, r_n represents the number of bits in an nth macroblock of the frame, q_n represents the quantization parameter of the nth macroblock of the frame, j represents a number of macroblocks in the frame, and n is an index variable that indexes sequentially through the macroblocks in the frame.

15. A method as recited in claim 14 further comprising:

determining whether Δ

B is greater than zero; and

if Δ

B is greater than zero, recalculating the quantization parameter q_j according to an equation, the equation being;

$q_j=\frac{\sqrt[3]{x_j}\times <br><br>\underset{n=1}{\overset{N}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sqrt[3]{x_n^2}}{\mathrm{\Delta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>B},n=1,2,\mathrm{\dots <br><br>}<br><br>\text{\hspace{1em}<br><br>},N.$

16. A method as recited in claim 15 further comprising:

determining a difference re-quantization parameter, Δ

qj, according to an equation, the equation being;

Δ

q_j=q_j−

Q′

; and

adding Δ

q_j to Q′

to generate a target re-quantization parameter.

17. A method as recited in claim 1 further comprising re-quantizing the error-compensated video data, the re-quantizing comprising looking up a target quantization parameter in a look-up table having a plurality of quantization parameters to achieve the target bit rate.

18. A method as recited in claim 1 wherein the accumulating operation comprises accumulating transcoding error in a frequency domain.

19. A system for transcoding input video data encoded at an initial bit rate into output video data encoded at a target bit rate, the input video data and the output video data comprising one or more of an intra frame (I-frame), a predictive frame (P-frame), and a bidirectional frame (B-frame), the system comprising:

an accumulating module accumulating transcoding error associated with transcoding the input video into the output video data to generate accumulated transcoding error;

a motion compensation module compensating the accumulated transcoding error with motion estimation data to generate motion-compensated accumulated transcoding error;

an error-compensating module compensating the input video data with the motion-compensated accumulated transcoding error to generate error-compensated video data;

a re-quantization module controlling a re-quantization level to achieve the target bit rate;

a compensation switching module operable to disable the error-compensating module in response to the motion-compensated accumulated transcoding error being less than a threshold value.

20. A system as recited in claim 19 further comprising a B-frame switching module operable to disable the accumulating module and the compensating module if the input video data comprises bidirectional frame (B-frame) video data.

21. A system as recited in claim 19 wherein the quantization parameter control module uses rate information and a quantization parameter in the input video data to determine a re-quantization parameter.

22. A system as recited in claim 21 wherein the re-quantization module comprises a look-up table, the look-up table outputting a re-quantization level and a re-quantization error corresponding to the target bit rate if the input video data comprises a value in a first range.

23. A system as recited in claim 22 wherein the re-quantization module further comprises:

a dequantization/quantization path, the dequantization/quantization path used to determine the re-quantization level and the re-quantization error associated with the target bit rate if the input video data comprises a value in a second range; and

a level switching module operable to detect the value of the input video data and select the look-up table if the value is in the first range and otherwise select the dequantization/quantization path.

24. A system as recited in claim 22 wherein the look-up table is user-modifiable.

25. A system as recited in claim 23 wherein the first range and the second range are user-selectable.

26. A system as recited in claim 23 wherein the re-quantization module further comprises an error calculating module, the error calculating module determining transcoding error as a function of an element in the input video data, an initial quantization parameter associated with the initial bit rate, a target quantization parameter associated with the target bit rate, and a quantization parameter corresponding to the element in the video data.

27. A system as recited in claim 26 wherein the re-quantization module re-quantizes an element of the input video data to a corresponding element of the output video data using an equation, the equation being:

$X_3=\frac{X_3^{\mathrm{\prime <br><br>}}\times <br><br>q_m}{16},$

wherein X₃ represents the corresponding element of the output video data, X₃′

represents a preliminary re-quantized element obtained from the look-up table based on the element of the input video data, and qm represents the quantization parameter corresponding to the element in the input video data.

28. A system as recited in claim 26 wherein the error calculating module determines the transcoding error based on an equation, the equation being:

$X_5=\frac{X_5^{\mathrm{\prime <br><br>}}\times <br><br>q_m}{16},$

wherein X₅ represents the transcoding error associated with the element, X₅′

represents a preliminary re-quantization error obtained from the look-up table based on the element of the input video data, and q_m represents the quantization parameter corresponding to the element in the input video data.

29. A system as recited in claim 19 wherein the input video data comprises video data encoded according to a Motion Picture Experts Group-2 (MPEG-2) video format and the output video data comprises video data encoded according to an MPEG-2 format.

30. A system for transcoding video data comprising:

input video data encoded at an initial bit rate;

means for transcoding the input video data to generate corresponding output video data encoded at a target bit rate, the means for transcoding comprising a re-quantization module including a look-up table having dynamically selectable quantization parameters corresponding to associated bit rates.

31. A system as recited in claim 30 wherein the means for transcoding further comprises an error-compensating module for compensating the input video data with motion-compensated accumulated transcoding error, a motion-compensating module operable to generate the motion-compensated accumulated transcoding error, an accumulating module operable to accumulate transcoding error related to differences between the input video data and the output video data, and one or more switching modules operable to disable one or more of the error-compensating module and the accumulating module.

32. A computer-readable medium having stored thereon computer-executable instructions for causing a processor to perform a method comprising:

accumulating transcoding error associated with transcoding input video having an associated initial bit rate into output video data having an associated target bit rate, to generate accumulated transcoding error;

motion-compensating the accumulated transcoding error to generate motion-compensated accumulated transcoding error;

error-compensating the input video data with the motion-compensated accumulated transcoding error to generate error-compensated video data;

selectively disabling one or more of the accumulating and the error-compensating operations in response to detecting one or more conditions related to transcoding the input video data.

33. A computer-readable medium as recited in claim 32, wherein the selectively disabling operation comprises disabling the error-compensating operation if the motion-compensated accumulated transcoding error is less than an error threshold.

34. A computer-readable medium as recited in claim 32, wherein the selectively disabling operation comprises:

selecting an error threshold from a plurality of predetermined error thresholds; and

disabling the error-compensating operation if the motion-compensated accumulated transcoding error is less than the error threshold.

35. A computer-readable medium as recited in claim 32, wherein the selectively disabling operation comprises:

determining whether the input video data comprises bidirectional frame (B-frame) video data;

determining whether the processor is being utilized at a utilization rate greater than a predetermined utilization rate; and

if the input video data comprises B-frame video data and the processor is being utilized at a utilization rate greater than a predetermined utilization rate, disabling the accumulating operation and the error-compensating operation.

36. A computer-readable medium as recited in claim 32, the method further comprising re-quantizing the error-compensated video data using a re-quantization parameter corresponding to the target bit rate.

37. A computer-readable medium as recited in claim 32, the method further comprising:

determining a preliminary re-quantization parameter based on a function of the initial bit rate, the target bit rate, and an initial quantization parameter corresponding to the initial bit rate of the input video data;

determining a target number of bits in a frame of the output video data, the target number of bits being based on a function of an initial number of bits in a frame of the input video data, the initial bit rate, and the target bit rate;

determining a macroblock re-quantization parameter associated with a macroblock in the frame of the output video data, the macroblock re-quantization parameter being determined based on a function of a complexity value associated with the macroblock and the target number of bits in the frame of the output video data;

determining a re-quantization difference by subtracting the preliminary quantization parameter from the macroblock re-quantization parameter;

adding the re-quantization difference to the preliminary quantization parameter to generate a target re-quantization parameter; and

re-quantizing the error-compensated video data using the target re-quantization parameter.

38. A computer-readable medium as recited in claim 37, wherein the determining a target number of bits in a frame of the output video data comprises calculating the target number of bits according to an equation, the equation being:

B′

=(R′

/R)×

B,

wherein B′

represents the target number of bits in the frame of the output video data, R′

represents the target bit rate, R represents the initial bit rate, and B represents the initial number of bits in the frame of the input video data.

39. A computer-readable medium as recited in claim 37, wherein the determining a preliminary re-quantization parameter comprises calculating the preliminary re-quantization parameter according to an equation, the equation being:

Q′

=(R′

/R)×

Q,

wherein Q′

represents the preliminary re-quantization parameter, R′

represents the target bit rate, R represents the initial bit rate, and Q represents the initial quantization parameter associated with the initial bit rate of the input video data.

40. A computer-readable medium as recited in claim 32, wherein the accumulating operation comprises subtracting discrete cosine transform coefficients associated with the output video data from discrete cosine transform coefficients associated with the input video data.