Method for video transcoding with adaptive frame rate control

• US 20050058197A1
• Filed: 09/12/2003
• Published: 03/17/2005
• Est. Priority Date: 09/12/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Liu, Shan, Lu, Ligang
• Granted Patent: US 7,330,509 B2
• Time in Patent Office: Days
• US Class Current: 375/240.30
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream

Abstract

A video transcoding method is provided for transcoding a first signal stream compressed by a first coding scheme to a second signal stream compressed by a second coding scheme. The method employs an adaptive frame rate and a joint temporal-spatial rate control technique, such that the overall quality of compressed MPEG video can be significantly enhanced when the transcoding is controlled in the joint temporal (picture or frame rate) and spatial (quantization) domains. One embodiment considers transcoding from high bit rate video with larger image size (e.g. 4CIF/4SIF, CIF) coded by one coding technique, e.g., MPEG-2 to a lower bit rate video with smaller image size (e.g. CIF, QCIF) coded by the same or another coding technique, e.g., MPEG-4.

Claims

1. A method for transcoding an input compressed signal stream to an output compressed signal stream comprising down-sampling data elements of signal units of the input compressed signal stream, and using a joint temporal-spatial rate control to convert the input compressed signal stream to the output compressed signal stream by adjusting the signal unit rate, which is the number of signal units per unit time, and the signal unit quantization step size, which is the defined manner in which each signal unit is quantized and coded during data compression, simultaneously according to the channel bandwidth, to achieve a quality tradeoff between temporal and spatial resolution for the transcoded output compressed signal stream.

2. The method of claim 1, further comprising down-sampling temporal change vectors of the input compressed signal stream.

3. The method of claim 2, further comprising comparing candidate temporal change vectors and prediction modes decoded from the first compressed signal stream and selecting the temporal change vector with the minimum sum of absolute differences as the resulting temporal change vector.

4. A method for transcoding an input compressed video stream to an output compressed video stream comprising down-sampling picture pixels of the input compressed video stream, and using a joint temporal-spatial rate control to convert the input compressed video stream to the output compressed video stream by adjusting the picture rate, which is the number of pictures per unit time, and simultaneously adjusting the picture quantization step size, which is the defined manner in which each picture is quantized and coded during data compression, according to the channel bandwidth, to achieve a quality tradeoff between temporal and spatial resolution for the transcoded output compressed video stream.

5. The method of claim 4, further comprising down-sampling motion vectors of the input compressed video stream.

6. The method of claim 5, further comprising comparing candidate motion vectors and prediction modes decoded from the first compressed video stream and selecting the motion vector with the minimum sum of absolute differences as the resulting motion vector.

7. The method of claim 4, wherein the video streams are compressed pursuant to the standards of the Moving Picture Experts Group (MPEG), and within a Group of Pictures (GOP) three types of pictures are distinguished according to the compression method which is used, Intra-mode pictures (I pictures) which are compressed independently of any other picture, Predictively motion-compensated pictures (P pictures) which are reconstructed from the compressed data in that picture and two most recently reconstructed fields from previously displayed I or P pictures, and Bidirectionally motion-compensated pictures (B pictures) which are reconstructed from the compressed data in that picture plus two reconstructed fields from previously displayed I or P pictures and two reconstructed fields from I or P pictures that will be displayed in the future, wherein I and P pictures are termed reference pictures because they are used to reconstruct other pictures

8. The method of claim 7, further including transcoding all reference I and P pictures because of the reuse of motion vectors and to maintain the prediction sequential order, while not transcoding and skipping non-reference B pictures that carry less information.

9. The method of claim 8, wherein skipped B pictures are reconstructed at the decoder to ensure a full frame rate playback.

10. The method of claim 7, further including:

determining whether a picture is an I picture, a B picture or a P picture type from the picture header information;

transcoding all I pictures; and

if a series of pictures comprises a (B, P) pair, which starts with a B picture, may have zero, one or more intermediate B pictures, and ends with a P picture, applying adaptive picture rate transcoding with joint temporal-spatial rate control to the series of pictures of the (B, P) pair.

11. The method of claim 10, wherein the adaptive picture rate transcoding with joint temporal-spatial rate control comprises the following steps:

a. transcoding a P picture to obtain its rate denoted by R(P), which is the bit count consumed by transcoding the P picture, and its distortion denoted by D(P), such as PNR (Peak Signal-to-Noise Ratio) or SAD (Sum of Absolute Differences), in comparison with the input down-sampled video picture;

b. transcoding the following B picture to obtain its rate denoted by R(B), which is the bit count consumed by transcoding the B picture, and its distortion denoted by D(B), such as PNR (Peak Signal-to-Noise Ratio) or SAD (Sum of Absolute Differences), in comparison with the input down-sampled video picture;

c. summing R(B) and R(P) as a target bit count for the P picture, re-transcode it at one half of the incoming picture rate to obtain R(P₂) and D(P₂);

d. reconstructing a skipped picture and calculate its distortion D(S), while the rate R(S) is zero, wherein the B picture is the skipped picture (S) and is reconstructed by averaging the previous I or P picture, which is the last coded picture of the previous sub-GOP, and picture P₂ obtained in step c;

e. comparing the sum of distortion D(B)+D(P) with the sum of distortion D(S)+D(P₂).

12. The method of claim 11, following the comparing step, selecting the smaller sum of distortion is preferred, and if the sum of distortion D(B)+D(P) is chosen, both frames are transcoded, otherwise, only the P frame is transcoded, store the P frame in a reference frame buffer, and proceed to the next sub-GOP.