Apparatus and method for adjusting bitrate of coded scalable bitsteam based on multi-layer

• US 7,881,387 B2
• Filed: 10/18/2005
• Issued: 02/01/2011
• Est. Priority Date: 10/18/2004
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Jae-young, Ha, Ho-jin, Lee, Kyo-hyuk, Cha, Sang-chang, Han, Woo-jin, Lee, Bae-keun
• Primary Examiner(s): Ge; Yuzhen
• Application Number:
  US11/251,976
• Publication Number:
  US 20060083303A1
• Time in Patent Office: 1,932 Days
• Field of Search: None
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/127 Prioritisation of hardware or computational resources
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/517 by encoding
• CPC Class Codes: H04N 19/577 : Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 : involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/59 : involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/615 : using motion compensated temporal filtering [MCTF]
• CPC Class Codes: H04N 19/63 : using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/70 : characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/86 : involving reduction of coding artifacts, e.g. of blockiness

Abstract

Disclosed are a method and an apparatus for realizing signal to noise ratio (SNR) scalability in a video stream server in order to transmit a video stream in a variable network environment. An apparatus for adapting a bitrate of a coded scalable bit-stream based on multi-layers to a variable network environment, comprising a bit-stream parsing unit to parse an input bit-stream, a pre-decoding condition determining unit to determine a target transmission bitrate according to the variable network environment, a pre-decoding unit to skip at least one frame among the frames included in the parsed bit-stream according to the determined target transmission bitrate, and a bit-stream transmission unit to transmit to a client device the bit-stream that has been restructured due to frame skipping.

Claims

1. An apparatus for adapting a bitrate of an input bit-stream, comprising multi-layers, to a variable network environment, the apparatus comprising:

a processor;

a bit-stream parsing unit which parses the input bit-stream, under control of the processor;

a pre-decoding condition determining unit which determines a target transmission bitrate according to the variable network environment, under control of the processor;

a pre-decoding unit which, under control of the processor, selectively performs one of an operation of skipping at least one frame among a plurality of frames of an enhancement layer of the multi-layers included in the input bit-stream, and an operation of truncating a portion of the at least one frame of the plurality of frames of the enhancement layer, according to the target transmission bitrate, thereby generating an output bit-stream; and

a bit-stream transmission unit which transmits to a client device the output bit-stream, under control of the processor.

2. The apparatus of claim 1, wherein the pre-decoding condition determining unit determines the target transmission bitrate based on currently available bitrate information fed back to the pre-decoding condition determining unit from the client device.

3. The apparatus of claim 1, wherein, in the operation of skipping, the pre-decoding unit skips the at least one frame among the plurality of frames of the enhancement layer if the target transmission bitrate is less than a threshold bitrate.

4. The apparatus of claim 3, wherein the pre-decoding unit performs the operation of skipping the at least one enhancement layer frame, if at least one frame temporally corresponding to the at least one enhancement layer frame is present in a layer lower that the enhancement layer among the multi-layers in the input bit-stream, and

wherein the at least one frame in the lower layer has a lower frame-rate or a lower pixel resolution than the at least one frame in the enhancement layer.

5. The apparatus of claim 4, wherein the pre-decoding unit performs the operation of skipping the at least one enhancement layer frame if the at least one enhancement layer frame is not used as a reference frame of another frame in frame prediction.

6. The apparatus of claim 1, wherein, if the pre-decoding unit performs the operation of skipping the at least one enhancement layer frame, the pre-decoding unit records “

0”

in a field included in each of the skipped at least one enhancement layer frame to indicate a size of texture data of the each of the skipped at least one enhancement layer frame.

7. An apparatus for adapting a bitrate of an input bit-stream, comprising multi-layers, to a variable network environment, the apparatus comprising:

a processor;

a bit-stream parsing unit which parses the input bit-stream, under control of the processor;

a pre-decoding condition determining unit which determines a target transmission bitrate according to the variable network environment, under control of the processor;

a pre-decoding unit which, under control of the processor, skips at least one frame among a plurality of frames of an enhancement layer of the multi-layers included in the input bit-stream according to the target transmission bitrate, thereby generating an output bit-stream; and

a bit-stream transmission unit which transmits to a client device the output bit-stream, under control of the processor,

wherein the pre-decoding unit skips the at least one frame among the plurality of frames of the enhancement layer if the target transmission bitrate is less than a threshold bitrate, and

wherein the threshold bitrate is provided from a video encoder that generates the input bit-stream.

8. The apparatus of claim 7, wherein the threshold bit rate is obtained at the video encoder by comparing a frame of the enhancement layer, decoded by decoding a texture component of the enhancement layer frame, a part of which is truncated, with a frame of a base layer of the multi-layers that temporally corresponds to the enhancement layer frame, and is decoded and upsampled at a pixel resolution of the enhancement layer, and

wherein, as a result of the comparing, the threshold bitrate is determined by a bitrate when both of the enhancement layer frame and the base layer frame have a substantially same picture quality.

9. The apparatus of claim 7, wherein the threshold bitrate refers to a bitrate when a size ratio of a texture component to a motion component is a specific value according to truncation of the texture component.

10. An apparatus for adapting a bitrate of an input bit-stream, comprising multi-layers, to a variable network environment, the apparatus comprising:

a processor;

a bit-stream parsing unit which parses the input bit-stream, under control of the processor;

a pre-decoding condition determining unit which determines a target transmission bitrate according to the variable network environment, under control of the processor;

a pre-decoding unit which, under control of the processor, skips at least one frame among a plurality of frames of an enhancement layer of the multi-layers included in the input bit-stream according to the target transmission bitrate, thereby generating an output bit-stream; and

a bit-stream transmission unit which transmits to a client device the output bit-stream, under control of the processor,

wherein the pre-decoding unit skips the at least one frame among the plurality of frames of the enhancement layer if the target transmission bitrate is less than a threshold bitrate, and

wherein the pre-decoding unit skips the at least one frame, in an inverse order starting from a last frame included in a current pre-decoding unit of the plurality of frames of the enhancement layer.

11. A video decoder for decoding an input bit-stream, comprising multi-layers, the video decoder comprising:

a processor;

a skip confirmation unit which, under control of the processor, confirms whether a frame of an enhancement layer among the multi-layers has been skipped by reading out a value of a field of the enhancement layer frame that indicates a texture data size of the enhancement layer frame from the input bit-stream;

a base layer decoder which, under control of the processor, restores a frame of a base layer having a temporal position corresponding to a temporal position of the enhancement layer frame if the value indicates that the enhancement layer frame has been skipped; and

an upsampling unit which upsamples the restored base layer frame to a pixel resolution of the enhancement layer, under control of the processor.

12. The video decoder of claim 11, further comprising an enhancement layer decoder which restores the enhancement layer frame from texture data corresponding to the value, if the value does not indicate that the enhancement layer frame has been skipped.

13. The video decoder of claim 11, wherein the value to indicate that the enhancement layer frame has been skipped is “

0”

.

14. A method of adapting a bitrate of an input bit-stream, comprising multi-layers, to a variable network environment using an apparatus comprising a processor, a parsing unit, a condition determining unit, a pre-decoding unit and a transmission unit, the method comprising:

parsing the input bit-stream by the parsing unit under control of the processor;

determining a target transmission bitrate according to the variable network environment by the condition determining unit under control of the processor;

selectively performing, by the pre-decoding unit under control of the processor, one of an operation of skipping at least one frame among a plurality of frames of an enhancement layer of the multi-layers included in the input bit-stream, and an operation of truncating a portion of the at least one frame of the plurality of frames of the enhancement layer, according to the target transmission bitrate, thereby generating an output bit-stream; and

transmitting, by the transmission unit under control of the processor, to a client device the output bit-stream.

15. The method of claim 14, wherein the operation of skipping is performed if the target transmission bitrate is less than a threshold bitrate.

16. A method of adapting a bitrate of an input bit-stream, comprising multi-layers, to a variable network environment using an apparatus comprising a processor;

a parsing unit, a condition determining unit, a pre-decoding unit and a transmission unit, the method comprising;

parsing the input bit-stream by the parsing unit under control of the processor;

determining a target transmission bitrate according to the variable network environment, by the condition determining unit under control of the processor;

skipping, by the pre-decoding unit under control of the processor, at least one frame among a plurality of frames of an enhancement layer of the multi-layers included in the input bit-stream according to the target transmission bitrate, thereby generating an output bit-stream; and

transmitting to a client device the output bit-stream, by the transmission unit under control of the processor,

wherein the skipping is performed by skipping the at least one frame among the plurality of frames of the enhancement layer if the target transmission bitrate is less than a threshold bitrate, and

wherein the threshold bitrate is provided from a video encoder that generates the input bit-stream.

17. The method of claim 16, wherein the skipping the at least one frame among the plurality of frames of the enhancement layer is performed, if a frame temporally corresponding to the enhancement layer frame is present in a layer lower than the enhancement layer among the multi-layers in the input bit-stream.

18. A video decoding method for decoding an input bit-stream, comprising multi-layers, using a video decoder comprising a processor, a skip confirmation unit, a base layer decoder and an upsampling unit, the method comprising:

confirming, by the skip confirmation unit under control of the processor, whether a frame of an enhancement layer among the multi-layers has been skipped by reading out a value of a field of the enhancement layer frame that indicates a texture data size of the enhancement layer frame from the input bit-stream;

restoring, by the base layer decoder under control of the processor, a frame of a base layer having a temporal position corresponding to a temporal position of the enhancement layer frame if the value indicates that the enhancement layer frame has been skipped; and

upsampling the restored base layer frame to a pixel resolution of the enhancement layer, by the unsampling unit under control of the processor.

19. The method of claim 18, further comprising restoring the enhancement layer frame from texture data corresponding to the value, if the value does not indicate that the enhancement layer frame has been skipped.

20. The method of claim 18, wherein the value to indicate that the enhancement layer frame has been skipped is “

0”

.