Method and apparatus for transmitting a coded video signal

• US 7,983,160 B2
• Filed: 09/08/2004
• Issued: 07/19/2011
• Est. Priority Date: 09/08/2004
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Gunatilake, Priyan
• Primary Examiner(s): Vu; Huy D
• Assistant Examiner(s): MIAN, OMER S
• Application Number:
  US10/935,163
• Publication Number:
  US 20060050970A1
• Time in Patent Office: 2,505 Days
• Field of Search: 370/252, 370/230.1, 370/230, 370/231, 370/234
• US Class Current: 370/230.1
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer
• CPC Class Codes: H04N 21/6377 directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/658 Transmission by the client directed to the server

Abstract

A method and apparatus for transmitting a digitally encoded video image in real time via a data network wherein the video image is digitally encoded using a compression algorithm having a variable bit rate, e.g., JPEG2K, to provide an encoded data stream, the encoded data stream is fed into a buffer of known size, and the data is read from the buffer via a streamer with a controllable bit rate to provide a bit stream at a desired stream bit rate for transmission via the network. The bit rate of the encoded data stream from the encoder is monitored by a predictor and predicts a future bit rate of the encoded data stream; and, the bit rate of at least one of the encoder and the streamer is adjusted to provide the desired streaming bit rate.

Claims

1. A method of transmitting a coded video image in real time via a data network, comprising:

encoding a video image using a compression algorithm having a variable bit rate to provide an encoded data stream having a streaming bit rate;

feeding the encoded data stream into a buffer of known size;

reading data from the buffer via a streamer with a controllable bit rate to provide a bit stream at a desired stream bit rate for transmission via the network;

monitoring the streaming bit rate of the encoded data stream from the encoder and predicting a future bit rate of the encoded data stream; and

adjusting the streaming bit rate of at least one of the encoder and the streamer to provide the desired streaming bit rate; and

defining a set of strip-wise constant bit rate streaming levels,

wherein the adjusting step includes a step selected from a group consisting of;

raising the streaming bit rate proportionately to an increase of the predicted encoded bit rate level using the set of strip-wise streaming levels; and

lowering the streaming bit rate proportionately to a decrease of the predicted encoded bit rate level using the set of strip-wise streaming levels,

wherein the predicting step comprises using a prediction model that maps a curve corresponding to the streaming bit rate as a function of time based on a polynomial in at least a second order for ensuring smoothness of the curve,

wherein the predicting step comprises presuming that a high content-based correlation in time exists between any two adjacent frames, and

wherein the presuming step comprises disregarding a statistical outlier, the statistical outlier comprising at least one element selected from a group consisting essentially of a scene change and a scene cut.

2. The method according to claim 1, wherein the encoding algorithm comprises a JPEG2k algorithm.

3. The method according to claim 1, further comprising enlarging the buffer size by an amount equal to the momentary encoded bit rate minus the momentary streaming bit rate when the streaming bit rate is set to a maximum value such that further increase would result in a buffer overflow condition.

4. The method according to claim 1, further comprising defining a relationship between the encoding and streaming bit rate as follows:

the encoding bit rate at a time N=data input to the buffer at a time N,

the data in the buffer at a time N=the data in the buffer at a time N−

1+the encoding bit rate at a time N−

the streaming bit rate at a time N,

the encoding bit rate at a time N=delta at a time N+the streaming bit rate at a time N, and

the delta at a time N=the data in the buffer at a time N−

data in the buffer at a time N−

1.

5. The method according to claim 1, further comprising providing the encoding algorithm with a defined set of quality levels proportional to the encoding bit rate, wherein the adjusting step comprises decreasing the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate is greater than a maximum streaming bit rate level.

6. The method according to claim 5,

wherein the adjusting step further comprises increasing the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate plus a given threshold value is less than the maximum streaming bit rate level, and

wherein the threshold value comprises an empirical number based on an analysis of a plurality of representative video streams.

7. The method according to claim 1, further comprising providing the encoding algorithm with a defined set of quality levels proportional to the encoding bit rate,

wherein the adjusting step comprises increasing the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate plus a given threshold value is less than the maximum streaming bit rate level, and

wherein the threshold value is an empirical number based on an analysis of a plurality of representational video streams.

8. A method of transmitting a coded video signal in real time via a data network, comprising:

encoding a video signal using a compression algorithm, having a variable bit rate and a defined plurality of quality levels with a corresponding plurality of differing bit rates, to provide an encoded data stream having a streaming bit rate;

feeding the encoded data stream to a buffer of known size;

reading data from the buffer via a streamer with a controllable bit rate to provide a bit stream at a desired streaming bit rate for transmission via the network;

monitoring the streaming bit rate at a time N of the encoded data signal and predicting a future bit rate at a time N+1 of the encoded data stream based on the bit rate at a time N;

decreasing the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate at a time N+1 is greater than a maximum permitted streaming bit rate;

adjusting the streaming bit rate of at least one of the encoder and the streamer to provide the desired streaming bit rate; and

defining a set of strip-wise constant bit rate streaming levels,

wherein the adjusting step includes a step selected from a group consisting of;

raising the streaming bit rate proportionately to an increase of the predicted encoded bit rate level using the set of strip-wise streaming levels; and

lowering the streaming bit rate proportionately to a decrease of the predicted encoded bit rate level using the set of strip-wise streaming levels, and

wherein the predicting step comprises using a prediction model that maps a curve corresponding to the streaming bit rate as a function of time based on a polynomial in at least a second order for ensuring smoothness of the curve,

wherein the predicting step comprises presuming that a high content-based correlation in time exists between any two adjacent frames, and

wherein the presuming step comprises disregarding a statistical outlier, the statistical outlier comprising at least one element selected from a group consisting essentially of a scene change and a scene cut.

9. The method according to claim 8, further comprising increasing the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate at a time N+1 plus a threshold value is greater than a maximum permitted streaming bit rate, wherein the threshold value is an empirically derived number based on an analysis of a plurality of representative video signal bit streams.

10. The method according to claim 9, wherein the encoder algorithm comprises a JPEG2k algorithm.

11. A method of transmitting a coded video signal in real time via a data network comprising:

encoding a video signal using a compression algorithm having a variable bit rate and a defined plurality of quality levels with corresponding differing bit rates, to provide an encoded data stream having a streaming bit rate;

feeding the encoded data stream to a buffer of known size;

reading data from the buffer via a streamer with a controllable bit rate to provide a bit stream at a desired streaming bit rate for transmission via the network;

monitoring the streaming bit rate at a time N of the encoded data signal and predicting a future bit rate at a time N+1 of the encoded data stream based on the bit rate at a time N;

defining a set of step-wise constant bit rate streaming levels;

increasing or decreasing the step level of the streaming bit rate at a time n+1, and thus the streaming bit rate, based on an increase or decrease, respectively, of the predicted encoding bit rate at a time N+1;

adjusting the streaming bit rate of at least one of the encoder and the streamer to provide the desired streaming bit rate; and

defining a set of strip-wise constant bit rate streaming levels,

wherein the adjusting step includes a step selected from a group consisting of;

raising the streaming bit rate proportionately to an increase of the predicted encoded bit rate level using the set of strip-wise streaming levels; and

lowering the streaming bit rate proportionately to a decrease of the predicted encoded bit rate level using the set of strip-wise streaming levels, and

wherein the predicting step comprises using a prediction model that maps a curve corresponding to the streaming bit rate as a function of time based on a polynomial in at least a second order for ensuring smoothness of the curve,

wherein the predicting step comprises presuming that a high content-based correlation in time exists between any two adjacent frames, and

wherein the presuming step comprises disregarding a statistical outlier, the statistical outlier comprising at least one element selected from a group consisting essentially of a scene change and a scene cut.

12. The method according to claim 11, further comprising enlarging the size of the buffer by an amount equal to the encoding bit rate at a time N minus the streaming bit rate at a time N when the streaming bit rate is at a maximum level such that further increase in the streaming bit rate would result in a buffer overflow condition.

13. The method according to claim 11, further comprising defining a relationship between the encoding and the streaming bit rates as follows:

the encoding bit rate at a time N=the data input to the buffer at a time N,

the data in the buffer at a time N=the data in the buffer at a time N−

1+the encoding bit rate at a time N−

the streaming bit rate at a time N,

the encoding bit rate at a time N=a delta at a time N+the streaming bit rate at a time N, and

the delta at a time N=the data in the buffer at a time N−

the data in the buffer at a time N−

1.

14. The method according to claim 11, wherein the encoder algorithm comprises a JPEG2k algorithm.

15. An apparatus for transmitting a coded video image in real time via a data network, comprising:

a video encoder for encoding a video image using a compression algorithm having a variable bit rate to provide an encoded data stream;

a buffer of known size connected to receive the encoded data stream;

a streamer with a controllable bit rate connected to the buffer to read data from the buffer and to provide a bit stream at a desired stream bit rate for transmission via the network;

a predictor monitoring the bit rate of the encoded data stream from the encoder and predicting a future bit rate of the encoded data stream; and

a control arrangement adjusting the bit rate of at least one of the encoder and the streamer to provide the desired streaming bit rate,

wherein at least on at least one of the encoder and the streamer is adapted to adjust the streaming bit rate for providing the desired streaming bit rate; and

wherein the predictor is adapted to define a set of strip-wise constant bit rate streaming levels,

wherein the control arrangement is adapted to effect an adjustment selected from a group consisting of;

raising the streaming bit rate proportionately to an increase of the predicted encoded bit rate level using the set of strip-wise streaming levels; and

lowering the streaming bit rate proportionately to a decrease of the predicted encoded bit rate level using the set of strip-wise streaming levels, and

wherein the predictor is adapted to use a prediction model that maps a curve corresponding to the streaming bit rate as a function of time based on a polynomial in at least a second order for ensuring smoothness of the curve,

wherein the predictor is adapted to presume that a high content-based correlation in time exists between any two adjacent frames, and

wherein the predictor is adapted to disregard a statistical outlier, the statistical outlier comprising at least one element selected from a group consisting essentially of a scene change and a scene cut.

16. The apparatus according to claim 15, wherein the encoder algorithm comprises a JPEG2k algorithm.

17. The apparatus according to claim 15, wherein the control arrangement enlarges the buffer size by an amount equal to the momentary encoded bit rate minus the momentary steaming bit rate when the streaming bit rate is set to a maximum value such that further increase would result in a buffer overflow condition.

18. The apparatus according to claim 15, wherein the control arrangement maintains the following defined relationship between the encoding and streaming bit rates:

the encoding bit rate at a time N=the data input to the buffer at a time N;

the data in the buffer at a time N=the data in the buffer at a time N−

1+the encoding bit rate at a time N−

the streaming bit rate at a time N;

the encoding bit rate at a time N=a delta at a time N+the streaming bit rate at a time N; and

the delta at a time N=the data in the buffer at a time N−

the data in the buffer at a time N−

1.

19. The apparatus according to claim 15, wherein the encoding algorithm includes a defined set of quality levels proportional to the encoding bit rate;

and the control arrangement decreases the encoding quality level and, thus, the encoding bit rate if the predicted encoding bit rate is greater than a maximum streaming bit rate level.

20. The apparatus according to claim 19,

wherein the control arrangement increases the encoding quality level, and, thus, the encoding bit rate if the predicted encoding bit rate plus a given threshold value is less than the maximum streaming bit rate level, and

wherein the threshold value is an empirical number based on an analysis of a plurality of representative video streams.

21. The apparatus according to claim 15,

wherein the encoding algorithm includes a defined set of quality levels proportional to the encoding bit rate,

wherein the control arrangement increases the encoding quality level, and, thus, the encoding bit rate if the predicted encoding bit rate plus a given threshold value is less than the maximum streaming bit rate level, and

wherein the threshold value is an empirical number based on an analysis of representative video streams.