System and method for reencoding segments of buffer constrained video streams

• US 6,522,693 B1
• Filed: 02/23/2000
• Issued: 02/18/2003
• Est. Priority Date: 02/23/2000
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lu, Ligang, Gonzales, Cesar A., Kouloheris, Jack L.
• Primary Examiner(s): Rao, Andy
• Application Number:
  US09/511,859
• Time in Patent Office: 1,091 Days
• Field of Search: 348/9-13, 375/240.02, 375/240.19
• US Class Current: 375/240.02
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A system and method for generating an MPEG compliant video stream with satisfactory visual quality. The method employs re-encoding at a higher (or lower) bit rate, only the segments with unsatisfactory visual picture quality while retaining other parts of the original video stream and then merging the re-encoded segments with the remaining original portions. If the original video stream is coded at a reasonable bit rate, the number and length of those segments with unsatisfactory visual picture quality will be few and short. Re-encoding only those segments can save very significant amount of cost and only requires minimal additional storage space. Furthermore, the portions with satisfactory picture quality will be retained. The system and method is applicable to MPEG and non-MPEG compliant data streams.

Claims

1. A method for reencoding a signal source corresponding to a previously encoded segment of a plurality of ordered segments including data comprising an original bit stream, each segment including a plurality of ordered coding units, the method comprising steps of:

allocating a target number of bits for each said coding unit as a function of a quality measurement, an estimated coding complexity, a number of coding units per unit time interval of said original bit stream, and a target average bit rate per coding unit, wherein the quality measurement is based on statistical differences between the signal source and a reconstruction of the signal source derived from the previously encoded segment;

re-encoding the signal source according to the allocated target number of bits of each coding unit; and

,

merging a new bit stream resulting from the re-encoding of the signal source into the original bit stream to replace a bit stream corresponding to said previously encoded segment.

2. The method according to claim 1, wherein the data is video data, said coding units comprising video pictures.

3. The method according to claim 2, wherein said original bit stream is MPEG compliant, said segment re-encoding step implementing an MPEG encoding system.

4. The method according to claim 1, wherein coding units of said segment are re-encoded at a higher or lower bit rate than an encoding bit rate for said segment of said original bit stream.

5. The method according to claim 2, wherein an encoding system implements one or more buffer constraints to ensure proper subsequent decoding operation of said generated bit streams, said method further including the steps of:

adjusting an allocated target bit budget to ensure that the re-encoded segment using the target bit budget conform to said buffer constraints; and

,

after said new bit stream representation for said segment is merged with the original bit stream, conforming the resulting new bit stream representation to said system constraints.

6. The method according to claim 5, wherein said target number of bits T per segment F_i is calculated according to:

$T_i=\frac{c_i<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{PQI}}_i}{\left(\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>c_l\right)<br><br>{\mathrm{PQI}}_{S_{2<br><br>k}}^t}<br><br>\text{\hspace{1em}<br><br>}<br><br>n_k<br><br>b_p<br><br>W_t$

where c_i is a complexity measure of a current picture;

t is the picture coding type

PQI_i is the picture quality index of the current picture F_i;

PQI_S2k^t is an average picture quality index of all pictures with the same picture coding type as picture F_i in a time interval S_2k;

$\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>c_l$

is the sum of the estimated coding complexity of all pictures in the current time interval S_2k;

n_k is the number of pictures in interval S_2k;

b_p is the average number of bits/picture; and

,

W_t is a weighting factor for picture type of t.

7. The method according to claim 6, wherein PQI_S2k^t is calculated according to:

$\frac{\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>{\mathrm{PQI}}_l}{n_t}$

where TP₁ is the picture coding type;

PQI₁ is the picture quality index of the picture F_i; and

,

n_t is the number of pictures of coding type t in the interval S_2k.

8. The method according to claim 6, wherein the picture quality index PQI_i is a function of one or a combination of:

a distance measure between a reconstructed picture and an original picture, and a subjective measure taking into account of a human visual system'"'"'s characteristics.

9. The method according to claim 6, wherein said average number of bits per picture b_p is related to said average bit rate per coding unit, said method further including the step of enabling a user to specify either said average number of bits per picture or said average bit rate per coding unit.

10. The method according to claim 6, wherein one system constraint includes no occurrence of buffer underflow, said step of adjusting an allocated target bit budget including the steps of:

a) determining if a potential buffer underflow condition exists at a current picture F_i;

b) calculating an amount of bits over allocated;

c) determining a total amount of target bits allocated to all the pictures between a picture index position i₁ where a full buffer level is reached or an initial picture index position and, a picture index position i₂ where the potential underflow is detected;

d) calculating an adjustment factor a based on said total amount of target bits allocated and said amount of bits over allocated;

e) adjusting the target allocation for all pictures between i₁ and i₂ according to T₁=α

*T₁, where 1=i₁. i₁+1, . . . , ₂; and

,

f) setting i to the next current picture position, and repeat steps a) through e) for all pictures within re-encoded segment S₂.

11. The method according to claim 10, wherein said determining step a) includes comparing an allocated target bit budget for picture F_i against a difference between a buffer level before removing the coded data of the ith picture in the re-encoded segment and a predetermined guard band G of bits.

12. The method according to claim 10, further including the step of finding a minimum required buffer fullness at the end of the reencoded segment S₂.

13. The method of claim 12, wherein said encoding system generates statistics files comprising information relating to said bit streams, wherein said step of merging a new bit stream representation of the re-encoded coding units into the original bit stream further includes merging corresponding statistics files of said new and original bit streams so that reencoding of successive segments may overlap.

14. A system for reencoding a signal source corresponding to a previously encoded segment of a plurality of segments comprising an original bit stream, each segment of said original data bit stream comprising one or more ordered coded units, the system comprising:

a storage device comprising statistical information relating to said segments of said original bit stream, said statistical information including a quality measurement, an estimated coding complexity and, a number of coding units per unit time interval of said specified original bit stream segment, wherein the quality measurement is based on statistical differences between the signal source and a reconstruction of the signal source derived from the previously encoded segment;

a target bit allocator for receiving said statistical information including said quality measurement and allocating said target number of bits for said segment as a function of said statistical information including said quality measurement and a target average bit rate per coding unit;

an encoding device for reencoding the signal source corresponding to the previously encoded segment of said original data bit stream according to said allocated target number of bits; and

,

a device for merging a new bit stream resulting from the reencoding of the signal source into the original bit stream to replace a bit stream corresponding to said previously encoded segment.

15. The system method according to claim 14, wherein the data is video data, said coding units comprising video pictures.

16. The system according to claim 15, wherein said original bit stream is MPEG compliant, said segment re-encoding step implementing an MPEG encoding system.

17. The system according to claim 14, wherein encoding device re-encodes said specified segment of coding units of said segment are re-encoded at a higher or lower bit rate than an encoding bit rate for said segment of said original bit stream.

18. The system according to claim 15, wherein said encoding system implements a decoder having a decoder buffer device characterized as imposing one or more buffer constraints to ensure proper subsequent decoding operation of said generated bit streams, said target encoder device adjusting an allocated target bit budget to ensure that the re-encoded segment using the target bit budget conforms to said buffer constraints, and, that a resulting new bit stream comprising said new bit stream representation merged with the original bit stream conforms to said system constraints.

19. The system according to claim 18, wherein said target number of bits T per picture F_i is calculated according to:

$T_i=\frac{c_i<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{PQI}}_i}{\left(\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>c_l\right)<br><br>{\mathrm{PQI}}_{S_{2<br><br>k}}^t}<br><br>\text{\hspace{1em}<br><br>}<br><br>n_k<br><br>b_p<br><br>W_t$

where c_i is a complexity measure of a current picture;

t is the picture coding type;

PQI_i is the picture quality index of the current picture F_i;

PQI_S2k^t is an average picture quality index of all pictures with the same picture coding type as picture F_i in a time interval S_2k;

$\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>c_l$

is the sum of the estimated coding complexity of all pictures in the current time interval S_2k;

n_k is the number of pictures in interval S_2k;

b_p is the average number of bits/picture; and

,

W_t is a weighting factor for picture type of t.

20. The system according to claim 19, wherein PQI_S2k^t is calculated according to:

$\frac{\underset{{\mathrm{TP}}_l=t,\text{\hspace{1em}<br><br>}<br><br>l\in <br><br>S_{2<br><br>k}}{\sum <br><br>}<br><br>{\mathrm{PQI}}_l}{n_t}$

where TP₁ is the picture coding type;

PQI₁ is the picture quality index of the picture F_i; and

,

n_t is the number of pictures of coding type t in the interval S_2k.

21. The system according to claim 19, wherein the picture quality index PQI_i is a function of one or a combination of:

a distance measure between a reconstructed picture and an original picture, and a subjective measure taking into account of a human visual system'"'"'s characteristics.

22. The system according to claim 19, wherein said average number of bits per picture b_p is related to said average bit rate per coding unit, said system further including interface device for enabling a user to specify either said average number of bits per picture or said average bit rate per coding unit.

23. The system according to claim 18, wherein one buffer constraint includes no occurrence of decoder buffer underflow, said encoder device further including:

a device for determining if a potential buffer underflow condition exists at a current picture F_i, and if said potential underflow condition exists, calculating an amount of bits over allocated;

device for determining a total amount of target bits allocated to all the pictures between an interval defined between a picture index position i₁ where a full buffer level is reached or an initial picture index position and, a picture index position i₂ where the potential underflow is detected;

a device for calculating an adjustment factor a based on said total amount of target bits allocated and said amount of bits over allocated and adjusting the target allocation for all pictures between i₁ and i₂ according to T₁=α

*T₁, where l=i₁. i₁+1, . . . , i₂, wherein said potential underflow determination and said adjustments are repeated for all pictures within said re-encoded segment S₂.

24. The system according to claim 23, wherein said device for determining if a potential buffer underflow condition exists at a current picture F_i includes a comparator device for comparing an allocated target bit budget for picture F_i against a difference between a decoder buffer level before removing the coded data of the ith picture in the re-encoded segment and a predetermined guard band G of bits.

25. The system according to claim 23, further including a means for determining a minimum required buffer fullness at the end of the reencoded segment S₂.

26. The system of claim 24, wherein said encoder device generates said statistics information comprising information relating to said bit streams for storage in said storage device, said system further including merging corresponding statistics files of new and original bit streams so that reencoding of successive segments may overlap.