Method and apparatus for providing rate control for panel-based real-time video encoder

• US 7,564,901 B2
• Filed: 10/09/2006
• Issued: 07/21/2009
• Est. Priority Date: 10/09/2006
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Zhou, Jian, Wang, Limin
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US11/539,786
• Publication Number:
  US 20080084928A1
• Time in Patent Office: 1,016 Days
• Field of Search: 37524001-24029
• US Class Current: 375/240.03
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]

Abstract

A method and apparatus for providing multi-panel rate control of real-time video digital encoders such as an MPEG or H series including the steps of bit allocation, rate control and adaptive quantization. Bit allocation assigns a target number of bits per group of pictures, and per picture of each type. Rate control adjusts the quantization parameter at the MB level to achieve that target number of bits per picture. Adaptive quantization further modulates the parameter per MB using local activity measure. Bit allocation and rate control can be implemented through a central control unit, or a central digital signal processor (DSP) while adaptive quantization can be implemented at the local panel.

Claims

1. A method for panel-based rate control for encoding real-time digital video, comprising:

performing bit allocation by assigning and updating a target rate for a group of pictures (GOP), and computing a target rate for a picture of type I, P or B in frame or field within said GOP from said target rate of said GOP, where said picture is processed on multiple panels; and

performing rate control by adjusting guantization parameters to achieve said target rates

protecting both encoder and decoder buffers from overflowing or underflowing before encoding a picture, wherein protecting the encoder buffers comprises checking and adjusting the target number of bits assigned for a picture according to the following two conditions;

if

buffer_occupancy +T<sub>pic—

target</sub>>

α

×

buffer_size, then

T<sub>pic—

target</sub>=α

×

buffer_size−

buffer_occupancy; and

if

buffer_occupancy+T<sub>pic—

target</sub>−

bit−

rate/pic_rate<

(1−

α

)×

buffer_size, then

T<sub>pic—

target</sub>=(1−

α

)×

buffer_size−

buffer<sub>—

occupancy+bit</sub>_rate/pic_rate.

2. The method of claim 1, wherein the step of bit allocation further comprises budgeting a nominal number of bits (R_GOP—

_nnominal) determined by the equation

$R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}}=N_{\mathrm{GOP}}\times <br><br>\frac{\mathrm{bit\_rate}}{\mathrm{pic\_rate}}$

for a group of pictures (GOP), where N_GOP is the length of the GOP, bit_rate is a target bit rate in bits per second, and pic_rate is a picture rate in pictures per second.

4. The method of claim 1 wherein the step of bit allocation further comprises initializing and updating the complexity measures of frame and field pictures of type I, P and B.

5. The method of claim 1 wherein the step of rate control further comprises updating the picture numbers of picture type I, P and B (N_I, N_P, and N_B).

7. The method of claim 1, wherein the step of bit allocation further comprises handling scene changes in a GOP.

12. The method of claim 7, wherein no action occurs if a scene change occurs within a GOP and an I picture in the GOP is in a new scene, otherwise a first P picture in the new scene is changed to an I picture.

13. The method of claim 7, wherein a short or long GOP (“

Irregular GOP”

) is created and followed by a regular GOP.

14. The method of claim 13, wherein the length of the Irregular GOP is between 0.5 to 1.5 times the regular GOP length.

15. The method of claim 12, wherein the first P picture in the new scene is the Nth picture of the GOP.

16. The method of claim 15, wherein, if the first P picture is in the first half of the GOP, then the scheduled I picture in the GOP is changed to a P picture.

17. The method of claim 16, wherein the long and short GOPs, with lengths determined by the expressions N_GOP=N_GOP+(N−

N<sub>sub—

GOP</sub>) and N_GOP=N_GOP−

(N−

N<sub>sub—

GOP</sub>),respectively, are created.

18. The method of claim 17, wherein the long GOP is followed by the short GOP.

19. The method of claim 17, wherein the corresponding GOP lengths for I, P and B frame and field pictures are determined by the expressions recited in claim 17 using an updated GOP length.

20. The method of claim 17, wherein the nominal numbers of bits for the long and short GOPs are determined using the expressions

$R_{\mathrm{GOP}}$

⁢

_ ⁢

⁢

nominal = R GOP ⁢

⁢

_ ⁢

⁢

nominal + N - N sub_GOP N GOP ×

R GOP ⁢

⁢

_ ⁢

⁢

nominal ⁢

⁢

and $R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}}=R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}}-\frac{N-N_{\mathrm{sub\_GOP}}}{N_{\mathrm{GOP}}}\times <br><br>R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}},$

respectively.

21. The method of claim 15, wherein, if the first P in the new scene is in the second half of the GOP, then the scheduled I picture in the next GOP is changed to a P picture.

22. The method of claim 21, wherein the short and long GOPs with lengths determined by the expressions N_GOP=N−

N<sub>sub—

GOP</sub> and N_GOP=2×

N_GOP−

(N−

N<sub>sub—

GOP</sub>),respectively, are created.

23. The method of claim 22, wherein the short GOP is followed by the long GOP.

24. The method of claim 23, wherein the corresponding GOP lengths for I, P and B frame and field pictures are determined by the expressions recited in claim 24 using an updated GOP length.

25. The method of claim 23, wherein the nominal numbers of bits for the long and short GOPs are determined using the expression

$R_{\mathrm{GOP}}$

⁢

_ ⁢

⁢

nominal = N - N sub_GOP N GOP ×

R GOP ⁢

⁢

_ ⁢

⁢

nominal ⁢

⁢

and $R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}}=2\times <br><br>R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}}=\frac{N-N_{\mathrm{sub}<br><br><br><br>\_<br><br><br><br>\mathrm{GOP}}}{N_{\mathrm{GOP}}}\times <br><br>R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{nominal}},$

respectively.

3. A method for panel-based rate control for encoding real-time digital video, comprising:

performing bit allocation by assigning and updating a target rate for a group of pictures (GOP), and computing a target rate for a picture of type I, P or B in frame or field within said GOP from said target rate of said GOP, where said picture is processed on multiple panels, wherein a central DSP computes said target rate for said picture of type intra picture (I), predictive coded picture (P), or bi-directional predictive coded picture (B), using the expressions

$T_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}=\frac{K_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>C_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{remaining}}}{\begin{array}{c}{K}_I<br><br>n_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>I}<br><br>C_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>I}+K_P<br><br>n_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>P}<br><br>C_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>P}+\\ K_B<br><br>n_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>B}<br><br>C_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>B}\end{array}}$

for frame pictures, and

$T_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}=\frac{K_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>C_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>R_{\mathrm{GOP}<br><br><br><br>\_<br><br><br><br>\mathrm{remaining}}}{\begin{array}{c}{K}_I<br><br>n_{\mathrm{field}<br><br><br><br>\_<br><br><br><br>I}<br><br>C_{\mathrm{field}<br><br><br><br>I}+K_P(n_{\mathrm{field}<br><br><br><br>0<br><br>\_<br><br><br><br>P}<br><br>C_{\mathrm{field}<br><br><br><br>0<br><br>\_<br><br><br><br>P}+\\ n_{\mathrm{field}<br><br><br><br>1<br><br><br><br>\_<br><br><br><br>P}<br><br>C_{\mathrm{field}<br><br><br><br>1<br><br>\_<br><br><br><br>P})+K_B<br><br>n_{\mathrm{field}<br><br><br><br>\_<br><br><br><br>B}<br><br>C_{\mathrm{field}<br><br><br><br>\_<br><br><br><br>B}\end{array}}$

for field pictures, and

performing rate control by adjusting guantization parameters to achieve said target rates.

33. The method of claim 3, wherein the step of bit allocation further comprises budgeting a nominal number of bits (R_GOP—

_nominal) determined by the equation

$R_{\mathrm{GOP}<br><br>\_<br><br>\mathrm{nominal}}=N_{\mathrm{GOP}}\times <br><br>\frac{\mathrm{bit\_rate}}{\mathrm{pic\_rate}}$

for a group of pictures (GOP), where N_GOP is the length of the GOP, bit_rate is a target bit rate in bits per second, and pic_rate is a picture rate in pictures per second.

34. The method of claim 3, wherein the step of bit allocation further comprises initializing and updating the complexity measures of frame and field pictures of type I, P and B.

35. The method of claim 3, wherein the step of rate control further comprises updating the picture numbers of picture type I, P and B (N_I, N_P, and N_B)

6. A method for panel-based rate control for encoding real-time digital video, comprising:

performing bit allocation by assigning and updating a target rate for a group of pictures (GOP), and computing a target rate for a picture of type I, P or B in frame or field within said GOP from said target rate of said GOP, where said picture is processed on multiple panels; and

performing rate control by adjusting guantization parameters to achieve said target rates, wherein the step of rate control further comprises determining six virtual buffer fullnesses, one for each picture of type I, P or B in frame or field, at a time instant t is using the expression

$d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{{\mathrm{MB}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)}{{\mathrm{MB}}_{\mathrm{pic}}}<br><br>T_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}};<br><br>$

or the expression

$d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{\underset{i=0}{\overset{{\mathrm{MB}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)}{\sum <br><br>}}<br><br>{\mathrm{act}}_i}{\mathrm{total\_act}}<br><br>T_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}},$

if the bit budget per MB is assigned according to its need, where act_i is the local activity measure of MB(i)

$\mathrm{total\_act}=\underset{i}{\sum <br><br>}<br><br><br><br>{\mathrm{act}}_i,$

and the index i is over all the MBs in the current picture;

or the expression

$d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{\underset{i=0}{\overset{{\mathrm{MB}}_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}}<br><br>\left(t\right)}{\sum <br><br>}}<br><br>{\mathrm{cost}}_i}{\mathrm{total\_cost}}<br><br>T_{\mathrm{pic}<br><br><br><br>\_<br><br><br><br>\mathrm{type}},$

where cost_i is the cost measure of MB(i) and

$\mathrm{total\_cost}=\underset{i}{\sum <br><br>}<br><br>{\mathrm{cost}}_i.$

8. The method of claim 6, wherein a central DSP maintains and updates said six virtual buffers, and determines six quantization parameters,—

^QP<sub>pic—

type</sub>, proportional to six virtual buffer fullnesses, one for each of the picture types I, P and B in frame or field.

9. The method of claim 8, wherein the central DSP sends the six quantization parameters to all the current pictures of I, P or B.

10. The method of claim 9, wherein the same quantization parameter is applied to all the panels (panel based architecture) in a picture until a new quantization parameter is received.

11. The method of claim 10, wherein a final quantization parameter for a macroblock (MB) j in frame or field mode in each panel is determined by using the expression QP_j=QP_pic—

_type(t)+6×

log₂(N_act_j) where N_act_j is the normalized local activity of MB j.

36. The method of claim 6, wherein the step of bit allocation further comprises budgeting a nominal number of bits (R_GOP—

_nominal) determined by the equation

$R_{\mathrm{GOP}<br><br>\_<br><br>\mathrm{nominal}}=N_{\mathrm{GOP}}\times <br><br>\frac{\mathrm{bit\_rate}}{\mathrm{pic\_rate}}$

for a group of pictures (GOP), where N_GOP is the length of the GOP, bit_rate is a target bit rate in bits per second, and pic_rate is a picture rate in pictures per second.

37. The method of claim 6, wherein the step of bit allocation further comprises initializing and updating the complexity measures of frame and field pictures of type I, P and B.

38. The method of claim 8, wherein the step of rate control further comprises updating the picture numbers of picture type I, P and B (N_I, N_P, and N_B)

26. A video data coding-apparatus comprising:

a processor;

an input/output device;

a memory; and

a video encoding module that performs bit allocation by assigning a target number of bits per GOP, performs rate control by adjusting the quantization parameter QP to achieve said target number of bits, and performs adaptive quantization by modulating the quantization parameter using the local activity measure wherein the step of rate control further comprises determining six virtual buffer fullnesses, one for each picture of type I, P or B in frame or field, at a time instant t is using the expression

$d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{{\mathrm{MB}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)}{{\mathrm{MB}}_{\mathrm{pic}}}<br><br>T_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}};<br><br>$

or the expression

$d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{\underset{i=0}{\overset{{\mathrm{MB}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)}{\sum <br><br>}}<br><br><br><br>{\mathrm{act}}_i}{\mathrm{total\_act}}<br><br>T_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}},$

if the bit budget per MB is assigned according to its need, where act_i is the local activity measure of MB(i),

$\mathrm{total\_act}=\underset{i}{\sum <br><br>}<br><br><br><br>{\mathrm{act}}_i,$

and the index i is over all the MBs in the current picture;

or the expression

$d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)=d_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(0\right)+{\mathrm{\alpha <br><br>}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}\times <br><br>B_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)-\frac{\underset{i=0}{\overset{{\mathrm{MB}}_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}}<br><br>\left(t\right)}{\sum <br><br>}}<br><br><br><br>{\mathrm{act}}_i}{\mathrm{total\_cost}}<br><br>T_{\mathrm{pic}<br><br>\_<br><br>\mathrm{type}},$

where cost_i is the cost measure of MB(i) and

$\mathrm{total\_cost}=\underset{i}{\sum <br><br>}<br><br>{\mathrm{cost}}_i.$

27. The video data coding apparatus of claim 26, wherein the video encoding module checks and adjusts the target number of bits assigned for a picture to prevent the system from overflowing and underflowing.

28. The video data coding apparatus of claim 26, wherein the video encoding module budgets a nominal number of bits for a GOP, assigns each picture of type I, P, or B a target number of bits, and initializes and updates the complexity measures of frame and field pictures.

29. The video data coding apparatus of claim 26, wherein the video encoding module updates the picture numbers of picture type I, P or B, determines the fullness of six virtual buffers, and maintains and updates said virtual buffers.

30. The video data coding apparatus of claim 26, wherein the video encoding determines the quantization parameter for the MB in frame or field mode and modulates said quantization parameter using a local activity measure.

31. The video data coding apparatus of claim 26, wherein the video encoding module generates, updates, and applies six quantization parameters to all the panels in a picture of type I, P or B.

32. The video coding apparatus of claim 26, wherein the video encoding module is capable of handling scene changes.