Apparatus and method for optimizing the rate control in a coding system

• US 6,160,846 A
• Filed: 10/23/1996
• Issued: 12/12/2000
• Est. Priority Date: 10/25/1995
• Status: Expired due to Term

• Current Assignee: Sharp Corporation (Hon Hai Precision Industry Co., Ltd.), MediaTek, Inc.
• Original Assignee: Sharp Corporation (Hon Hai Precision Industry Co., Ltd.), Sarnoff Corporation (SRI International, Inc.)
• Inventors: Sun, Huifang, Kwok, Wilson, Zhang, Ya-Qin, Chiang, Tihao, Chien, Max
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US08/738,228
• Time in Patent Office: 1,511 Days
• Field of Search: 348/384, 348/390, 348/400, 348/401, 348/402, 348/403, 348/405, 348/407, 348/409-413, 348/415, 348/416, 348/699, 382/232-245, 382/248-253, 382/279-281
• US Class Current: 375/240.05
• CPC Class Codes: H04N 19/10 using adaptive coding
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/517 by encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/64 characterised by ordering of coefficients or of bits for transmission

Abstract

A method and apparatus for selecting a quantizer scale for each macroblock to maintain the overall quality of the video image while optimizing the coding rate. A quantizer scale is selected for each macroblock such that target bit rate for the picture is achieved while an optimal quantization scale ratio is maintained for successive macroblocks to produce a uniform visual quality over the entire picture. One embodiment applies the method to the frame level while another embodiment applies the method in conjunction with a wavelet transform.

Claims

1. Apparatus for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said apparatus comprising:

a block motion compensator for computing a motion vector for the block and for generating a predicted image using said motion vector;

a transform module, coupled to said block motion compensator, for applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

a quantizer, coupled to said transform module, for quantizing said plurality of coefficients with a quantizer scale;

a controller, coupled to said quantizer, for selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded frame and wherein said coding information from said immediate previous encoded portion is used to determine T_P(AVG), a projected average number of bits needed to code a remaining frame, where said T_P(AVG) is expressed as;

space="preserve" listing-type="equation">T.sub.P(AVG) =Max(bitrate/frame rate, R/N)

where R is a remaining number of bits, N is a remaining number of frames in the image sequence, bitrate is a channel bitrate and frame rate is a frame rate of the image sequences; and

a coder, coupled to said quantizer, for coding said plurality of quantized coefficients.

2. The apparatus of claim 1, wherein said T_P(AVG) is used to determine a projected number of bits T_P (n) for a frame "n" in the image sequence, where said T_P (n) is expressed as:

space="preserve" listing-type="equation">T.sub.P (n)=T.sub.P(AVG) *(1-w)+B(n-1)*w

where B(n-1) is a number of bits used to code said immediate previous encoded frame and w is a weighing factor.

3. Apparatus for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said apparatus comprising:

a block motion compensator for computing a motion vector for the block and for generating a predicted image using said motion vector;

a transform module, coupled to said block motion compensator, for applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients, where said transform module applies a wavelet transform to produce a plurality of wavelet trees;

a quantizer, coupled to said transform module, for quantizing said plurality of coefficients with a quantizer scale;

a controller, coupled to said quantizer, for selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion; and

a coder, coupled to said quantizer, for coding said plurality of quantized coefficients.

4. The apparatus of claim 3, wherein said immediate previous encoded portion is an encoded frame and wherein said coding information from said immediate previous encoded portion is used to determine T_i, a target bit rate for a I-frame, where said T_i is expressed as:

##EQU19## where, N_p is a number of P frames in the image sequence, R is a number of remaining bits available for assignment, X_p is a complexity measure for a given P-frame, X_i is a complexity measure for a given I-frame, K_i is a weighting coefficient for an I-frame, and K_p is a weighting coefficient for a P-frame.

5. The apparatus of claim 4, wherein said R and N_p are used to determine T_pⁿ, a target bit rate for an n-th P-frame, where said T_pⁿ is expressed as:

space="preserve" listing-type="equation">T.sub.p.sup.n =R/N.sub.p,

where said R is modified in accordance with R=R-T_i.

6. The apparatus of claim 3, wherein said coding information from said immediate previous encoded frame is used to determine R_jⁿ, a buffer fullness measure before encoding a j-th tree, where said R_jⁿ is expressed as:

##EQU20## where R₀ⁿ is an initial buffer fullness measure, B_jⁿ is a number of bits generated by encoding all of said wavelet trees in a n-th frame up to and including said j-th tree, T_n is a target bit budget in a previous I or P frame and NT is a total number of said plurality of wavelet trees in a current frame.

7. The apparatus of claim 6, wherein said R_jⁿ is used to determine said quantizer scale.

8. apparatus for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said apparatus comprising:

a block motion compensator for computing a motion vector for the block and for generating a predicted image using said motion vector;

a transform module, coupled to said block motion compensator, for applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

a quantizer, coupled to said transform module, for quantizing said plurality of coefficients with a quantizer scale;

a controller, coupled to said quantizer, for selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded macroblock and wherein said coding information from said immediate previous encoded portion is used to adjust a complexity model; and

a coder, coupled to said quantizer, for coding said plurality of quantized coefficients.

9. The apparatus of claim 8, wherein said complexity model has a polynomial form.

10. The apparatus of claim 9, where said polynomial form is expressed as:

##EQU21## where R_i is a number of bits allocated to a macroblock i, Q_i is a quantizer scale of said macroblock i and X₀, X₁ and X₂ are constants.

11. The apparatus of claim 8, wherein said coding information from said immediate previous encoded portion is further used to determine a quantizer scale modifier, γ

, where said quantizer scale modifier is expressed as;

##EQU22## where T_p is a projected number of bits and T is a target number of bits for a current frame of the image sequence.

12. Method for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said method comprising the steps of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded frame and wherein said coding information from said immediate previous encoded portion is used to determine T_P(AVG), a projected average number of bits needed to code a remaining frame, where said T_P(AVG) is expressed as;

space="preserve" listing-type="equation">T.sub.P(AVG) =Max(bitrate/frame rate, R/N)

where R is a remaining number of bits, N is a remaining number of frames in the image sequence, bitrate is a channel bitrate and frame rate is a frame rate of the image sequence; and

coding said plurality of quantized coefficients.

13. Method for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said method comprising the steps of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients, where said transformation applying step applies a wavelet transform to produce a plurality of wavelet trees;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion; and

coding said plurality of quantized coefficients.

14. The method of claim 13, wherein said coding information from said immediate previous encoded frame is used to determine a buffer fullness measure before encoding a j-th tree.

15. The method of claim 14, wherein said buffer fullness measure is used to determine said quantizer scale.

16. Method for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said method comprising the steps of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded macroblock and wherein said coding information from said immediate previous encoded portion is used to adjust a complexity model; and

coding said plurality of quantized coefficients.

17. Method for encoding an input image sequence having at least one input frame, where said frame is partitioned into at least one block, said method comprising the steps of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded macroblock and wherein said coding information from said immediate previous encoded portion is used to determine a distortion measure for a current macroblock; and

coding said plurality of quantized coefficients.

18. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded frame and wherein said coding information from said immediate previous encoded portion is used to determine T_P(AVG), a projected average number of bits needed to code a remaining frame, where said T_P(AVG) is expressed as;

space="preserve" listing-type="equation">T.sub.P(AVG) =Max(bitrate/frame rate, R/N)

where R is a remaining number of bits, N is a remaining number of frames in the image sequence, bitrate is a channel bitrate and frame rate is a frame rate of the image sequence; and

coding said plurality of quantized coefficients.

19. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients, where said transformation applying step applies a wavelet transform to produce a plurality of wavelet trees;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion; and

coding said plurality of quantized coefficients.

20. The computer-readable medium of claim 19, wherein said coding information from said immediate previous encoded frame is used to determine a buffer fullness measure before encoding a j-th tree.

21. The computer-readable medium of claim 20, wherein said buffer fullness measure is used to determine said quantizer scale.

22. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded macroblock and wherein said coding information from said immediate previous encoded portion is used to adjust a complexity model; and

coding said plurality of quantized coefficients.

23. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps comprising of:

computing a motion vector for the block;

generating a predicted image using said motion vector;

applying a transformation to a difference signal between the input frame and said predicted image, where said transformation produces a plurality of coefficients;

quantizing said plurality of coefficients with a quantizer scale;

selectively adjusting said quantizer scale for a current frame in response to coding information from an immediate previous encoded portion, wherein said immediate previous encoded portion is an encoded macroblock and wherein said coding information from said immediate previous encoded portion is used to determine a distortion measure for a current macroblock; and

coding said plurality of quantized coefficients.