Rate control for stereoscopic digital video encoding

• US 6,072,831 A
• Filed: 07/15/1997
• Issued: 06/06/2000
• Est. Priority Date: 07/03/1996
• Status: Expired due to Term

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Chen, Xuemin
• Primary Examiner(s): LEE, Y YOUNG
• Application Number:
  US08/893,131
• Time in Patent Office: 1,057 Days
• Field of Search: 348/43, 348/42, 348/397, 348/399, 348/400, 348/401, 348/403, 348/404, 348/405, 348/407, 348/408, 348/409, 348/411, 348/412, 348/415, 348/416
• US Class Current: 375/240.03
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/218 using spatial multiplexing
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 2013/0085 : Motion estimation from stereoscopic image signals

Abstract

Rate control in a stereoscopic digital video communication system is accomplished by modifying the quantization level of P or B-frame data in the enhancement layer depending on whether the frame is temporally predicted (from the same layer) or disparity predicted (from the opposite layer). The invention can maintain a consistent image quality by providing additional quantization bits for disparity-predicted P-pictures, for example, where a P-frame may be encoded from a B-frame in the enhancement layer. The selected quantization level corresponds to an overall bit rate requirement of the enhancement layer. For disparity predicted P-frames, the quantization step size is modified according to the activity level of the frame being encoded in the enhancement layer, or of the reference frame, whichever is greater. Also, image quality is improved and frame freeze up is prevented during editing modes such as fast forward and fast rewind which require random access to the picture data. When the reference frame in the base layer is the first frame of a group of pictures (GOP), the corresponding enhancement layer frame will be encoded as an I or P frame instead of as a B frame to improve image quality and eliminate or reduce error propagation during random access.

Claims

1. A rate control method for a stereoscopic digital data signal having a base layer and an enhancement layer, wherein first and second reference pictures are available for use in coding at least a portion of a current picture of the enhancement layer, comprising the steps of:

selecting one of said first and second reference pictures for use in coding said at least a portion of said current picture according to a selection criteria;

wherein the first reference picture precedes the current picture in the enhancement layer and the second reference picture is concurrent with the current picture in the base layer;

calculating a first number of bits to allocate in coding said at least a portion of said current picture according to whether said first or second reference picture is selected in said selecting step;

calculating a quantization step size for coding said at least a portion of said current picture according to said first number of bits;

and coding said current picture as a disparity-predicted P-picture when said second reference picture is a first picture of a group of pictures of said base layer.

2. A method according to claim 1, comprising the further steps of:

calculating an initial number of bits to allocate for coding a group of pictures of the enhancement layer that comprises the current picture; and

maintaining a running total of remaining bits available as each of said pictures in said group is coded;

wherein said calculating of said first number of bits is responsive to said running total.

3. A method according to claim 1, wherein:

said calculating of said first number of bits is responsive to a desired data rate of said stereoscopic digital data signal.

4. A method according to claim 1, wherein:

said current picture is a disparity-predicted picture (P_D -picture); and

said calculating of said first number of bits increases said first number of bits when there are no intra-coded pictures (I-pictures) in said enhancement layer according to a ratio of a refresh period of the enhancement layer and a number of frames in a group of pictures of the base layer.

5. A method according to claim 1, wherein said second reference picture is selected in said selecting step, comprising the further steps of:

determining a first activity level which is an activity level of at least a portion of a picture which precedes said current picture in said enhancement layer; and

determining a second activity level which is an activity level of at least a portion of said second reference picture;

wherein;

said calculating of said first number of bits is responsive to the greater of said first and second activity levels.

6. A method according to claim 1, wherein said second reference picture is selected in said selecting step, comprising the further steps of:

determining a first activity level which is an activity level of at least a portion of a picture which precedes said current picture in said enhancement layer; and

determining a second activity level which is an activity level of at least a portion of said second reference picture;

wherein;

said calculating of said first number of bits is responsive to an average of said first and second activity levels.

7. A method according to claim 1, comprising the further steps of:

pre-computing and storing a value indicating a first activity level which is an activity level of said at least a portion of said current picture prior to when said at least a portion of said current picture is to be coded;

retrieving said value for use in coding said at least a portion of said current picture; and

determining a second activity level which is an activity level of at least a portion of whichever of said first and second reference pictures is selected in said selecting step;

wherein;

said calculating of said first number of bits is responsive to the greater of said first and second activity levels.

8. A method according to claim 1, comprising the further steps of:

pre-computing and storing a value indicating a first activity level which is an activity level of said at least a portion of said current picture prior to when said at least a portion of said current picture is to be coded;

retrieving said value for use in coding said at least a portion of said current picture; and

determining a second activity level which is an activity level of at least a portion of whichever of said first and second reference pictures is selected in said selecting step;

wherein;

said calculating of said first number of bits is responsive to an average of said first and second activity levels.

9. A method according to claim 1, wherein:

said selection criteria is to provide a minimized prediction error for said current picture.

10. A method according to claim 1, wherein:

said selection criteria is to provide a desired bit allocation for said current picture.

11. A method according to claim 1, wherein:

said selection criteria is to provide a desired image quality for said current picture.

12. A method according to claim 1, wherein:

said current picture has a pre-assigned B-picture type which is switched in said coding step.

13. An apparatus for rate control of a stereoscopic digital data signal having a base layer and an enhancement layer, wherein first and second reference pictures are available for use in coding at least a portion of a current picture of the enhancement layer, comprising:

means for selecting one of said first and second reference for use in coding said at least a portion of said current picture according to a selection criteria;

wherein the first reference picture precedes the current picture in the enhancement layer and the second reference picture is concurrent with the current picture in the base layer;

means for calculating a first number of bits to allocate in coding said at least a portion of said current picture according to whether said first or second reference picture is selected by said selecting means;

means for calculating a quantization step size for coding said at least a portion of said current picture according to said first number of bits;

and means for coding said current picture as a disparity-predicted P-picture when said second reference picture is a first picture of a group of pictures of said base layer.

14. An apparatus according to claim 13, comprising:

means for calculating an initial number of bits to allocate for coding a group of pictures of the enhancement layer that comprises the current picture; and

means for maintaining a running total of remaining bits available as each of said pictures in said group is coded;

wherein said means for calculating said first number of bits is responsive to said running total.

15. An apparatus according to claim 13, wherein:

said means for calculating said first number of bits is responsive to a desired data rate of said stereoscopic digital data signal.

16. An apparatus according to claim 13, wherein:

said current picture is a disparity-predicted picture (P_D -picture); and

said means for calculating said first number of bits increases said first number of bits when there are no intra-coded pictures (I-pictures) in said enhancement layer according to a ratio of a refresh period of the enhancement layer and a number of frames in a group of pictures of the base layer.

17. An apparatus according to claim 13, wherein said second reference picture is selected by said selecting means, further comprising:

means for determining a first activity level which is an activity level of at least a portion of a picture which precedes said current picture in said enhancement layer; and

means for determining a second activity level which is an activity level of at least a portion of said second reference picture;

wherein said means for calculating said first number of bits is responsive to the greater of said first and second activity levels.

18. An apparatus according to claim 13, wherein said second reference picture is selected by said selecting means, further comprising:

means for determining a first activity level which is an activity level of at least a portion of a picture which precedes said current picture in said enhancement layer; and

means for determining a second activity level which is an activity level of at least a portion of said first reference picture;

wherein said means for calculating said first number of bits is responsive to an average of said first and second activity levels.

19. An apparatus according to claim 13, comprising:

means for pre-computing and storing a value indicating a first activity level which is an activity level of said at least a portion of said current picture prior to when said at least a portion of said current picture is to be coded;

means for retrieving said value for use in coding said at least a portion of said current picture; and

means for determining a second activity level which is an activity level of at least a portion of whichever of said first and second reference pictures is selected by said selecting means;

wherein said means for calculating said first number of bits is responsive to the greater of said first and second activity levels.

20. An apparatus according to claim 13, comprising:

means for pre-computing and storing a value indicating a first activity level which is an activity level of said at least a portion of said current picture prior to when said at least a portion of said current picture is to be coded;

means for retrieving said value for use in coding said at least a portion of said current picture; and

means for determining a second activity level which is an activity level of at least a portion of whichever of said first and second reference pictures is selected by said selecting means;

wherein said means for calculating said first number of bits is responsive to an average of said first and second activity levels.

21. An apparatus according to claim 13, wherein:

said selection criteria is to provide a minimized prediction error for said current picture.

22. An apparatus according to claim 13, wherein:

said selection criteria is to provide a desired bit allocation for said current picture.

23. An apparatus according to claim 13, wherein:

said selection criteria is to provide a desired image quality for said current picture.

24. An apparatus according to claim 13, wherein:

said current picture has a pre-assigned B-picture type which is switched by said coding means.