Complexity-based adaptive preprocessing for multiple-pass video compression

• US 8,238,424 B2
• Filed: 02/09/2007
• Issued: 08/07/2012
• Est. Priority Date: 02/09/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Chang, Cheng, Lin, Chih-Lung, Holcomb, Thomas W.
• Primary Examiner(s): Truong, Thanhnga B
• Assistant Examiner(s): LANE, GREGORY A
• Application Number:
  US11/673,516
• Publication Number:
  US 20080192822A1
• Time in Patent Office: 2,006 Days
• Field of Search: 375/240.01, 375/240.03, 375/240.06, 375/240.16, 348/14.07, 382/236, 382/239
• US Class Current: 375/240.06
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Multiple-pass video encoding systems and techniques are described which utilize statistics taken during a first-pass encoding to create complexity measurements for video data which is to be encoded. By analyzing these complexity measurements, preprocessing decisions, such as, for example, the determination of strength of denoise filters, can be made with greater accuracy. In one implementation, these complexity measurements take the form of calculation of temporal and spatial complexity parameters, which are then used to compute a unified complexity parameter for each group of pictures being encoded.

Claims

1. A method of determining parameters for pre-processing of a group of one or more pictures, the method comprising:

determining one or more complexity parameters for the group of pictures; and

encoding, using an encoder, the group of pictures in a video stream based at least in part on the one or more complexity parameters to transform the group of one or more pictures into an encoded group of pictures;

wherein determining the one or more complexity parameters is based at least in part on spatial and temporal complexity of the group of pictures and wherein determining one or more complexity parameters comprises;

determining a spatial complexity parameter for the group of pictures;

determining a temporal complexity parameter for the group of pictures;

determining a unified complexity parameter by combining the spatial and temporal complexity parameters; and

adjusting one or both of the spatial and temporal complexity parameters for the group of pictures by a predetermined factor to adjust a relative strength of the parameters and wherein either or both of the spatial and temporal complexity parameters are normalized before calculating a unified complexity parameter.

2. The method of claim 1, wherein determining a spatial complexity parameter for the group of pictures comprises multiplying a quantization parameter for at least one frame found in the group of pictures with a frame size for at least one frame found in the group of pictures.

3. The method of claim 1, wherein the quantization parameter and frame size used in determining a spatial complexity parameter are calculated with reference to an I-frame in the group of pictures.

4. The method of claim 1, wherein determining a temporal complexity parameter comprises:

calculating a temporal complexity parameter for each of one or more P-frames in the group of pictures; and

calculating an average of the temporal complexity parameters calculated for each of the one or more P-frames in the group of pictures.

5. The method of claim 4, wherein calculating a temporal complexity parameter for a frame comprises multiplying a quantization parameter for each of the one or more frames with a frame size for each of the one or more frames.

6. The method of claim 5, wherein calculating a temporal complexity parameter for a frame further comprises dividing the product of the quantization parameter and the frame size by the spatial complexity parameter for that frame.

7. The method of claim 1, wherein determining a unified complexity parameter by combining the spatial and temporal complexity parameters comprises:

after adjusting, multiplying the spatial and temporal complexity parameters together to calculate a unified complexity parameter.

8. The method of claim 1, wherein encoding the group of pictures in a video stream based at least in part on the one or more complexity parameters comprises selecting a filter strength based on the value of the one or more complexity parameters.

9. The method of claim 8, wherein selecting a filter strength based on the value of the one or more complexity parameters comprises:

scaling a unified complexity parameter by a predetermined exponential value; and

normalizing the scaled parameter to a predetermined set of filter-strength values.

10. A system for encoding video, comprising:

a first-pass video encoding module in an encoding device, configured to analyze one or more frames in a video sequence and to calculate one or more encoding parameters to be used in encoding the one or more frames in the video sequence;

a complexity-based adaptive preprocessing module, configured to determine one or more complexity parameters for the one or more frames and to determine preprocessing filters to be used during encoding the one or more frames based on the one or more complexity parameters, the complexity parameters being based on a combined spatial and temporal complexity wherein one or both of the spatial or temporal complexity parameters are normalized for the one or more frames of the video sequence and scaled to adjust a relative strength thereof; and

a second-pass video encoding module, configured to apply preprocessing filters to the one or more frames based on the preprocessing filter parameters and to encode the filtered frames into encoded video stream data.

11. The system of claim 10, wherein the first-pass video encoding module is configured to calculate, for each of a plurality of frames in the video sequence, a quantization parameter and a frame size and the adaptive preprocessing module is configured to determine a spatial complexity parameter and a temporal complexity parameter for each group of pictures in the video sequence based on the calculated quantization parameters and frame sizes.

12. The system of claim 11, wherein the adaptive preprocessing module is configured to determine a spatial complexity parameter for a group of pictures in the video sequence by multiplying a quantization parameter for an I-frame by a frame size for that I-frame;

and

wherein the adaptive preprocessing module is configured to determine a temporal complexity parameter for a group of pictures in the video sequence by;

for each of a plurality of P-frames in the group of pictures, multiplying a quantization parameter for the P-frame by a frame size for that P-frame to determine a temporal complexity value for that P-frame;

scaling each temporal complexity value for each P-frame by dividing the temporal complexity value by the spatial complexity value for the group of pictures; and

taking the average of the scaled temporal complexity values for the plurality of P-frames to determine a temporal complexity parameter for the group of pictures.

13. The system of claim 12, wherein the adaptive preprocessing module is further configured to calculate a unified complexity parameter for each group of pictures in the video sequence by:

normalizing temporal and spatial complexity parameters for the group of pictures;

scaling the temporal complexity parameter for the group of pictures by exponentiating it by a first pre-determined value; and

multiplying the scaled temporal complexity for the group of pictures by the temporal complexity for the group of pictures to determine a unified complexity parameter for the group of pictures; and

wherein the adaptive preprocessing module is further configured to determine preprocessing filters by;

scaling the unified complexity parameter for the group of pictures by exponentiating it by a second pre-determined value;

normalizing the scaled unified complexity parameter for the group of pictures to a filter value within a range of filter strength values; and

selecting a filter strength according to the normalized filter value.

14. One or more computer-readable storage devices containing instructions which, when executed by a computer, cause the computer to perform a method for encoding video, the method comprising:

performing a first-pass analysis on one or more frames in a video sequence in order to calculate one or more encoding parameters to be used in encoding the one or more frames in a video sequence;

determining one or more complexity parameters for the one or more frames based on the one or more encoding parameters, the complexity parameters being based on spatial and temporal complexity that are combined by multiplying the spatial and temporal complexity parameters together to calculate a unified complexity parameter, and wherein at least one of the complexity parameters are normalized;

determining preprocessing filters to be used during encoding the one or more frames based on the one or more complexity parameters;

applying preprocessing filters to the one or more frames based on the preprocessing filter parameters; and

performing a second-pass analysis on the one or more frames to encoding the filtered frames into encoded video stream data.

15. The computer readable media of claim 14, wherein performing a first-pass analysis on one or more frames in a video sequence in order to calculate one or more encoding parameters comprises calculating, for each of a plurality of frames in the video sequence, a quantization parameter and a frame size;

and

wherein determining one or more complexity parameters for the one or more frames comprises determining a spatial complexity parameter and a temporal complexity parameter for each group of pictures in the video sequence based on the calculated quantization parameters and frame sizes.

16. The computer-readable media of claim 15, wherein determining one or more complexity parameters for the one or more frames based on the one or more encoding parameters comprises:

determining a spatial complexity parameter for a group of pictures in the video sequence by multiplying a quantization parameter for an I-frame by a frame size for that I-frame; and

determine a temporal complexity parameter for a group of pictures in the video sequence by;

for each of a plurality of P-frames in the group of pictures, multiplying a quantization parameter for the P-frame by a frame size for that P-frame to determine a temporal complexity value for that P-frame;

scaling each temporal complexity value for each P-frame by dividing the temporal complexity value by the spatial complexity value for the group of pictures; and

taking the average of the scaled temporal complexity values for the plurality of P-frames to determine a temporal complexity parameter for the group of pictures.

17. The computer-readable media of claim 16, wherein determining one or more complexity parameters for the one or more frames based on the one or more encoding parameters further comprises calculating a unified complexity parameter for each group of pictures in the video sequence by:

normalizing temporal and spatial complexity parameters for the group of pictures;

scaling the temporal complexity parameter for the group of pictures by exponentiating it by a first pre-determined value; and

multiplying the scaled temporal complexity for the group of pictures by the temporal complexity for the group of pictures to determine a unified complexity parameter for the group of pictures; and

wherein determining preprocessing filters to be used during encoding the one or more frames based on the one or more complexity parameters comprises;

scaling the unified complexity parameter for the group of pictures by exponentiating it by a second pre-determined value;

normalizing the scaled unified complexity parameter for the group of pictures to a filter value within a range of filter strength values; and

selecting a filter strength according to the normalized filter value.