Drifting reduction and macroblock-based control in progressive fine granularity scalable video coding

• US 20050147164A1
• Filed: 03/03/2005
• Published: 07/07/2005
• Est. Priority Date: 12/15/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Shipeng, Wu, Feng, Zhang, Ya-Qin, Sun, Xiaoyan, Zeng, Bing
• Granted Patent: US 7,583,730 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation

Abstract

A motion-compensated video encoding scheme employs progressive fine-granularity layered coding to encode macroblocks of video data into frames having multiple layers, including a base layer of comparatively low quality video and multiple enhancement layers of increasingly higher quality video. Some of the enhancement layers in a current frame are predicted from different quality layers in reference frames. The video encoding scheme estimates drifting errors during the encoding and chooses a coding mode for each macroblock in the enhancement layer to maximize high coding efficiency while minimizing drifting errors.

Claims

1. A system for reducing drifting errors in a video encoding scheme, the system comprising:

a processor;

a video encoder that accepts video data; and

a memory into which a plurality of instructions are loaded, the plurality of instructions performing a method to encode the video data into encoded video data having multi-layered frames, each frame having a plurality of references in multiple layers ranging from a base layer of a low quality to enhancement layers of increasingly higher quality, the method comprising;

selectively choosing a coding mode from a plurality of coding modes for each macroblock in the video data, the plurality of coding modes differing with respect to references used for prediction and reconstruction; and

encoding each macroblock in the enhancement layers with the chosen coding mode.

2. The system of claim 1, wherein at least one of the coding modes uses a different reference for prediction and for reconstruction.

3. The system of claim 1, wherein the plurality of coding modes comprise at least three INTER coding modes.

4. The system of claim 3, wherein one of the INTER coding modes comprises a LPLR mode in which the macroblock is predicted from a previous low quality reference and a high quality reference for a next frame is reconstructed from the same reference.

5. The system of claim 3, wherein one of the INTER coding modes comprises an HPHR mode in which the macroblock is predicted from a previous high quality reference and reconstructed from the same reference.

6. The system of claim 3, wherein one of the INTER coding modes comprises an HPLR mode in which the macroblock is predicted from a previous high quality reference and the high quality reference is reconstructed from a previous low quality reference.

7. A computer-readable medium having computer-executable instructions, which when executed on a processor, direct a computer to:

encode video data into multi-layers frames, each frame having a plurality of references in multiple layers ranging from a base layer of a low quality to enhancement layers of increasingly higher quality, the encoding occurring by;

selectively choosing a coding mode from a plurality of coding modes for each macroblock in the video data, the plurality of coding modes differing with respect to references used for prediction and reconstruction; and

encoding each macroblock in the enhancement layers with the chosen coding mode.

8. The computer-readable medium of claim 7, wherein at least one of the coding modes uses a different reference for prediction and for reconstruction.

9. The computer-readable medium of claim 7, wherein the plurality of coding modes comprise at least three INTER coding modes.

10. The computer-readable medium of claim 9, wherein one of the INTER coding modes comprises a LPLR mode in which the macroblock is predicted from a previous low quality reference and a high quality reference for a next frame is reconstructed from the same reference.

11. The computer-readable medium of claim 9, wherein one of the INTER coding modes comprises an HPHR mode in which the macroblock is predicted from a previous high quality reference and reconstructed from the same reference.

12. The computer-readable medium of claim 9, wherein one of the INTER coding modes comprises an HPLR mode in which the macroblock is predicted from a previous high quality reference and the high quality reference is reconstructed from a previous low quality reference.

13. The computer-readable medium of claim 7, wherein the coding mode is selected based on an iterative drifting model that controls the drifting error in the coding scheme.

14. The computer-readable medium of claim 7, wherein the coding mode is selected based on a temporal prediction of each frame.

15. The computer-readable medium of claim 7, wherein each coding mode is utilized for at least one of the macroblocks in the video data.

16. The computer-readable medium of claim 7, wherein at least two of the plurality of coding modes are utilized when encoding the macroblocks in the enhancement layers.

17. The computer-readable medium of claim 7, wherein the video encoding scheme comprises a progressive fine granularity scalable (PFGS) video encoding.

18. At least one computer-readable medium having computer-executable instructions for performing a method, comprising:

encoding macroblocks of video data to produce a first bitstream representing a base layers;

selecting a coding mode for each macroblock, the coding mode being selected from a group comprising at least three coding modes where each coding mode specifies a prediction reference and a reconstruction reference, the prediction reference determining a predicted macroblock for a current frame and the reconstruction reference determining a reference for a next frame;

encoding each macroblock of the video data using the coding mode selected for the macroblock, the encoding of the macroblock producing a second bitstream representing one or more enhancement layers, the group of at least three coding modes comprising;

an LPLR coding mode that specifies the prediction reference as a low quality reference in a previous frame and the reconstruction reference as the low quality reference in the previous frame;

an HPHR coding mode that specifies the prediction reference as a high quality reference in a previous frame and the reconstruction reference as the high quality reference in the previous frame; and

an HPLR coding mode that specifies the prediction reference as a high quality reference in a previous frame and the reconstruction reference as a low quality reference in the previous frame, wherein the high quality reference in the previous frame is not the same as the low quality reference in the previous frame and wherein at least two of the coding modes are utilized to produce the second bitstream.

19. The computer-readable medium of claim 18, wherein encoding macroblocks to produce the first and second bitstream further comprises estimating drifting information that occurs from the encoding and the coding mode is selected from the group based upon a derivation that uses the estimated drifting information to maximize coding efficiency while minimizing drifting error.