Skip macroblock coding

• US 20030113026A1
• Filed: 12/16/2002
• Published: 06/19/2003
• Est. Priority Date: 12/17/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Srinivasan, Sridhar, Hsu, Pohsiang
• Granted Patent: US 7,200,275 B2
• Time in Patent Office: Days
• US Class Current: 382/239
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/129 Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/507 using conditional replenishment
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 : involving reduction of coding artifacts, e.g. of blockiness

Abstract

Various techniques and tools for encoding and decoding (e.g., in a video encoder/decoder) binary information (e.g., skipped macroblock information) are described. In some embodiments, the binary information is arranged in a bit plane, and the bit plane is coded at the picture/frame layer. The encoder and decoder process the binary information and, in some embodiments, switch coding modes. For example, the encoder and decoder use normal, row-skip, column-skip, or differential modes, or other and/or additional modes. In some embodiments, the encoder and decoder define a skipped macroblock as a predicted macroblock whose motion is equal to its causally predicted motion and which has zero residual error. In some embodiments, the encoder and decoder use a raw coding mode to allow for low-latency applications.

Claims

1. In a computer system, a computer-implemented method of processing one or more video images, the method comprising:

selecting a coding mode from a group of plural available coding modes; and

processing a bit plane according to the selected coding mode, wherein the bit plane includes binary information for plural units of a video image, wherein the binary information represents characteristics of the plural units of the video image, wherein each of the plural units includes plural pixels, and wherein the binary information includes one bit for each of the plural units of the video image.

2. The method of claim 1 wherein the processing is performed at picture level.

3. The method of claim 1 wherein the group of plural available coding modes comprises a row-prediction coding mode and a column-prediction coding mode.

4. The method of claim 1 wherein the group of plural available coding modes further comprises one or more vector variable length coding modes.

5. The method of claim 1 wherein the group of plural available coding modes further comprises one or more differential coding modes.

6. The method of claim 1 wherein the plural units are macroblocks, and wherein the binary information in the bit plane comprises skipped macroblock information.

7. The method of claim 1 wherein the plural units are macroblocks, and wherein the binary information in the bit plane comprises motion vector count information.

8. The method of claim 1 wherein the plural units are macroblocks, and wherein the binary information in the bit plane comprises field/frame flags.

9. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 1 during video decoding, wherein the processing comprises entropy decoding according to the selected coding mode.

10. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 1 during video encoding, wherein the processing comprises entropy encoding according to the selected coding mode.

11. In a computer system, a computer-implemented method of processing plural video images in a video image sequence, the method comprising:

selecting a coding mode from a group of plural available coding modes; and

processing a bit plane according to the selected coding mode, wherein the bit plane includes binary information signifying whether macroblocks in a predicted video image are skipped or not skipped, and wherein a macroblock in the predicted video image is skipped if the motion for the macroblock is equal to the predicted motion of the macroblock and the macroblock has no residual error.

12. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 11 during video encoding.

13. In a computer system, a method of processing plural video images of a video sequence, the plural video images including one or more predicted video images, wherein each of the one or more predicted video images includes plural macroblocks, the method comprising:

processing one or more skipped macroblocks among the plural macroblocks of at least one of the one or more predicted video images, wherein each of the at least one of the one or more predicted video images is predicted from no more than one reference video image, wherein each of the one or more skipped macroblocks uses predicted motion for the skipped macroblock based upon motion of one or more other macroblocks, and wherein each of the one or more skipped macroblocks lacks residual information.

14. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 13 during video encoding.

15. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 13 during video decoding.

16. In a computer system, a method of processing plural video images of a video sequence, the plural video images including plural predicted macroblocks, wherein each of the plural predicted macroblocks is predicted from no more than one reference video image, the method comprising:

processing one or more skipped macroblocks among the plural predicted macroblocks, wherein each of the one or more skipped macroblocks uses predicted motion for the skipped macroblock based upon motion of one or more other predicted macroblocks, and wherein each of the one or more skipped macroblocks lacks residual information.

17. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 16 during video encoding.

18. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 16 during video decoding.

19. In a computer system, a computer-implemented method of processing one or more images in a video image sequence, wherein a bitstream syntax for the video image sequence includes plural hierarchical layers, the plural hierarchical layers including at least a sequence layer, a picture layer, and a macroblock layer, the method comprising:

processing a bit plane at a layer higher than the macroblock layer in the bitstream syntax for the video image sequence, wherein the bit plane includes binary information for plural macroblocks in an image in the video image sequence, wherein each of the plural macroblocks includes plural blocks, wherein each of the plural blocks includes plural pixels, and wherein the binary information includes one bit for each of the plural macroblocks in the image.

20. The method of claim 19 wherein the binary information comprises skipped macroblock information.

21. The method of claim 19 wherein the binary information comprises motion vector count information.

22. The method of claim 19 wherein the binary information comprises interlace mode information.

23. The method of claim 19 wherein the higher layer is the picture layer.

24. The method of claim 19 wherein a header at the higher layer includes the bit plane during transmission.

25. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 19 during video encoding.

26. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 19 during video decoding.

27. In a computer system, a computer-implemented method of processing one or more images in a video image sequence, wherein a bitstream syntax for the video image sequence includes plural hierarchical layers, the plural hierarchical layers including at least a picture layer and a macroblock layer, the method comprising:

processing skipped macroblock information at a layer higher than the macroblock layer in the bitstream syntax for the video image sequence, wherein the skipped macroblock information is for plural macroblocks in an image in the video image sequence, wherein each of the plural macroblocks includes plural blocks, wherein each of the plural blocks includes plural pixels.

28. The method of claim 27 wherein the higher layer is the picture layer.

29. The method of claim 27 wherein a header at the higher layer includes the skipped macroblock information.

30. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 27 during video encoding.

31. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 27 during video decoding.

32. In a computer system, a computer-implemented method of processing plural images in a video image sequence, wherein skipped macroblock information is associated with the plural images, the method comprising:

selecting a coding mode from a group of plural available coding modes, wherein at least two of the plural available coding modes involve reduction of bitrate associated with the skipped macroblock information; and

processing the skipped macroblock information according to the selected coding mode.

33. The method of claim 32 wherein the group of plural available coding modes comprises a row-prediction mode, a column-prediction mode, and a normal mode.

34. The method of claim 32 wherein the group of plural available coding modes comprises a row-skip mode, a column-skip mode, a differential mode, an entropy coding mode, and a raw mode.

35. The method of claim 32 wherein the processing is low latency processing, and wherein the selected coding mode is a one-pass macroblock-level coding mode.

36. The method of claim 32 wherein a bitstream syntax for the video image sequence includes plural hierarchical layers, the plural hierarchical layers including a sequence layer, a picture layer, and a macroblock layer, and wherein the processing occurs at the picture layer.

37. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 32 during video encoding.

38. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 32 during video decoding.

39. In a computer system, a computer-implemented method of processing plural units in a video image sequence, wherein binary information represents characteristics of plural units, wherein each of the plural units includes plural pixels, and wherein the binary information includes one bit for each of the plural units, the method comprising:

selecting a coding mode from a group of plural available coding modes, wherein the plural available coding modes include a low latency coding mode; and

if the selected coding mode is the low latency coding mode, processing the binary information according to the selected coding mode at a unit layer in a bitstream syntax for the video image sequence,

otherwise, processing the binary information as a bit plane according to the selected coding mode at a layer higher than the unit layer in the bitstream syntax for the video image.

40. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 39 during video encoding.

41. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 39 during video decoding.

42. In a computer system, a computer-implemented method of processing plural units in a video image sequence, wherein a bitstream syntax for the video image sequence includes plural hierarchical layers, the plural hierarchical layers including a sequence layer and a unit layer, the method comprising:

selecting a coding mode from a group of plural available coding modes, wherein the plural available coding modes include a low latency coding mode; and

if the selected coding mode is the low latency coding mode, processing binary information according to the selected coding mode at the unit layer,

otherwise, processing the binary information as a bit plane according to the selected coding mode at a layer higher than the unit layer.

43. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 42 during video encoding.

44. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 42 during video decoding.

45. In a computer system, a computer-implemented method of compressing a bit plane comprising binary symbols in a matrix arranged in rows or columns, the method comprising:

for each row in the bit plane where each binary symbol in the row is equal to a first value, or for each column in the bit plane where each binary symbol in the column is equal to a first value, encoding the row or column with a single bit in an encoded bit stream; and

for each row in the bit plane where at least one binary symbol in the row is not equal to the first value, or for each column in the bit plane where at least one binary symbol in the column is not equal to the first value, encoding the row or column with one bit per binary symbol in the row or column.

46. The method of claim 45 wherein the binary symbols in the matrix represent skipped macroblock information in a video sequence.

47. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 45 during video encoding.

48. In a computer system, a computer-implemented method of processing one or more video images, the method comprising:

processing a bit plane according to a coding mode, wherein the bit plane includes binary information for plural units of a video image, wherein the binary information represents characteristics of the plural units of the video image, wherein each of the plural units includes plural pixels, wherein the binary information includes one bit for each of the plural units of the video image, and wherein an invert flag indicates whether to invert the bit plane as part of the processing.

49. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 48 during video encoding, wherein the processing includes entropy encoding according to the coding mode, and wherein inversion of the bit plane occurs, if at all, before the entropy encoding.

50. A computer-readable medium storing computer-executable instructions for causing the computer system to perform the method of claim 48 during video decoding, wherein the processing includes entropy decoding according to the coding mode, and wherein inversion of the bit plane occurs, if at all, after the entropy decoding.