Coding of motion vector information

• US 8,687,697 B2
• Filed: 04/24/2012
• Issued: 04/01/2014
• Est. Priority Date: 07/18/2003
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Srinivasan, Sridhar, Hsu, Pohsiang, Holcomb, Thomas W., Mukerjee, Kunal, Lin, Bruce Chih-Lung
• Primary Examiner(s): Anyikire, Chikaodili E
• Application Number:
  US13/455,094
• Publication Number:
  US 20120213280A1
• Time in Patent Office: 707 Days
• Field of Search: 37524012-24016, 375/240.23, 382/232, 382/239, 382244-246
• US Class Current: 375/240.16
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 7/52 Systems for transmission of a pulse code modulated video signal with one or more other pulse code modulated signals, e.g. an audio signal or a synchronizing signal assembling of a multiplex stream by combining a video stream with other content or additional data, remultiplexing of multiplex streams, insertion of stuffing bits into the multiplex stream, assembling of a packetised elementary stream at server side H04N21/236; disassembling of a multiplex stream, remultiplexing of multiplex streams, extraction or processing of Service Information, disassembling of packetised elementary stream at client side H04N21/434

Abstract

Techniques and tools for encoding and decoding motion vector information for video images are described. For example, a video encoder yields an extended motion vector code by jointly coding, for a set of pixels, a switch code, motion vector information, and a terminal symbol indicating whether subsequent data is encoded for the set of pixels. In another aspect, an encoder/decoder selects motion vector predictors for macroblocks. In another aspect, a video encoder/decoder uses hybrid motion vector prediction. In another aspect, a video encoder/decoder signals a motion vector mode for a predicted image. In another aspect, a video decoder decodes a set of pixels by receiving an extended motion vector code, which reflects joint encoding of motion information together with intra/inter-coding information and a terminal symbol. The decoder determines whether subsequent data exists for the set of pixels based on e.g., the terminal symbol.

Claims

1. One or more tangible computer-readable storage media, wherein the one or more tangible computer-readable storage media are one or more of volatile memory, non-volatile memory, optical storage media, and magnetic storage media, storing computer-executable instructions for causing a computer system programmed thereby to perform a method of reconstructing one or more video images in a video sequence, the method comprising:

receiving encoded data from a bit stream, wherein the encoded data includes entropy coded data for a macroblock of a video image, the entropy coded data for the macroblock being signaled in the bit stream as part of macroblock syntax at macroblock level, and the entropy coded data for the macroblock representing;

(a) a terminal symbol indicating whether transform coefficient data for the macroblock is included in the bit stream;

(b) intra/inter decision information indicating whether the macroblock is intra-coded or inter-coded, wherein the macroblock is inter-coded;

(c) information indicating which of multiple reference frames is to be used in motion compensation for the inter-coded macroblock; and

(d) motion information for the inter-coded macroblock; and

decoding the macroblock using the encoded data from the bit stream, wherein the decoding the macroblock comprises;

entropy decoding the entropy coded data for the macroblock to determine the terminal symbol, the intra/inter decision information, the infoimation indicating which of the multiple references frames is to be used in motion compensation for the macroblock, and the motion information for the macroblock;

determining whether transform coefficient data for the macroblock is included in the bit stream based at least in part upon the terminal symbol;

reconstructing a motion vector for the macroblock using the motion information for the macroblock; and

reconstructing the macroblock, including performing motion compensation for the macroblock using the motion vector and the indicated one of the multiple reference frames.

2. The one or more computer-readable storage media of claim 1 wherein the motion information represents plural values used to reconstruct a differential motion vector for the macroblock.

3. The one or more computer-readable storage media of claim 1 wherein the entropy decoding comprises Huffman decoding.

4. The one or more computer-readable storage media of claim 1 wherein the entropy decoding comprises arithmetic decoding.

5. The one or more computer-readable storage media of claim 1 wherein the macroblock is a current macroblock in the video image, the current macroblock is a one motion vector macroblock having a single motion vector, and wherein the decoding the current macroblock further comprises:

computing a motion vector predictor for the current macroblock, including;

calculating an initial predictor for the current macroblock based on one or more neighbor motion vector candidates of one or more neighboring macroblocks;

comparing the initial predictor with at least one of the one or more neighbor motion vector candidates; and

based at least in part on results of the comparing, selecting one of the one or more neighbor motion vector candidates as the motion vector predictor for the current macroblock;

wherein the entropy coded data signaled in the bit stream as part of macroblock syntax at macroblock level for the current macroblock further includes a motion indicator for the current macroblock, and wherein the motion indicator indicates use of the selected one of the one or more neighbor motion vector candidates of the one or more neighboring macroblocks.

6. The one or more computer-readable storage media of claim 5 wherein the entropy coded data for the macroblock further represents the motion indicator with the terminal symbol, the intra/inter decision information, the information indicating which of multiple reference frames is to be used, and the motion information for the current macroblock.

7. A method of encoding one or more video images in a video sequence using a computing device that implements a video encoder, the method comprising:

with the computing device that implements the video encoder;

performing motion estimation for a macroblock of a video image, wherein the motion estimation uses, at least in part, one of multiple reference frames, and wherein the motion estimation produces, at least in part, a motion vector;

reconstructing the macroblock, including performing motion compensation for the macroblock using the motion vector and the indicated one of the multiple reference frames;

determining, based at least in part upon the motion estimation and the motion compensation, data for the macroblock of the video image, the data for the macroblock representing;

(a) a terminal symbol indicating whether transform coefficient data for the macroblock is included in a bit stream;

(b) intra/inter decision information indicating whether the macroblock is intra-coded or inter-coded, wherein the macroblock is inter-coded;

(c) information indicating the reference frame of the multiple reference frames used in motion compensation for the inter-coded macroblock; and

(d) motion information for the inter-coded macroblock; and

entropy encoding the data for the macroblock; and

outputting, in the bit stream, the entropy encoded data for the macroblock, wherein the entropy encoded data for the macroblock is signaled in the bit stream as part of macroblock syntax at macroblock level.

8. The method of claim 7 wherein the motion information represents plural values used to reconstruct a differential motion vector for the macroblock.

9. The method of claim 7 wherein the entropy encoding comprises Huffman encoding.

10. The method of claim 7 wherein the entropy encoding comprises arithmetic encoding.

11. The method of claim 7 wherein the macroblock is a current macroblock in the video image, the current macroblock is a one motion vector macroblock having a single motion vector, and wherein the performing motion estimation for the current macroblock comprises:

computing a motion vector predictor for the current macroblock, including;

calculating an initial predictor for the current macroblock based on one or more neighbor motion vector candidates of one or more neighboring macroblocks;

comparing the initial predictor with at least one of the one or more neighbor motion vector candidates; and

based at least in part on results of the comparing, selecting one of the one or more neighbor motion vector candidates as the motion vector predictor for the current macroblock;

wherein the entropy encoded data signaled in the bit stream as part of macroblock syntax at macroblock level for the current macroblock further includes a motion indicator for the current macroblock, and wherein the motion indicator indicates use of the selected one of the one or more neighbor motion vector candidates of the one or more neighboring macroblocks.

12. The method of claim 11 wherein the entropy encoded data for the macroblock further represents the motion indicator with the terminal symbol, the intra/inter decision information, the information indicating which of multiple reference frames is to be used, and the motion information for the current macroblock.

13. A computing device that implements a video encoder, the computing device comprising one or more processing units and memory, the computing device being adapted to perform a method comprising:

performing motion estimation for a macroblock of a video image, wherein the motion estimation uses, at least in part, one of multiple reference frames, and wherein the motion estimation produces, at least in part, a motion vector;

reconstructing the macroblock, including performing motion compensation for the macroblock using the motion vector and the indicated one of the multiple reference frames;

determining, based at least in part upon the motion estimation and the motion compensation, data for the macroblock of the video image, the data for the macroblock representing;

(a) a terminal symbol indicating whether transform coefficient data for the macroblock is included in a bit stream;

(b) intra/inter decision information indicating whether the macroblock is intra-coded or inter-coded, wherein the macroblock is inter-coded;

(c) information indicating the reference frame of the multiple reference frames used in motion compensation for the inter-coded macroblock; and

(d) motion information for the inter-coded macroblock; and

entropy encoding the data for the macroblock; and

outputting, in the bit stream, the entropy encoded data for the macroblock, wherein the entropy encoded data for the macroblock is signaled in the bit stream as part of macroblock syntax at macroblock level.

14. The computing device of claim 13 wherein the motion information represents plural values used to reconstruct a differential motion vector for the macroblock.

15. The computing device of claim 13 wherein the entropy encoding comprises Huffman encoding.

16. The computing device of claim 13 wherein the entropy encoding comprises arithmetic encoding.

17. The computing device of claim 13 wherein the macroblock is a current macroblock in the video image, the current macroblock is a one motion vector macroblock having a single motion vector, and wherein the performing motion estimation for the current macroblock comprises:

computing a motion vector predictor for the current macroblock, including;

calculating an initial predictor for the current macroblock based on one or more neighbor motion vector candidates of one or more neighboring macroblocks;

comparing the initial predictor with at least one of the one or more neighbor motion vector candidates; and

based at least in part on results of the comparing, selecting one of the one or more neighbor motion vector candidates as the motion vector predictor for the current macroblock;

wherein the entropy encoded data signaled in the bit stream as part of macroblock syntax at macroblock level for the current macroblock further includes a motion indicator for the current macroblock, and wherein the motion indicator indicates use of the selected one of the one or more neighbor motion vector candidates of the one or more neighboring macroblocks.

18. The computing device of claim 17 wherein the entropy encoded data for the macroblock further represents the motion indicator with the terminal symbol, the intra/inter decision information, the information indicating which of multiple reference frames is to be used, and the motion information for the current macroblock.

19. The computing device of claim 13 further comprising a wireless communication connection, a display, and one or more speakers.

20. The computing device of claim 13 further comprising a display and a wireless communication connection, the bit stream being signaled over the wireless communication connection.