NEIGHBOR DETERMINATION IN VIDEO DECODING

• US 20140098890A1
• Filed: 12/09/2013
• Published: 04/10/2014
• Est. Priority Date: 06/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sermadevi, Yegnaswamy, Zhao, Weidong, Wu, Yongjun
• Granted Patent: US 9,554,134 B2
• Time in Patent Office: Days
• US Class Current: 375/240.24
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 : Motion estimation or motion compensation
• CPC Class Codes: H04N 19/593 : involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/89 : involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 19/91 : Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

Claims

1-20. -20. (canceled)

21. A method of video decoding using a video decoder, the method comprising:

with the video decoder, getting one or more tables indicating neighbor availability relationships between macroblocks, blocks and/or sub-blocks; and

with the video decoder, using the one or more tables to determine neighbor availability during decoding operations.

29. The method of claim 21 wherein the one or more tables include:

an availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

30. The method of claim 29 wherein the using method further comprises:

setting up a state machine for plural macroblocks in a slice, wherein the state machine associates the plural macroblocks with macroblock neighbor availability patterns.

31. The method of claim 30 wherein the using the one or more tables comprises, during the decoding operations for a current partition of a current macroblock:

determining macroblock neighbor availability using the state machine; and

determining sub-macroblock neighbor availability using the macroblock neighbor availability and the availability table.

22. The method of 21 wherein the decoding operations include one or more of context-adaptive binary arithmetic decoding, spatial intra prediction, mode computation for intra prediction, and context-adaptive variable length decoding.

23. The method of 21 wherein the using the one or more tables involves a hierarchical determination at macroblock level then sub-macroblock level.

24. The method of 21 wherein the using the one or more tables includes storing location information for neighboring macroblocks, and wherein the location information is stored as offsets from a current macroblock.

25. The method of 21 wherein the one or more tables include:

a first availability table that associates different index values with different macroblock neighbor availability patterns; and

a second availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

26. The method of claim 25 wherein the method further comprises:

setting up a state machine for plural macroblocks in a slice, wherein the state machine associates the plural macroblocks with respective ones of the index values for the first availability table.

27. The method of claim 26 wherein the using the one or more tables comprises, during the decoding operations, for a current partition of a current macroblock:

determining macroblock neighbor availability using the state machine and the first availability table; and

determining sub-macroblock neighbor availability using the macroblock neighbor availability and the second availability table.

28. The method of 26 wherein, for a given state, the state machine stores information indicating number of consecutive macroblocks in the state and one of the index values for the first availability table to indicate macroblock neighbor availability for the state.

32. The method of 30 wherein, for a given state, the state machine stores information indicating number of consecutive macroblocks in the state and one of the macroblock neighbor availability patterns to indicate macroblock neighbor availability for the state.

33. The method of 21 wherein the method further comprises:

setting up a state machine for plural macroblock pairs in a slice; and

setting up a neighbor context vector for each macroblock of the plural macroblock pairs in the slice.

34. The method of 33 wherein the using the one or more tables includes, during the decoding operations for a current partition of a current macroblock:

determining macroblock pair neighbor availability using the state machine and a first availability table, wherein the first availability table associates different index values with different macroblock pair neighbor availability patterns;

determining sub-macroblock neighbor availability using the macroblock pair neighbor availability and a second availability table that associates different macroblock pair neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

35. The method of 33 wherein the using the one or more tables includes, during the decoding operations for a current partition of a current macroblock:

determining macroblock pair neighbor availability using the state machine; and

determining sub-macroblock neighbor availability using the macroblock pair neighbor availability and an availability table that associates different macroblock pair neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

36. The method of 33 wherein, for each macroblock of the plural macroblock pairs in the slice, the neighbor context vector indicates:

field or frame mode for a macroblock pair including the macroblock;

field or frame mode for each of plural neighbor macroblock pairs; and

whether the current macroblock is a top or bottom macroblock in its macroblock pair.

38. The one or more computer-readable storage media of claim 36 wherein the using the one or more tables involves a hierarchical determination at macroblock level then sub-macroblock level.

39. The one or more computer-readable storage media of claim 36 wherein the one or more tables include:

a first availability table that associates different index values with different macroblock neighbor availability patterns; and

a second availability table that associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns.

37. One or more computer-readable storage media storing computer-executable instructions for causing a computer system programmed thereby to perform a method of video decoding, the method comprising:

getting one or more tables indicating neighbor availability relationships between macroblocks, blocks and/or sub-blocks; and

using the one or more tables to determine neighbor availability during decoding operations.

40. A computer system that includes a video decoder adapted to perform a method of video decoding, the computer system including:

means for getting a first availability table and second availability table, wherein;

the first availability table associates different index values with different macroblock neighbor availability patterns;

the second availability table associates different macroblock neighbor availability patterns and partition locations with different sub-macroblock neighbor availability patterns; and

means for setting up a state machine for plural macroblocks in a slice, wherein the state machine associates the plural macroblocks with the index values for the first availability table;

means for determining macroblock neighbor availability using the state machine and the first availability table; and

means for determining sub-macroblock neighbor availability using the macroblock neighbor availability and the second availability table.