FRACTIONAL INTERPOLATION FOR HARDWARE-ACCELERATED VIDEO DECODING

• US 20130287114A1
• Filed: 06/27/2013
• Published: 10/31/2013
• Est. Priority Date: 06/30/2007
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Arevalo Baeza, Juan Carlos, Dinu, Daniel, Chen, William, Wu, Yongjun
• Application Number:
  US13/929,573
• Publication Number:
  US 20130287114A1
• Time in Patent Office: Days
• US Class Current: 375/240.17
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/112 according to a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 : Motion estimation or motion compensation
• CPC Class Codes: H04N 19/593 : involving spatial prediction techniques
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/89 : involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 19/91 : Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

Claims

1-20. -20. (canceled)

21. A method of video decoding using a hardware-accelerated video decoder, the method comprising:

classifying plural blocks according to plural motion vector types, wherein the plural motion vector types differ in terms of complexity of sample value interpolation; and

with a graphics processing unit, performing motion compensation operations for the plural blocks in plural passes corresponding to the plural motion vector types, respectively.

22. The method of claim 21 wherein each of the plural motion vector types is associated with a quantum of work for the motion vector type, and wherein the quantum of work for each of the plural motion vector types is 8×

8 block.

23. The method of claim 21 wherein plural motion vectors for the plural blocks are applied for 4×

4 blocks in the motion compensation operations.

24. The method of claim 21 wherein the plural motion vector types are:

an integer motion vector type that represents motion vectors with offsets at integer sample positions;

a center offset motion vector type that represents motion vectors with offsets at fractional sample positions whose sample values depend on sample values at sample positions with ½

-pixel horizontal offset and ½

-pixel vertical offset; and

an off-center offset motion vector type that represents motion vectors with offsets at other fractional sample positions.

25. The method of claim 21 wherein the plural passes include a first pass for an integer motion vector type, followed by a second pass for a center offset motion vector type, followed by a third pass for an off-center offset motion vector type.

26. The method of claim 21 wherein, for a first pass of the plural passes, the motion compensation operations include fetching sample values from one or more reference pictures in memory.

27. The method of claim 26 wherein the one or more reference pictures for the motion compensation operations are represented as a 3D texture.

28. The method of claim 21 wherein, for a second pass of the plural passes, a center offset shader implements at least some of the motion compensation operations.

29. The method of claim 28 wherein the center offset shader supports fractional interpolation for plural 4×

4 blocks in parallel, and wherein for a given 4×

4 block of sample values, the center offset shader includes operations for;

loading a 9×

9 block of sample values as a support region for filtering;

computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and

computing final sample values by filtering the intermediate sample values in a second direction.

30. The method of claim 21 wherein, for a third pass of the plural passes, an off-center offset shader implements at least some of the motion compensation operations.

31. The method of claim 30 wherein the off-center offset shader supports fractional interpolation for plural 4×

4 blocks in parallel, and wherein for a given 4×

4 block of sample values, the off-center offset shader includes operations for;

loading sample values as a support region for filtering; and

computing final sample values by filtering in one or more directions, without buffering intermediate sample values from block-to-block for use in other interpolation operations.

32. A computer system that includes a hardware-accelerated video decoder adapted to perform a method of video decoding, the computer system including:

a classifier module for classifying plural blocks according to plural motion vector types that differ in terms of complexity of sample value interpolation; and

a graphics processing unit adapted to run as an integer pixel shader, center offset shader or off-center offset shader depending on motion vector type.

33. The computer system of claim 32 wherein each of the plural motion vector types is associated with a quantum of work for the motion vector type, and wherein the quantum of work for each of the plural motion vector types is 8×

8 block

34. The computer system of claim 32 wherein the plural motion vectors are applied for 4×

4 blocks in the motion compensation operations.

35. The computer system of claim 32 wherein the integer pixel shader includes operations for fetching sample values from one or more reference pictures.

36. The computer system of claim 35 wherein the center offset shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values depend on sample values at sample positions with ½

-pixel horizontal offset and ½

-pixel vertical offset.

37. The computer system of claim 36 wherein the center offset shader supports fractional interpolation for plural 4×

4 blocks in parallel, and wherein for a given 4×

4 block of sample values, the center offset shader includes operations for;

loading a 9×

9 block of sample values as a support region for filtering;

computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and

computing final sample values by filtering the intermediate sample values in a second direction.

38. The computer system of claim 32 wherein the off-center offset shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values do not depend on sample values at sample positions with ½

-pixel horizontal offset and ½

-pixel vertical offset.

39. The computer system of claim 38 wherein the off-center offset shader supports fractional interpolation for plural 4×

4 blocks in parallel, and wherein for a given 4×

4 block of sample values, the off-center offset shader includes operations for;

loading sample values as a support region for filtering; and

computing final sample values by filtering in one or more directions, without buffering intermediate sample values from block-to-block for use in other interpolation operations.

40. A computer system that includes a hardware-accelerated video decoder adapted to perform a method of video decoding, the computer system including:

a graphics processing unit adapted to run as an integer pixel shader, center offset shader or off-center offset shader depending on motion vector type, wherein the center offset pixel shader includes operations for interpolation to determine sample values at fractional sample positions whose sample values depend on sample values at sample positions with ½

-pixel horizontal offset and ½

-pixel vertical offset, wherein the center offset shader supports fractional interpolation for plural 4×

4 blocks in parallel, and wherein for a given 4×

4 block of sample values, the center offset shader includes operations for;

loading a 9×

9 block of sample values as a support region for filtering;

computing intermediate sample values by filtering loaded sample values in a first direction, wherein at least some of the intermediate sample values are buffered from block-to-block for use in other interpolation operations; and

computing final sample values by filtering the intermediate sample values in a second direction.