Method of operating a video decoding system

• US 20030185298A1
• Filed: 04/01/2002
• Published: 10/02/2003
• Est. Priority Date: 04/01/2002
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Alvarez, Jose?apos; R., MacInnis, Alexander G., Zhong, Sheng, Hsiun, Vivian, Xie, Xiaodong
• Granted Patent: US 7,034,897 B2
• Time in Patent Office: Days
• US Class Current: 375/240.03
• CPC Class Codes: G06F 9/3861 Recovery, e.g. branch miss-prediction, exception handling error detection or correction G06F11/00
• CPC Class Codes: G06F 9/3877 using a slave processor, e.g. coprocessor peripheral processor G06F13/12; vector processor G06F15/8053
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/122 Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
• CPC Class Codes: H04N 19/129 Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/157 Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals

Abstract

A system and method for decoding a digital video data stream. In one aspect, a plurality of hardware acceleration modules are used together with a core processor. The accelerators operate in a decoding pipeline wherein, in any given stage, each accelerator operates on a particular macroblock of video data. In the subsequent pipeline stage, each accelerator works on the next macroblock in the data stream, which was worked on by another one of the accelerators in the previous stage. The core processor polls all of the accelerators during each stage. When all accelerators finish their tasks for a given stage, the core processor initiates the next stage. In another aspect, two variable-length decoders are employed to simultaneously decode two macroblock rows of a video frame. Each variable-length decoder works to decode an assigned row and the rows are variable-length decoded in parallel. The variable-length decoders operate as part of a pipeline wherein the variable-length decoders alternate, stage-by-stage, decoding macroblocks.

Claims

1. A method of decoding an encoded digital data stream, comprising:

(a) executing a first stage comprising;

(i) performing a first decoding function on an n^th data element of the data stream; and

(ii) simultaneously performing a second decoding function on an n+1^st data element of the data stream; and

(b) when the first function is completed with respect to the n^th data element and the second function is completed with respect to the n+1^st data element, executing a second stage comprising;

(i) performing the first decoding function on the n+1^st data element; and

(ii) simultaneously performing the second decoding function on an n+2^nd element in the data stream.

2. The method of claim 1 further comprising:

(c) sequentially executing additional stages in which the first decoding function is performed on the data element processed by the second decoding function in the previous stage and the second decoding function is performed on the next data element in the data stream, and wherein each successive stage begins when both decoding functions from the pervious stage have been completed.

3. The method of claim 1 wherein the second decoding function for a given data element is dependent upon a result of the first decoding function for the given data element.

4. The method of claim 3 wherein the second decoding function for a given data element is performed on a result of the first decoding function for the given data element.

5. The method of claim 1 wherein performing a decoding function on a data element according to steps (a)(i), (a)(ii), (b)(i) and (b)(ii) comprises performing a decoding function on a processed version of the associated data element.

6. The method of claim 1 wherein the first stage further comprises a step (a)(iii) of performing a third decoding function on an n+2^nd data element of the data stream while performing the first and second decoding functions according to performing steps (a)(i) and (a)(ii), and wherein the second stage further comprises a step (b)(iii) of performing the third decoding function on an n+3^rd data element in the data stream while performing the first and second decoding functions according to performing steps (b)(i) and (b)(ii).

7. The method of claim 6 further comprising a step (c) of:

(c) when the first function is completed with respect to the n+1^st data element, the second function is completed with respect to the n+2nd data element and the third function is completed with respect to the n+3^rd data element, executing a third stage comprising;

(i) performing the first decoding function on the n+2^nd data element; and

(ii) simultaneously performing the second decoding function on the n+3^rd element in the data stream; and

(iii) simultaneously performing the third decoding function on an n+4^th element in the data stream.

8. The method of claim 1 further comprising a step (c) of:

(c) when the first function is completed with respect to the n+1^st data element and the second function is completed with respect to the n+2^nd data element, executing a third stage comprising;

(i) performing the first decoding function on the n+2^nd data element; and

(ii) simultaneously performing the second decoding function on an n+3^rd element in the data stream.

9. The method of claim 1 wherein the first stage further comprises a step (iii) of simultaneously performing a third decoding function on one of the n^th and the n+1^st data elements, and wherein the second stage is not executed until the first function is completed with respect to the n^th data element, the second function is completed with respect to the n+1^st data element and the third function is completed with respect to the one of the n^th and n+1^st data elements.

10. The method of claim 1 wherein the encoded digital data stream is a media data stream.

11. The method of claim 10 wherein the media data stream is a video data stream.

12. The method of claim 11 wherein the data elements represent macroblocks of a digital video image.

13. A method of decoding a digital video data stream, comprising:

(a) in a first stage, performing variable-length decoding on an n^th data element of the data stream; and

(b) in a second stage, performing inverse quantization on a product of the variable-length decoding of the n^th data element, while performing variable-length decoding on an n+1^st data element in the data stream, wherein the second stage begins when the variable-length decoding of the first stage is completed.

14. The method of claim 13 further comprising:

(c) in a third stage, performing an inverse transform operation on a product of the inverse quantization of the n^th data element while performing inverse quantization on a product of the variable-length decoding of the n+1^st data element and performing variable-length decoding on an n+2^nd data element in the data stream, wherein the third stage begins when the last of the variable-length decoding and the inverse quantization of the second stage is completed.

15. The method of claim 14 wherein performing step (c) further comprises:

in the third stage, performing pixel filtering on the n^th data element while performing the inverse transform operation on the product of the inverse quantization of the n^th data element.

16. The method of claim 15 further comprising:

(d) in a fourth stage, performing motion compensation on the n^th data element using a product of the inverse transform operation performed on the n^th data element and a product of the pixel filtering performed on the n^th data element, while performing an inverse transform operation on a product of the inverse quantization of the n+1^st data element, performing inverse quantization on a product of the variable-length decoding of the n+2^nd data element, and performing variable-length decoding on an n+3^rd data element in the data stream, wherein the fourth stage begins when the last of the variable-length decoding, the inverse quantization and the inverse transform operation of the third stage is completed.

18. The method of claim 13 wherein the data elements represent macroblocks of a digital video image.

17. The method of 16 further comprising:

(e) iteratively repeating performing step (d), incrementing n with each iteration, as long as the data stream contains further data elements to be decoded.

19. A method of decoding an encoded digital data stream, comprising:

(a) in a first stage, initiating performance of a first decoding function on a first data element by a first decoding element;

(b) in a second stage, initiating performance of the first decoding function on a second data element by a second decoding element; and

(c) in a third stage, performing a second decoding function on the first data element and initiating performance of the first decoding function on a third data element by the first decoding element.

20. The method of claim 19 wherein step (b) further comprises completing performance of the first decoding function on the first data element by the first decoding element during the second stage.

21. The method of claim 20 wherein step (c) further comprises completing performance of the first decoding function on the second data element by the second decoding element during the third stage.

22. The method of claim 19 further comprising:

(d) in each of a plurality of subsequent sequential stages, initiating performance of the first decoding function on a next data element to be decoded and performing the second decoding function on the data element on which the first decoding function was initiated two stages prior, wherein the first decoding function in each successive stage is performed alternately by the first and second decoding elements.

23. The method of claim 22 wherein each successive stage begins when the second decoding function from the previous stage is completed.

24. The method of claim 22 wherein the first data element is a first macroblock of a first row of a video frame, the second data element is a first macroblock of a second row of the video frame and the third data element is a second macroblock of the first row of the video frame, and wherein the first decoding function in each subsequent stage is performed alternately on subsequent macroblocks of the first and second rows of the video frame, the first decoding element performing the first decoding function on the macroblocks of the first row and the second decoding element performing the first decoding function on the macroblocks of the second row.

25. The method of claim 19 farther comprising:

(d) in a fourth stage, performing the second decoding function on the second data element and initiating performance of the first decoding function on a fourth data element by the second decoding element.

26. The method of claim 25 wherein the fourth stage begins when the second decoding function is completed with respect to the first data element.

27. The method of claim 26 wherein step (d) further comprises performing a third decoding function on the first data element while performing the second decoding function on the second data element.

28. The method of claim 27 further comprising:

(e) in each of a plurality of subsequent sequential stages, initiating performance of the first decoding function on a next data element to be decoded, performing the second decoding function on the data element on which the first decoding function was initiated two stages prior and performing the third decoding function on the data element that was operated on by the second decoding function in the prior stage, wherein the first decoding function in each successive stage is performed alternately by the first and second decoding elements.

29. The method of claim 28 wherein each successive stage begins when the second and third decoding functions from the previous stage are completed.

30. The method of claim 27 further comprising:

(e) in a fifth stage, performing the second decoding function on the third data element while performing the third decoding function on the second data element, and initiating performance of the first decoding function on a fifth data element by the second decoding element.

31. The method of claim 30 wherein the fifth stage begins when the second decoding function is completed with respect to the second data element and the third decoding function is completed with respect to the first data element.

32. The method of claim 30 wherein step (d) further comprises performing a fourth decoding function on the first data element while performing the third decoding function on the second data element and the second decoding function on the third data element.

33. The method of claim 19 wherein the second decoding function for a given data element is dependent upon a result of the first decoding function for the given data element.

34. The method of claim 33 wherein the second decoding function for a given data element is performed on a result of the first decoding function for the given data element.

35. The method of claim 19 wherein performing a decoding function on a data element according to steps (a), (b) and (c) comprises performing a decoding function on a processed version of the associated data element.

36. The method of claim 19 wherein the encoded digital data stream is a media data stream.

37. The method of claim 36 wherein the media data stream is a video data stream.

38. The method of claim 37 wherein the data elements represent macroblocks of a digital video image.

39. The method of claim 38 wherein the first data element is a first macroblock of a first row of a video frame, the second data element is a first macroblock of a second row of the video frame and the third data element is a second macroblock of the first row of the video frame.

40. A method of decoding an encoded digital video data stream, comprising:

(a) in a first stage, initiating variable-length decoding on a first macroblock with a first variable-length decoder;

(b) in a second stage, initiating variable-length decoding on a second macroblock with a second variable-length decoder; and

(c) in a third stage, performing inverse quantization on the first macroblock and initiating variable-length decoding on a third macroblock with the first variable-length decoder.

41. The method of claim 40 wherein step (b) further comprises completing the variable-length decoding of the first macroblock with the first variable-length decoder during the second stage.

42. The method of claim 41 wherein step (c) further comprises completing the variable-length decoding of the second macroblock with the second variable-length decoder during the third stage.

43. The method of claim 40 further comprising:

(d) in each of a plurality of subsequent sequential stages, initiating variable-length decoding on a next macroblock to be decoded and performing inverse quantization on the macroblock on which variable-length decoding was initiated two stages prior, wherein the variable-length decoding in each successive stage is performed alternately by the first and second variable-length decoders.

44. The method of claim 43 wherein each successive stage begins when the inverse quantization from the previous stage is completed.

45. The method of claim 43 wherein the first macroblock is a first macroblock of a first row of a video frame, the second macroblock is a first macroblock of a second row of the video frame and the third macroblock is a second macroblock of the first row of the video frame, and wherein the variable-length decoding in each subsequent stage is performed alternately on subsequent macroblocks of the first and second rows of the video frame, the first variable-length decoder performing the variable-length decoding of the macroblocks of the first row and the second variable-length decoder performing the variable-length decoding of the macroblocks of the second row.

46. The method of claim 40 further comprising:

(d) in a fourth stage, performing inverse quantization on the second macroblock and initiating variable-length decoding on a fourth macroblock with the second variable-length decoder.

47. The method of claim 46 wherein the fourth stage begins when the inverse quantization is completed with respect to the first macroblock.

48. The method of claim 47 wherein step (d) further comprises performing an inverse transform operation on the first macroblock while performing the inverse quantization on the second macroblock.

49. The method of claim 48 further comprising:

(e) in each of a plurality of subsequent sequential stages, initiating variable-length decoding on a next macroblock to be decoded, performing inverse quantization on the macroblock on which variable-length decoding was initiated two stages prior and performing an inverse transform operation on the macroblock that was inverse quantized in the prior stage, wherein variable-length decoding in each successive stage is performed alternately by the first and second variable-length decoders.

50. The method of claim 49 wherein each successive stage begins when the inverse quantization and inverse transform operations from the previous stage are completed.

51. The method of claim 48 further comprising:

(e) in a fifth stage, performing inverse quantization on the third macroblock while performing an inverse transform operation on the second macroblock, and initiating variable-length decoding on a fifth macroblock with the second variable-length decoder.

52. The method of claim 51 wherein the fifth stage begins when both the inverse quantization of the second macroblock and the inverse transform operation performed on the first macroblock are completed.

53. The method of claim 51 wherein step (d) further comprises performing motion compensation on the first macroblock while performing an inverse transform operation on the second macroblock and inverse quantization on the third macroblock.

54. The method of claim 40 wherein the inverse quantization for a given macroblock is performed on a result of the variable-length decoding for the given macroblock.

55. The method of claim 40 wherein the first macroblock is a first macroblock of a first row of a video frame, the second macroblock is a first macroblock of a second row of the video frame and the third macroblock is a second macroblock of the first row of the video frame.

56. A method of decoding data elements in a bitstream comprising:

(a) initiating a plurality of external decoding functions, each of the plurality of decoding functions operating on a respective one of a sequence of data elements in the bitstream;

(b) waiting an undefined time period until each of the plurality of external decoding functions has been completed;

(c) initiating the plurality of extended decoding functions, each of the plurality of decoding functions operating on a respective one of a next sequence of data elements in the bitstream; and

(d) repeating steps (b) and (c) for a plurality of different sequences of data elements in the bitstream.

57. The method of claim 56 wherein waiting step (b) comprises waiting a time period corresponding to a slowest one of the plurality of external decoding functions.

58. The method of claim 56 wherein the next sequence of data elements comprises a prior sequence of data elements minus a first data element of the prior sequence plus a next data element in the bitstream.

59. The method of claim 56 wherein the slowest one of the plurality of external decoding functions comprises any one of the plurality of decoding functions.

60. A method of decoding a digital media data stream, comprising:

(a) in a first stage, performing a first decoding function on an n^th data element of the data stream with a first decoding accelerator; and

(b) in a second stage, after the first stage, performing a second decoding function on the n^th data element with a second decoding accelerator, while performing the first decoding function on an n+1^st data element in the data stream with the first decoding accelerator.

61. The method of claim 60 wherein the second decoding function for a given data element is dependent upon a result of the first decoding function for the given data element.

62. The method of claim 60 further comprising:

(c) in a third stage, after the second stage, performing a third decoding function on the n^th data element with a third decoding accelerator, while performing the second decoding function on the n+1^st data element with the second decoding accelerator and performing the first decoding function on an n+2^nd data element in the data stream with the first decoding accelerator.

63. The method of claim 62 wherein the second decoding function for a given data element is dependent upon a result of the first decoding function for the given data element, and wherein the third decoding function for a given data element is dependent upon a result of the second decoding function for the given data element.

64. The method of claim 60 wherein step (a) comprises:

in the first stage, performing a third decoding function on the n^th data element of the data stream with a third decoding accelerator while performing the first decoding function on the n^th data element with the first decoding accelerator, and wherein step (b) comprises;

in the second stage, performing the second decoding function on the n^th data element with a second decoding accelerator, while performing the first and third decoding functions on an n+1^st data element in the data stream with the first and third decoding accelerators, respectively.

65. The method of claim 64 wherein the second decoding function for a given data element is dependent upon a result of both the first decoding function and the third decoding function for the given data element.

66. The method of claim 64 wherein the data stream is a video data stream and wherein the first decoding function comprises an inverse discrete cosine transform operation, the second decoding function comprises a motion compensation operation, and the third decoding operation comprises a pixel filtering operation.

67. The method of claim 60 wherein the data stream is a video data stream.

68. The method of claim 67 wherein the data elements represent macroblocks of a digital video image.