Innovations in video decoder implementations

• US 20090003447A1
• Filed: 06/30/2007
• Published: 01/01/2009
• Est. Priority Date: 06/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wu, Yongjun, Christoffersen, Eric S., Zhao, Weidong
• Granted Patent: US 8,265,144 B2
• Time in Patent Office: Days
• US Class Current: 375/240.160
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Abstract

Video decoding innovations for multithreading implementations and graphics processor unit (“GPU”) implementations are described. For example, for multithreaded decoding, a decoder uses innovations in the areas of layered data structures, picture extent discovery, a picture command queue, and/or task scheduling for multithreading. Or, for a GPU implementation, a decoder uses innovations in the areas of inverse transforms, inverse quantization, fractional interpolation, intra prediction using waves, loop filtering using waves, memory usage and/or performance-adaptive loop filtering. Innovations are also described in the areas of error handling and recovery, determination of neighbor availability for operations such as context modeling and intra prediction, CABAC decoding, computation of collocated information for direct mode macroblocks in B slices, reduction of memory consumption, implementation of trick play modes, and picture dropping for quality adjustment.

Claims

1. A method comprising:

parsing plural parameters for a first picture from an encoded video bit stream;

initializing the first picture by creating and filling one or more data structures for decoding the first picture;

repeating the parsing and the initializing for a second picture;

decoding the first picture, wherein the parsing and the initializing for the second picture finish before the decoding the first picture begins; and

decoding the second picture.

2. The method of claim 1 further comprising, before the decoding the first picture begins, repeating the parsing and the initializing for each of plural other pictures.

3. The method of claim 1 further comprising tracking dependencies, if any, between the first picture, the second picture, and one or more other pictures.

4. The method of claim 1 further comprising, before the decoding the first picture begins, entering the initialized first picture into a simulated decoded picture buffer.

5. The method of claim 1 further comprising, before the decoding the first picture begins, recording one or more picture commands associated with the first picture in a command queue.

6. The method of claim 5 wherein the one or more picture commands include an output command indicating a to-be-output output picture in a decoded picture buffer.

7. The method of claim 5 wherein the one or more picture commands include a delete command indicating a to-be-deleted picture in a decoded picture buffer.

8. The method of claim 1 wherein a picture extent discovery task performs the parsing and the initializing.

9. The method of claim 1 wherein the initializing includes populating at least part of the data structures with one or more of the plural parsed parameters for the first picture.

10. A method comprising:

putting one or more picture commands in a picture command queue, each of the one or more picture commands having an associated condition and one or more actions;

decoding one or more pictures; and

removing a first picture command of the one or more picture commands put in the picture command queue, wherein the removal of the first picture command is contingent on completion of the associated condition for the first picture command.

11. The method of claim 10 further comprising executing the one or more actions for the first picture command, wherein the associated condition for the first picture command is completion of decoding for a first picture of the one or more pictures.

12. The method of claim 10 wherein the picture command queue is a first-in-first-out queue.

13. The method of claim 10 wherein the one or more picture commands include an output command indicating a to-be-output output picture in a decoded picture buffer.

14. The method of claim 10 wherein the one or more picture commands include a delete command indicating a to-be-deleted picture in a decoded picture buffer.

15. The method of claim 10 wherein the one or more picture commands include a memory re-allocation command whose associated condition is processing of a sequence parameter set change command.

16. The method of claim 10 wherein the one or more picture commands include a memory re-allocation command whose associated condition is processing of a picture parameter set change command.

17. A method comprising:

generating a task dependency graph that indicates dependencies between plural decoding tasks, wherein the plural decoding tasks include at least one entropy decoding task, at least one intra prediction task, at least one motion-compensated prediction task, and at least one loop filtering task; and

decoding video with multiple threads of execution using the task dependency graph.

18. The method of claim 17 wherein each of the plural decoding tasks is represented as a node in the task dependency graph.

19. The method of claim 18 wherein, for the node representing a first task of the plural decoding tasks, a node structure includes a counter for tracking a count of dependencies to other tasks on which the first task is dependent.

20. The method of claim 18 wherein, for the node representing a first task of the plural decoding tasks, a node structure includes a list for tracking dependencies of other tasks dependent on the first task.