Efficient decoded picture buffer management for scalable video coding

• US 20070086521A1
• Filed: 10/11/2006
• Published: 04/19/2007
• Est. Priority Date: 10/11/2005
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Wang, Ye-Kui, Hannuksela, Miska, Wenger, Stephan
• Application Number:
  US11/546,622
• Publication Number:
  US 20070086521A1
• Time in Patent Office: Days
• US Class Current: 375/240.100
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/34 Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/44004 involving video buffer management, e.g. video decoder buffer or video display buffer

Abstract

A system and method for enabling the removal of decoded pictures from a decoded picture buffer as soon as the decoded pictures are no longer needed for prediction reference and future output. An indication is introduced into the bitstream as to whether a picture may be used for inter-layer prediction reference, as well as a decoded picture buffer management method which uses the indication. The present invention includes a process for marking a picture as being used for inter-layer reference or unused for inter-layer reference, a storage process of decoded pictures into the decoded picture buffer, a marking process of reference pictures, and output and removal processes of decoded pictures from the decoded picture buffer.

Claims

1. A method of managing a decoded picture buffer for scalable video coding, comprising:

receiving a first decoded picture belonging to a first layer in a bitstream into the decoded picture buffer;

receiving a second decoded picture belonging to a second layer;

determining whether the first decoded picture is required for inter-layer prediction reference in light of the receipt of the second decoded picture; and

if the first decoded picture is no longer required for inter-layer prediction reference, inter prediction reference and future output, removing the first decoded picture from the decoded picture buffer.

2. The method of claim 1, further comprising carrying information related to an indication of possible inter-layer prediction reference of a subsequent picture in decoding order signaled in the bitstream.

3. The method of claim 2, wherein the indication of possible inter-layer prediction reference is signaled in the slice header.

4. The method of claim 2, wherein the indication of possible inter-layer prediction reference is signaled in the Network Abstraction Layer (NAL) unit header.

5. The method of claim 2, wherein the determining of whether the first decoded picture is required for inter-layer prediction reference includes selectively marking the first decoded picture as “

unused for inter-layer reference.”

6. The method of claim 5, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture belongs to the same access unit as the second picture.

7. The method of claim 6, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

8. The method of claim 5, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has the indication of possible inter layer prediction reference being positive and is marked as “

used for inter-layer reference”

.

9. The method of claim 8, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

10. The method of claim 5, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has a smaller value of dependency_id than the second picture or identical value of dependency_id but a smaller value of quality_level than the second picture.

11. The method of claim 10, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

12. The method of claim 2, wherein the first decoded picture is determined to be no longer required for inter-layer prediction reference if the first picture is marked as “

unused for reference”

or a non-reference picture;

if the first picture is marked as “

unused for inter-layer reference”

or has the indication of possible inter layer prediction reference being negative; and

if the first picture is either marked as “

non-existing”

is not in the desired scalable layer or has a decoded picture buffer output time less than or equal to a coded picture buffer removal time of the second picture.

13. The method of claim 12, wherein, if the first decoded picture is a reference frame, the first decoded picture is considered to be marked as “

unused for reference”

only when both of the first decoded picture'"'"'s fields have been marked as “

unused for reference.”

14. The method of claim 1, wherein the first decoded picture is not needed for future output if the first decoded picture is not in the desired scalable layer for playback.

15. The method of claim 1, wherein the bitstream comprises a first sub-bitstream and a second sub-bitstream, the first sub-bitstream comprising coded pictures belonging to the first layer and the second sub bit stream comprising pictures of said second layer.

16. A decoder for decoding an encoded stream of a plurality of pictures, the plurality of pictures being defined as reference pictures or non-reference pictures, and information relating to decoding order and output order of a picture is defined for pictures of the picture stream, the decoder configured to perform the method of claim 1.

30. The computer program product of claim 16, wherein the first decoded picture is not needed for future output if the first decoded picture is not in the desired scalable layer for playback.

31. The computer program product of claim 16, wherein the bitstream comprises a first sub-bitstream and a second sub-bitstream, the first sub-bitstream comprising coded pictures belonging to the first layer and the second sub bit stream comprising pictures of said second layer.

51. A bit stream comprising

a syntax element providing an indication to selectively remove a first decoded picture of a first layer from the decoded picture buffer in light of a second decoded picture of a second layer, wherein the syntax element is set according to the method of claim 1.

17. A computer program product for managing a decoded picture buffer for scalable video coding, comprising:

computer code for receiving a first decoded picture belonging to a first layer in a bitstream into the decoded picture buffer;

computer code for receiving a second decoded picture belonging to a second layer;

computer code for determining whether the first decoded picture is required for inter-layer prediction reference in light of the receipt of the second decoded picture; and

computer code for, if the first decoded picture is no longer required for inter-layer prediction reference, inter prediction reference and future output, removing the first decoded picture from the decoded picture buffer.

18. The computer program product of claim 17, further comprising computer code for carrying information related to an indication of possible inter-layer prediction reference of a subsequent picture in decoding order signaled in the bitstream.

19. The computer program product of claim 18, wherein the indication of possible inter-layer prediction reference is signaled in the slice header.

20. The computer program product of claim 18, wherein the indication of possible inter-layer prediction reference is signaled in the Network Abstraction Layer (NAL) unit header.

21. The computer program product of claim 18, wherein the determining of whether the first decoded picture is required for inter-layer prediction reference includes selectively marking the first decoded picture as “

unused for inter-layer reference.”

22. The computer program product of claim 21, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture belongs to the same access unit as the second picture.

23. The computer program product of claim 22, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

24. The computer program product of claim 21, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has the indication of possible inter layer prediction reference being positive and is marked as “

used for inter-layer reference”

.

25. The computer program product of claim 24, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

26. The computer program product of claim 21, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has a smaller value of dependency_id than the second picture or identical value of dependency_id but a smaller value of quality_level than the second picture.

27. The computer program product of claim 26, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

28. The computer program product of claim 17, wherein the first decoded picture is determined to be no longer required for inter-layer prediction reference if the first picture is marked as “

unused for reference”

or a non-reference picture;

if the first picture is marked as “

unused for inter-layer reference”

or has the indication of possible inter layer prediction reference being negative; and

if the first picture is either marked as “

non-existing”

is not in the desired scalable layer or has a decoded picture buffer output time less than or equal to a coded picture buffer removal time of the second picture.

29. The computer program product of claim 28, wherein, if the first decoded picture is a reference frame, the first decoded picture is considered to be marked as “

unused for reference”

only when both of the first decoded picture'"'"'s fields have been marked as “

unused for reference.”

32. An electronic device, comprising:

a processor; and

a memory unit operatively connected to the processor and including a computer program product for managing a decoded picture buffer for scalable video coding, comprising;

computer code for receiving a first decoded picture belonging to a first layer in a bitstream into the decoded picture buffer;

computer code for receiving a second decoded picture belonging to a second layer;

computer code for determining whether the first decoded picture is required for inter-layer prediction reference in light of the receipt of the second decoded picture; and

computer code for, if the first decoded picture is no longer required for inter-layer prediction reference, inter prediction reference and future output, removing the first decoded picture from the decoded picture buffer.

33. The electronic device of claim 32, wherein the memory unit further comprises computer code for carrying information related to an indication of possible inter-layer prediction reference of a subsequent picture in decoding order signaled in the bitstream.

34. The electronic device of claim 33, wherein the indication of possible inter-layer prediction reference is signaled in the slice header.

35. The electronic device of claim 33, wherein the indication of possible inter-layer prediction reference is signaled in the Network Abstraction Layer (NAL) unit header.

36. The electronic device of claim 33, wherein the determining of whether the first decoded picture is required for inter-layer prediction reference includes selectively marking the first decoded picture as “

unused for inter-layer reference.”

37. The electronic device of claim 36, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture belongs to the same access unit as the second picture.

38. The electronic device of claim 37, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

39. The electronic device of claim 36, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has the indication of possible inter layer prediction reference being positive and is marked as “

used for inter-layer reference”

.

40. The electronic device of claim 39, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

41. The electronic device of claim 36, wherein the first decoded picture is marked as “

unused for inter-layer reference”

if the first picture has a smaller value of dependency_id than the second picture or identical value of dependency_id but a smaller value of quality_level than the second picture.

42. The electronic device of claim 41, wherein the determining of whether the first decoded picture is marked as “

unused for inter-layer reference”

according to a signaling in the bitstream.

43. The electronic device of claim 36, wherein the first decoded picture is determined to be no longer required for inter-layer prediction reference if the first picture is marked as “

unused for reference”

or a non-reference picture;

if the first picture is marked as “

unused for inter-layer reference”

or has the indication of possible inter layer prediction reference being negative; and

if the first picture is either marked as “

non-existing”

is not in the desired scalable layer or has a decoded picture buffer output time less than or equal to a coded picture buffer removal time of the second picture.

44. The electronic device of claim 43, wherein, if the first decoded picture is a reference frame, the first decoded picture is considered to be marked as “

unused for reference”

only when both of the first decoded picture'"'"'s fields have been marked as “

unused for reference.”

45. The electronic device of claim 32, wherein the first decoded picture is not needed for future output if the first decoded picture is not in the desired scalable layer for playback.

46. The electronic device of claim 32, wherein the bitstream comprises a first sub-bitstream and a second sub-bitstream, the first sub-bitstream comprising coded pictures belonging to the first layer and the second sub bit stream comprising pictures of said second layer.

47. The electronic device of claim 32, wherein the electronic device comprises a decoder configured to read syntax elements for the indication of possible reference and memory management control operations from the bit stream.

48. An encoder for forming an encoded stream of pictures, the pictures being defined as reference pictures or non-reference pictures, and information relating to decoding order and output order of a picture is defined for pictures in the stream, wherein the encoder places syntax elements for the indication of possible reference and memory management control operation into the stream, the syntax elements being generated by the electronic device of claim 32.

50. A computer device implementing an encoder that generates a bitstream according to claim 48.

49. A bit stream comprising

a syntax element providing an indication to selectively removing a first decoded picture of a first layer from the decoded picture buffer in light of a second decoded picture of a second layer.

52. A method of managing a decoded picture buffer for scalable video coding, comprising:

receiving a first decoded picture belonging to a first layer in a bitstream into the decoded picture buffer;

receiving a second decoded picture belonging to a second layer;

determining whether the first decoded picture is required for inter-layer prediction reference, inter prediction reference and future output, in light of the receipt of the second decoded picture; and

if the first decoded picture is no longer required for inter-layer prediction reference, inter prediction reference and future output, removing the first decoded picture from the decoded picture buffer.