Low-delay buffering model in video coding

• US 9,374,585 B2
• Filed: 09/25/2013
• Issued: 06/21/2016
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Wang, Ye-Kui
• Primary Examiner(s): Elahi, Shan
• Application Number:
  US14/036,615
• Publication Number:
  US 20140169448A1
• Time in Patent Office: 1,000 Days
• Field of Search: 375/240.02
• US Class Current: 1/1
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/433 characterised by techniques for memory access
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

Techniques for low-delay buffering in a video coding process are disclosed. Video decoding techniques may include receiving a first decoded picture buffer (DPB) output delay and a second DPB output delay for a decoded picture, determining, for the decoded picture, a first DPB output time using the first DPB output delay in the case a hypothetical reference decoder (HRD) setting for a video decoder indicates operation at a picture level, and determining, for the decoded picture, a second DPB output time using the second DPB output delay in the case that the HRD setting for the video decoder indicates operation at a sub-picture level.

Claims

1. A method of decoding video, the method comprising:

receiving a first decoded picture buffer (DPB) output delay and a second DPB output delay;

determining whether a hypothetical reference decoder (HRD) operates at an access unit level or operates at a sub-picture level; and

based on a determination that the HRD operates at the access unit level, determining, by a video decoding device, for a decoded picture, a first DPB output time based on the first DPB output delay and a picture clock tick, or

based on a determination that the HRD operates at the sub-picture level;

deriving a sub-picture clock tick based on the picture clock tick and a tick divisor value; and

determining, by the video decoding device, for the decoded picture, a second DPB output time based on the second DPB output delay and the sub-picture clock tick.

2. The method of claim 1, further comprising:

processing a sub-picture coded picture buffer (CPB) flag that indicates whether the HRD operates at the access unit level or operates at the sub-picture level; and

outputting the decoded picture from a DPB based on the first DPB output time or the second DPB output time, wherein the decoded picture is output based on the first DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the access unit level, or the decoded picture is output based on the second DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the sub-picture level.

3. The method of claim 1, wherein determining the second DPB output time comprises multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a product and adding the product to a CPB removal time.

4. The method of claim 1, wherein determining the first DPB output time comprises multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a product and adding the product to a CPB removal time.

5. A method of encoding video, the method comprising:

signaling, in a picture timing supplemental enhancement information (SEI) message associated with an access unit, a first decoded picture buffer (DPB) output delay, the first DPB output delay being indicative of a number of picture clock ticks to wait after removal of a last decoding unit in the access unit from a coded picture buffer (CPB) before a decoded picture is output from a DPB;

signaling a presence of sub-picture level parameters in the bitstream;

signaling a tick divisor value; and

in response to signaling the presence of sub-picture level parameters, signaling, in the picture timing SEI message associated with the access unit a second DPB output delay, the second DPB output delay being indicative of a number of sub-picture clock ticks to wait after removal of the last decoding unit in the access unit from the CPB before the decoded picture is output from the DPB, sub-picture clock ticks being derived based on picture clock ticks.

6. The method of claim 5, further comprising:

determining a first DPB output time, wherein determining the first DPB output time comprises multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a product and adding the product to a CPB removal time; and

determining a second DPB output time, wherein determining the second DPB output time comprises multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a product and adding the product to a CPB removal time.

7. An apparatus configured to decode video data, the apparatus comprising:

a memory configured to store the video data; and

a video decoder in communication with the memory, the video decoder configured to;

receive a first decoded picture buffer (DPB) output delay and a second DPB output delay;

determine whether a hypothetical reference decoder (HRD) operates at an access unit level or operates at a sub-picture level; and

based on a determination that the HRD operates at the access unit level, determine, for a decoded picture, a first DPB output time based on the first DPB output delay and a picture clock tick, and

based on a determination that the HRD operates at the sub-picture level;

derive a sub-picture clock tick based on the picture clock tick and a tick divisor value; and

determine, for the decoded picture, a second DPB output time based on the second DPB output delay and the sub-picture clock tick.

8. The apparatus of claim 7, wherein the video decoder is further configured to:

process a sub-picture coded picture buffer (CPB) flag that indicates whether the HRD operates at the access unit level or operates at the sub-picture level; and

output the decoded picture from a DPB based on the first DPB output time or the second DPB output time, wherein the decoded picture is output based on the first DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the access unit level, or the decoded picture is output based on the second DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the sub-picture level.

9. The apparatus of claim 7, wherein the video decoder is configured to determine the second DPB output time by multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a product and adding the product to a CPB removal time.

10. The apparatus of claim 7, wherein the video decoder is configured to determine the first DPB output time by multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a product and adding the product to a CPB removal time.

11. An apparatus configured to encode video data, the apparatus comprising:

a memory configured to store the video data; and

a video encoder in communication with the memory, the video encoder configured to;

signal, in a picture timing supplemental enhancement information (SEI) message associated with an access unit, a first decoded picture buffer (DPB) output time delay, the first DPB output delay being indicative of a number of picture clock ticks to wait after removal of a last decoding unit in the access unit from a coded picture buffer (CPB) before a decoded picture is output from a DPB;

signal a presence of sub-picture level parameters in the bitstream;

signal a tick divisor value; and

in response to signaling the presence of sub-picture level parameters, signal, in the picture timing SEI message associated with the access unit, a second DPB output delay, the second DPB output delay being indicative of a number of sub-picture clock ticks to wait after removal of the last decoding unit in the access unit from the CPB before the decoded picture is output from the DPB, sub-picture clock ticks being derived based on picture clock ticks.

12. The apparatus of claim 11, wherein the video encoder is further configured to:

determine the first DPB output time by multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a first DPB output product and adding the first DPB output product to a CPB removal time; and

determine the second DPB output time by multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a second DPB output product and adding the second DPB output product to a CPB removal time.

13. An apparatus configured to decode video data, the apparatus comprising:

means for receiving a first decoded picture buffer (DPB) output delay and a second DPB output delay;

indication means for determining whether a hypothetical reference decoder (HRD) operates at an access unit level or operates at a sub-picture level;

means, operable when the indication means determines operation of the HRD at the access unit level, for determining, for a decoded picture, a first DPB output time based on the first DPB output delay and a picture clock tick;

means for deriving a sub-picture clock tick based on the picture clock tick and a tick divisor value; and

means, operable when the indication means determines operation of the HRD at the sub-picture level, for determining, for the decoded picture, a second DPB output time based on the second DPB output delay and the sub-picture clock tick.

14. The apparatus of claim 13, further comprising:

means for processing a sub-picture coded picture buffer (CPB) flag that indicates whether the HRD operates at the access unit level or operates at the sub-picture level; and

means for outputting the decoded picture from a DPB based on the first DPB output time or the second DPB output time, wherein the decoded picture is output based on the first DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the access unit level, or the decoded picture is output based on the second DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the sub-picture level.

15. The apparatus of claim 13, wherein the means for determining the second DPB output time comprises means for multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a product and adding the product to a CPB removal time.

16. The apparatus of claim 13, wherein the means for determining the first DPB output time comprises means for multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a product and adding the product to a CPB removal time.

17. An apparatus configured to encode video data, the apparatus comprising:

means, for signaling, in a picture timing supplemental enhancement information (SEI) message associated with an access unit, a first decoded picture buffer (DPB) output delay, the first DPB output delay being indicative of a number of picture clock ticks to wait after removal of a last decoding unit in the access unit from a coded picture buffer (CPB) before a decoded picture is output from a DPB;

means for signaling a presence of sub-picture level parameters in the bitstream;

means for signaling a tick divisor value;

means, operable in response to signaling the presence of sub-picture level parameters, for signaling, in the picture timing SEI message associated with the access unit, a second DPB output delay, the second DPB output delay being indicative of a number of sub-picture clock ticks to wait after removal of the last decoding unit in the access unit from the CPB before the decoded picture is output from the DPB, sub-picture clock ticks being derived based on picture clock ticks.

18. The apparatus of claim 17,

wherein the means for determining the first DPB output time comprises means for multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a first DPB output product and adding the first DPB output product to a CPB removal time;

and

wherein the means for determining the second DPB output time comprises means for multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a second DPB output product and adding the second DPB output product to a CPB removal time.

19. A non-transitory computer-readable storage medium storing instructions that, when executed, causes one or more processors of a device configured to decode video data to:

receive a first decoded picture buffer (DPB) output delay and a second DPB output delay;

determine whether a hypothetical reference decoder (HRD) operates at an access unit level or operates at a sub-picture level; and

based on a determination that the HRD operates at the access unit level, determine, for a decoded picture, a first DPB output time based on the first DPB output delay and a picture clock tick; and

based on a determination that the HRD operates at the sub-picture level;

derive a sub-picture clock tick based on the picture clock tick and a tick divisor value; and

determine, for the decoded picture, a second DPB output time based on the second DPB output delay and the sub-picture clock tick.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions further cause the one or more processors to:

process a sub-picture coded picture buffer (CPB) flag that indicates whether the HRD operates at the access unit level or operates at the sub-picture level; and

output the decoded picture from a DPB based on the first DPB output time or the second DPB output time, wherein the decoded picture is output based on the first DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the access unit level, or the decoded picture is output based on the second DPB output time based on the sub-picture CPB flag indicating that the HRD operates at the sub-picture level.

21. The non-transitory computer-readable storage medium of claim 19, wherein the instructions further cause the one or more processors to multiply the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a product and add the product to a CPB removal time.

22. The non-transitory computer-readable storage medium of claim 19, wherein the instructions further cause the one or more processors to multiply the first DPB output delay by a value corresponding to the picture clock tick to obtain a product and add the product to a CPB removal time.

23. A non-transitory computer-readable storage medium storing instructions that, when executed, causes one or more processors of a device configured to encode video data to:

signal, in a picture timing supplemental enhancement information (SEI) message associated with an access unit, a first decoded picture buffer (DPB) output time delay, the first DPB output delay being indicative of a number of picture clock ticks to wait after removal of a last decoding unit in the access unit from a coded picture buffer (CPB) before a decoded picture is output from a DPB;

signal a presence of sub-picture level parameters in the bitstream;

signal a tick divisor value; and

in response to signaling the presence of sub-picture level parameters, signal, in the picture timing SEI message associated with the access unit, a second DPB output delay, the second DPB output delay being indicative of a number of sub-picture clock ticks to wait after removal of the last decoding unit in the access unit from the CPB before the decoded picture is output from the DPB, sub-picture clock ticks being derived based on picture clock ticks.

24. The non-transitory computer-readable storage medium of claim 23,

wherein the instructions further include instructions that, when executed, cause the one or more processors to determine the first DPB output time by multiplying the first DPB output delay by a value corresponding to the picture clock tick to obtain a first DPB output product and adding the first DPB output product to a CPB removal time;

and

wherein the instructions further include instructions that, when executed, cause the one or more processors to determine the second DPB output time by multiplying the second DPB output delay by a value corresponding to the sub-picture clock tick to obtain a second DPB output product and adding the second DPB output product to a CPB removal time.