Decoding variable coded resolution video with native range/resolution post-processing operation

• US 8,243,820 B2
• Filed: 04/29/2005
• Issued: 08/14/2012
• Est. Priority Date: 10/06/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Holcomb, Thomas W., Lin, Chih-Lung, Hsu, Pohsiang, Srinivasan, Sridhar
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): TORRENTE, RICHARD T
• Application Number:
  US11/118,686
• Publication Number:
  US 20060072673A1
• Time in Patent Office: 2,664 Days
• Field of Search: 375/240.25
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence

Abstract

A video codec provides for encoding and decoding pictures of a video sequence at various coded resolutions, such that pictures can be encoded at lower coded resolutions based on bit rate or other constraints while maintaining a consistent display resolution. The video codec further provide for encoding and decoding pictures of the video sequence at ranges lower than that used for display, and then expanding the range after decoding for display. The video codec applies post-processing operations, such as de-blocking, de-ringing, and color conversion, at the native resolution and range of the decoded video, prior to range expansion and upsampling for display.

Claims

1. A method of decoding video using a computing device that implements a video decoder, comprising:

with the computing device that implements the video decoder, parsing plural syntax elements from a sequence header for a video bit stream, where the sequence header is integral to the video bit stream, the video bit stream having a plurality of entry point segments of a video sequence, at least one of said entry point segments comprising coded content of a group of pictures separate from that of another of said entry point segments of the video sequence, and wherein the plural syntax elements parsed from the sequence header include plural coded resolution syntax elements that indicate a maximum coded width and a maximum coded height for the video sequence;

with the computing device that implements that video decoder, decoding one of the entry point segments of the video sequence, wherein values of the pictures in the entry point segment have a first range, and wherein the decoding includes;

for the entry point segment, determining whether to use the maximum coded width and the maximum coded height for the video sequence as coded width and coded height, respectively, for the pictures of the entry point segment, and, if not, determining the coded width and the coded height for the pictures of the entry point segment from plural coded resolution syntax elements of an entry point header for the entry point segment; and

decoding the pictures of the entry point segment;

with the computing device that implements that video decoder, applying one or more post-processing operations to the respective pictures of the decoded entry point segment, including performing range remapping to remap the values of the respective pictures from the first range to a second range, wherein the one or more post-processing operations further comprise a color conversion operation; and

with the computing device that implements that video decoder, reconstructing at least part of the video sequence with the decoded entry point segment.

2. The method of claim 1 further comprising:

applying said one or more post-processing operations to the respective pictures of the decoded entry point segment at a native resolution of the decoded entry point segment; and

up-sampling the respective pictures of the decoded entry point segment to the maximum coded width and maximum coded height for display of the video.

3. The method of claim 1 further comprising:

for the respective pictures of the entry point segment, decoding a post-processing operation-signaling syntax element signaling whether one or more additional post-processing operations are to be applied to the respective pictures of the decoded entry point segment; and

applying the additional post-processing operations as signaled by the post-processing operation-signaling syntax element.

4. The method of claim 3 wherein the post-processing operation-signaling syntax element is encoded at least in part at a frame level of the video bit stream.

5. The method of claim 3 wherein the post-processing operation-signaling syntax element signals to apply de-blocking and de-ringing.

6. The method of claim 3 wherein the post-processing operation-signaling syntax element signals to optionally apply de-blocking, de-ringing, or a combination of de-blocking and de-ringing.

7. The method of claim 6 comprising in the case that the post-processing operation-signaling syntax element signals to apply de-blocking and de-ringing in combination, applying de-blocking before de-ringing.

8. The method of claim 1 wherein the one or more post-processing operations further comprise one or more de-blocking filter stages.

9. The method of claim 1 wherein the one or more post-processing operations further comprise one or more de-ringing filter stages.

10. The method of claim 1 wherein the one or more post-processing operations further comprise an artifact removal operation.

11. The method of claim 1 wherein the plural coded resolution syntax elements of the sequence header include a first sequence-layer syntax element indicating the maximum coded width for the video sequence and a second sequence-layer syntax element indicating the maximum coded height for the video sequence, and wherein the entry point header includes:

a first entry point-layer syntax element indicating if the video bitstream includes syntax elements for the coded width and the coded height for the pictures of the entry point segment; and

if so, a second entry point-layer syntax element indicating the coded width for the pictures of the entry point segment and a third entry point-layer syntax element indicating the coded height for the pictures of the entry point segment.

12. A video decoder, comprising:

means for reading elements specifying a maximum coded resolution for a video sequence from a sequence header of the video sequence in a compressed video bit stream, the video sequence having a plurality of entry point segments for separate sub-sequences of pictures of the video sequence, each of the entry point segments being marked by a separate entry point and having coded content of a group of at least one picture, at least two of the entry point segments having coded content of separate groups of at least one picture;

means for reading a flag indicative of whether all of one of the entry point segments in the video sequence is encoded at a lower coded resolution, the flag being signaled in an entry point header for the entry point segment;

means for reading elements specifying the lower coded resolution from the entry point header for the entry point segment when indicated for the entry point segment by the flag;

means for decoding the at least one picture of the entry point segment;

means for reading elements in the compressed video bit stream specifying whether post-processing is to be performed for the at least one picture of the entry point segment, wherein the elements specifying whether post-processing is to be performed comprise bits encoding whether to perform de blocking, de ringing, de blocking and de-ringing, or neither;

means for optionally applying a post-processing operation to the at least one decoded picture while at the lower coded resolution, wherein the means for optionally applying the post-processing operation applies the post-processing operation in accordance with the elements specifying whether post-processing is to be performed; and

means for up-sampling the at least one picture of the entry point segment from the lower coded resolution to the maximum coded resolution of the video sequence after said optionally applying the post-processing operation.

13. The video decoder of claim 12 wherein the post-processing operation further comprises any one or more of range remapping and color conversion.

14. A method of decoding video using a computing device that implements a video decoder, comprising:

with the computing device that implements the video decoder, parsing plural syntax elements from a sequence header for a video bit stream, where the sequence header is integral to the video bit stream, the video bit stream having a plurality of entry point segments of a video sequence, at least one of said entry point segments comprising coded content of a group of pictures separate from that of another of said entry point segments of the video sequence, and wherein the plural syntax elements parsed from the sequence header include plural coded resolution syntax elements that indicate a maximum coded width and a maximum coded height for the video sequence;

with the computing device that implements that video decoder, decoding one of the entry point segments of the video sequence, wherein values of the pictures in the entry point segment have a first range, and wherein the decoding includes;

for the entry point segment, determining whether to use the maximum coded width and the maximum coded height for the video sequence as coded width and coded height, respectively, for the pictures of the entry point segment, and, if not, determining the coded width and the coded height for the pictures of the entry point segment from plural coded resolution syntax elements of an entry point header for the entry point segment; and

decoding the pictures of the entry point segment, including for each of the pictures of the entry point segment, decoding a post-processing operation-signaling syntax element signaling whether one or more post-processing operations are to be applied to the respective pictures of the decoded entry point segment, wherein the post-processing operation-signaling syntax element signals to optionally apply de-blocking, de-ringing, or a combination of de-blocking and de-ringing; and

with the computing device that implements that video decoder, applying one or more post-processing operations to the respective pictures of the decoded entry point segment, including performing range remapping to remap the values of the respective pictures from the first range to a second range, and further including for each of the respective pictures, applying the post-processing operations as signaled by the post-processing operation-signaling syntax element for that picture; and

with the computing device that implements that video decoder, reconstructing at least part of the video sequence with the decoded entry point segment.

15. The method of claim 14 further comprising:

applying said one or more post-processing operations to the respective pictures of the decoded entry point segment at a native resolution of the decoded entry point segment; and

up-sampling the respective pictures of the decoded entry point segment to the maximum coded width and maximum coded height for display of the video.

16. The method of claim 14 wherein the post-processing operation-signaling syntax element is encoded at least in part at a frame level of the video bit stream.

17. The method of claim 14 comprising in the case that the post-processing operation-signaling syntax element signals to apply de-blocking and de-ringing in combination, applying de-blocking before de-ringing.

18. A method of decoding video using a computing device that implements a video decoder, comprising:

with the computing device that implements the video decoder, parsing plural syntax elements from a sequence header for a video bit stream, where the sequence header is integral to the video bit stream, the video bit stream having a plurality of entry point segments of a video sequence, at least one of said entry point segments comprising coded content of a group of pictures separate from that of another of said entry point segments of the video sequence, and wherein the plural syntax elements parsed from the sequence header include plural coded resolution syntax elements that indicate a maximum coded width and a maximum coded height for the video sequence;

with the computing device that implements that video decoder, decoding one of the entry point segments of the video sequence, wherein values of the pictures in the entry point segment have a first range, and wherein the decoding includes;

for the entry point segment, determining whether to use the maximum coded width and the maximum coded height for the video sequence as coded width and coded height, respectively, for the pictures of the entry point segment, and, if not, determining the coded width and the coded height for the pictures of the entry point segment from plural coded resolution syntax elements of an entry point header for the entry point segment; and

decoding the pictures of the entry point segment;

with the computing device that implements that video decoder, applying one or more post-processing operations to the respective pictures of the decoded entry point segment, including performing range remapping to remap the values of the respective pictures from the first range to a second range, wherein the one or more post-processing operations further comprise one or more de-ringing filter stages; and

with the computing device that implements that video decoder, reconstructing at least part of the video sequence with the decoded entry point segment.

19. The method of claim 18 further comprising:

applying said one or more post-processing operations to the respective pictures of the decoded entry point segment at a native resolution of the decoded entry point segment; and

up-sampling the respective pictures of the decoded entry point segment to the maximum coded width and maximum coded height for display of the video.

20. The method of claim 18 wherein the one or more post-processing operations further comprise one or more de-blocking filter stages.