Efficient repeat padding for hybrid video sequence with arbitrary video resolution

• US 7,822,123 B2
• Filed: 10/06/2004
• Issued: 10/26/2010
• Est. Priority Date: 10/06/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hsu, Pohsiang, Lin, Chih-Lung, Srinivasan, Sridhar, Holcomb, Thomas W.
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): HOLDER, ANNER N
• Application Number:
  US10/960,384
• Publication Number:
  US 20060072669A1
• Time in Patent Office: 2,211 Days
• Field of Search: 375/240.24, 375/240.01, 375/240.29, 348/441, 348/445, 348/449, 348/446, 348/447
• US Class Current: 375/240.24
• CPC Class Codes: H04N 19/102 characterised by the element, parameter or selection affected or controlled by the adaptive coding
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/16 for a given display mode, e.g. for interlaced or progressive display mode
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/563 Motion estimation with padding, i.e. with filling of non-object values in an arbitrarily shaped picture block or region for estimation purposes
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

A video codec provides efficient repeat padding of hybrid video sequences having arbitrary video resolution. The video codec repeat pads to expand the active content of pictures in the video sequence out to meet an adaptive vertical macroblock alignment restriction that varies by picture type. For progressive type pictures, the video codec repeats the last row or horizontal boundary edge of the active content. For interlaced type pictures, the video coded repeats the last two rows (last row of each interlaced field) of the active content. This repeat padding differing by picture type provides a better prediction (lower prediction error residual) for macroblocks in following predicted frames whose motion vector points into the padded region.

Claims

1. A method of repeat padding for hybrid video sequences by a video codec device having a processor and a memory, comprising:

with the video codec device, encoding a hybrid video sequence having plural frames, the plural frames of the video sequence including both progressive frames and interlaced frames, wherein the encoding the hybrid video sequence includes;

setting a maximum coded width and maximum coded height for the video sequence;

for an entry point segment, determining whether to signal coded width and coded height for the entry point segment apart from signaling of the maximum coded width and the maximum coded height, respectively, for the video sequence and, if so, setting the coded width and the coded height for the entry point segment;

determining an active content frame size for a first frame, wherein the active content frame size is a resolution in terms of pixels, wherein the active content frame size of the first frame is indicated by the coded width and the coded height for the entry point segment if the coded width and the coded height for the entry point segment are set, and otherwise the active content frame size is indicated by the maximum coded width and the maximum coded height for the video sequence;

encoding and decoding the first frame, the first frame having a macroblock-aligned vertical size that depends at least in part on frame coding mode of the first frame, wherein the decoded first frame is used as a reference frame;

if the reference frame was coded as a frame of progressive video, expanding from the active content frame size of the reference frame using a first form of repeat padding, wherein the first form of repeat padding includes repeating a last row of active content of the reference frame, the active content being pixels in an area defined by the active content frame size;

if the reference frame was coded as separate fields of interlaced video or frame of interlaced video, expanding from the active content frame size of the reference frame using a second form of repeat padding, wherein the second form of repeat padding includes repeating a last row of each field of the active content of the reference frame; and

for a second frame of plural frames of the entry point segment, setting a frame coding mode that indicates whether the second frame is coded as a frame of progressive video, coded as separate fields of interlaced video or coded as a frame of interlaced video, and encoding the second frame using motion compensation relative to the reference frame; and

outputting, from the video codec device, encoded data for the video sequence in an output bitstream.

2. The method of claim 1, wherein the expanding using the first form of repeat padding comprises:

for each row in a padded region after the active content of the reference frame, repeating the last row of the active content.

3. The method of claim 2, wherein the expanding using the second form of repeat padding comprises:

for rows in a padded region after the active content of the reference frame, repeating the last two rows of the active content, the last two rows including the last row of each field of the active content.

4. The method of claim 1, wherein the expanding using the second form of repeat padding comprises:

for rows in a padded region after active content of the reference frame, repeating the last two rows of the active content, the last two rows including the last row of each field of the active content.

5. The method of claim 1, wherein the expanding using the first form of repeat padding comprises:

for each row after the active content of the reference frame up to a first macroblock alignment restriction which is a multiple of the macroblock height, repeating the last row of the active content.

6. The method of claim 5, wherein the macroblock height is 16 pixels.

7. The method of claim 5, wherein the expanding using the second form of repeat padding comprises:

for rows after the active content of the reference frame up to a second macroblock alignment restriction which is a multiple of twice the macroblock height, repeating the last two rows of the active content, the last two rows including the last row of each field of the active content.

8. The method of claim 7, wherein the macroblock height is 16 pixels.

11. The method of claim 1 wherein the encoded data for the video sequence comprises:

a sequence header including a first sequence-layer syntax element indicating the maximum coded width for the video sequence and a second sequence-layer syntax element indicating the maximum coded height for the video sequence;

an entry point header including a first entry point-layer syntax element indicating if the bitstream includes syntax elements for the coded width and the coded height for the entry point segment and, if so, a second entry point-layer syntax element indicating the coded width for the entry point segment and a third entry point-layer syntax element indicating the coded height for the entry point segment;

for the first frame, a first frame header including a first frame-layer syntax element indicating the frame coding mode for the first frame; and

for the second frame, a second frame header including a second frame-layer syntax element indicating the frame coding mode for the second frame.

12. The method of claim 1 wherein the active content frame size can align with macroblock boundaries or be non-aligned with macroblock boundaries.

13. The method of claim 1 wherein the encoding the hybrid video sequence further includes determining whether the first frame was coded as a frame of progressive video, separate fields of interlaced video, or a frame of interlaced video based at least in part upon the frame coding mode for the first frame.

9. A method of padding for hybrid video sequences in a video codec, comprising:

with the video codec, determining a picture type of a frame in an entry point segment of a hybrid video sequence comprising progressive frames and interlaced frames, wherein the determining the picture type is based at least in part upon a frame coding mode for the frame;

with the video codec, determining an active content frame size for the frame, wherein the active content frame size is a resolution in terms of pixels, wherein the active content frame size of the frame is indicated by a coded width and a coded height for the entry point segment if the coded width and the coded height for the entry point segment are set, and otherwise the active content frame size is indicated by a maximum coded width and a maximum coded height for the video sequence;

with the video codec, varying a selection of padding content with which the frame of the hybrid video sequence is padded based on the determined picture type of the frame, wherein a first form of padding is used if the frame is a progressive frame, and a second form of padding is used if the frame is an interlaced frame, and wherein the varying the padding content selection further comprises;

if the frame is a progressive frame, selecting a last row of active content in the frame as the padding content selection, the active content being pixels in an area defined by the active content frame size; and

if the frame is an interlaced frame, selecting a last row of each field of the active content in the frame as the padding content selection; and

with the video codec, repeatedly adding the padding content selection after the active content in the frame to expand the frame to a first padded frame size which is a next multiple of a macroblock height if the frame is a progressive frame, and to a second padded frame size which is a next multiple of twice the macroblock height if the frame is an interlaced frame.

10. The method of claim 9, wherein the macroblock height is 16 pixels.

14. A method of repeat padding for hybrid video sequences by a video decoder device having a processor and a memory, comprising:

receiving, at the video decoder device, encoded data in a bitstream for a hybrid video sequence having plural frames, the plural frames of the video sequence including both progressive frames and interlaced frames; and

with the video decoder device, decoding at least some of the encoded data, including;

determining a maximum coded width and maximum coded height for the video sequence;

for an entry point segment, determining whether to use the maximum coded width and the maximum coded height for the video sequence as coded width and coded height, respectively, for the entry point segment, and, if not, determining the coded width and the coded height for the entry point segment;

determining an active content frame size for a first frame, wherein the active content frame size is a resolution in terms of pixels, wherein the active content frame size of the first frame is indicated by the coded width and the coded height for the entry point segment if the coded width and the coded height for the entry point segment are used, and otherwise the active content frame size is indicated by the maximum coded width and the maximum coded height for the video sequence;

decoding the first frame, the first frame having a macroblock-aligned vertical size that depends at least in part on frame coding mode of the first frame, wherein the decoded first frame is used as a reference frame;

if the reference frame was coded as a frame of progressive video, expanding from the active content frame size of the reference frame using a first form of repeat padding, wherein the first form of repeat padding includes repeating a last row of active content of the reference frame, the active content being pixels in an area defined by the active content frame size;

if the reference frame was coded as separate fields of interlaced video or frame of interlaced video, expanding from the active content frame size of the reference frame using a second form of repeat padding, wherein the second form of repeat padding includes repeating a last row of each field of the active content of the reference frame; and

for a second frame of plural frames of the entry point segment, decoding the second frame using motion compensation relative to the reference frame.

15. The method of claim 14 wherein the encoded data comprises:

a sequence header including a first sequence-layer syntax element indicating the maximum coded width for the video sequence and a second sequence-layer syntax element indicating the maximum coded height for the video sequence;

an entry point header including a first entry point-layer syntax element indicating if the bitstream includes syntax elements for the coded width and the coded height for the entry point segment and, if so, a second entry point-layer syntax element indicating the coded width for the entry point segment and a third entry point-layer syntax element indicating the coded height for the entry point segment;

for the first frame, a first frame header including a first frame-layer syntax element indicating the frame coding mode for the first frame; and

for the second frame, a second frame header including a second frame-layer syntax element indicating frame coding mode for the second frame.

16. The method of claim 15 wherein the decoding at least some of the encoded data further includes determining whether the first frame was coded as a frame of progressive video, separate fields of interlaced video, or a frame of interlaced video based at least in part upon the frame coding mode for the first frame.