Variable coding resolution in video codec

• US 9,479,796 B2
• Filed: 05/26/2015
• Issued: 10/25/2016
• Est. Priority Date: 10/06/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Holcomb, Thomas W., Lin, Chih-Lung, Srinivasan, Sridhar, Hsu, Pohsiang
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US14/721,629
• Publication Number:
  US 20150334416A1
• Time in Patent Office: 518 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/156 Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

A video codec provides for encoding and decoding pictures of a video sequence at various coded resolutions, such that pictures can be encoded at lower coded resolutions based on bit rate or other constraints while maintaining a consistent display resolution. The video codec employs a coding syntax where a maximum coded resolution is signaled at the sequence level of the syntax hierarchy, whereas a lower coded resolution is signaled at the entry point level for a segment of one or more intra-coded frames and frames predictively encoded based thereon. This allows the use of a separate out-of-loop resampler after the decoder to up-sample the pictures to the display resolution.

Claims

1. A computer system comprising a processor and memory, wherein the computer system implements a video capture system comprising:

a video encoder configured to;

determine a maximum coded resolution for a video sequence, the video sequence having a plurality of entry point segments for separate sub-sequences of pictures of the video sequence, each of the entry point segments being marked by a separate entry point and having at least one picture;

signal, using syntax elements in a sequence header, the maximum coded resolution for the video sequence;

signal, using a flag in an entry point header for a given one of the entry point segments in the video sequence, whether the at least one picture of the given entry point segment is encoded at a lower coded resolution;

signal, using syntax elements in the entry point header for the given entry point segment, the lower coded resolution for the at least one picture of the given entry point segment when indicated for the given entry point segment by the flag; and

encode the at least one picture of the given entry point segment, including down-sampling the at least one picture of the given entry point segment from the maximum coded resolution for the video sequence to the lower coded resolution when indicated for the given entry point segment by the flag; and

a multiplexor configured to produce a video bit stream, wherein the sequence header is integral to part of the video bit stream.

2. The computer system of claim 1, wherein the syntax elements used to signal the maximum coded resolution are width and height values.

3. The computer system of claim 1, wherein the syntax elements used to signal the lower coded resolution are width and height values.

4. The computer system of claim 1, further comprising:

a buffer configured to buffer the at least one picture of the given entry point segment at the lower coded resolution for encoding any motion-compensated pictures without resampling of the buffered at least one picture.

5. The computer system of claim 1, wherein the syntax elements in the sequence header include first and second sequence-layer syntax elements indicating maximum coded width and maximum coded height, respectively, for the video sequence, and wherein the syntax elements in the entry point header for the given entry point segment include first and second entry point-layer syntax elements indicating coded width and coded height, respectively, for the at least one picture of the given entry point segment.

6. At least one computer-readable medium having computer-executable instructions stored thereon for causing a processing unit, when programmed thereby, to perform video encoding operations using variable coded resolution, the at least one computer-readable medium being selected from the group consisting of non-volatile memory and magnetic disk, the video encoding operations comprising:

determining a maximum coded resolution for a video sequence, the video sequence having a plurality of entry point segments for separate sub-sequences of pictures of the video sequence, each of the entry point segments being marked by a separate entry point and having at least one picture;

signalling, using at least one syntax element in a sequence header, the maximum coded resolution for the video sequence;

for a given one of the entry point segments, determining whether to use the maximum coded resolution for the video sequence as coded resolution for the at least one picture of the given entry point segment;

signalling, using a flag in an entry point header for the given entry point segment, whether the at least one picture of the given entry point segment is encoded at the maximum coded resolution for the video sequence or a lower coded resolution;

if the at least one picture of the given entry point segment is encoded at the lower coded resolution, signalling, using at least one syntax element in the entry point header for the given entry point segment, the lower coded resolution for the at least one picture of the given entry point segment;

encoding the at least one picture of the given entry point segment, including, if the at least one picture of the given entry point segment is encoded at the lower coded resolution, down-sampling the at least one picture of the given entry point segment from the maximum coded resolution for the video sequence to the lower coded resolution; and

outputting a video bit stream, wherein the sequence header is part of the video bit stream.

7. The at least one computer-readable medium of claim 6, wherein the at least one syntax element used to signal the maximum coded resolution are maximum coded width and height values.

8. The at least one computer-readable medium of claim 7, wherein the maximum coded width and height values are binary coded values relating to width and height of the maximum coded resolution, respectively.

9. The at least one computer-readable medium of claim 6, wherein the determining whether to use the maximum coded resolution is based on one or more of:

content of the at least one picture of the given entry point segment;

available bit rate of a communication channel; and

computational resources available in a video encoder for encoding the at least one picture of the given entry point segment.

10. The at least one computer-readable medium of claim 6, wherein the at least one syntax element used to signal the lower coded resolution are coded width and height values.

11. The at least one computer-readable medium of claim 10, wherein the coded width and height values are binary coded values relating to width and height of the lower coded resolution, respectively.

12. The at least one computer-readable medium of claim 6, wherein the given entry point segment includes an I picture at the lower coded resolution and one or more P pictures encoded using motion compensation at the lower coded resolution, and wherein the encoding further comprises:

buffering the pictures of the given entry point segment at the lower coded resolution.

13. The at least one computer-readable medium of claim 6, wherein:

the at least one syntax element in the sequence header include first and second sequence-layer syntax elements indicating maximum coded width and maximum coded height, respectively, for the video sequence; and

the at least one syntax element in the entry point header include first and second entry point-layer syntax elements indicating coded width and coded height, respectively, for the at least one picture of the given entry point segment.

14. A method of encoding video using variable coded resolution, comprising, with a computer system that implements a video encoder:

determining a maximum coded resolution for a video sequence, the video sequence having a plurality of entry point segments for separate sub-sequences of pictures of the video sequence, each of the entry point segments being marked by a separate entry point and having at least one picture;

signalling, using at least one syntax element in a sequence header, the maximum coded resolution for the video sequence;

for a given one of the entry point segments, determining whether to use the maximum coded resolution for the video sequence as coded resolution for the at least one picture of the given entry point segment;

signalling, using a flag in an entry point header for the given entry point segment, whether the at least one picture of the given entry point segment is encoded at the maximum coded resolution for the video sequence or a lower coded resolution;

if the at least one picture of the given entry point segment is encoded at the lower coded resolution, signalling, using at least one syntax element in the entry point header for the given entry point segment, the lower coded resolution for the at least one picture of the given entry point segment;

encoding the at least one picture of the given entry point segment; and

outputting a video bit stream, wherein the sequence header is part of the video bit stream.

15. The method of claim 14, wherein the at least one syntax element used to signal the maximum coded resolution are maximum coded width and height values.

16. The method of claim 14, wherein the determining whether to use the maximum coded resolution is based on one or more of:

content of the at least one picture of the given entry point segment;

available bit rate of a communication channel; and

computational resources available in the video encoder for encoding the at least one picture of the given entry point segment.

17. The method of claim 14, wherein the at least one syntax element used to signal the lower coded resolution are coded width and height values.

18. The method of claim 14, wherein the encoding the at least one picture of the given entry point segment includes, if the at least one picture of the given entry point segment is encoded at the lower coded resolution, down-sampling the at least one picture of the given entry point segment from the maximum coded resolution for the video sequence to the lower coded resolution.

19. The method of claim 14, wherein the given entry point segment includes an I picture at the lower coded resolution and one or more P pictures encoded using motion compensation at the lower coded resolution, and wherein the encoding further comprises:

buffering the pictures of the given entry point segment at the lower coded resolution.

20. The method of claim 14, wherein:

the at least one syntax element in the sequence header include first and second sequence-layer syntax elements indicating maximum coded width and maximum coded height, respectively, for the video sequence; and

the at least one syntax element in the entry point header include first and second entry point-layer syntax elements indicating coded width and coded height, respectively, for the at least one picture of the given entry point segment.