Grouping of image frames in video coding

• US 7,894,521 B2
• Filed: 11/29/2002
• Issued: 02/22/2011
• Est. Priority Date: 01/23/2002
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Hannuksela, Miska
• Primary Examiner(s): DIEP, NHON THANH
• Application Number:
  US10/306,942
• Publication Number:
  US 20030138043A1
• Time in Patent Office: 3,007 Days
• Field of Search: None
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/65 using error resilience
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 19/895 in combination with error concealment

Abstract

A method for coding video frames for forming a scalable, compressed video sequence comprising video frames coded according to at least a first and a second frame format. The video frames of the first frame format are independent video frames, and the video frames of the second frame format are predicted from at least one of the other video frames. The video sequence has a first sub-sequence determined therein, at least part of the first sub-sequence being formed by coding at least video frames of the first frame format; and with at least a second sub-sequence, at least part of which is formed by coding video frames of the second frame format, and at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence. Frame identifier data of the second sub-sequence is determined into the video sequence.

Claims

1. A method for coding video frames for the purpose of forming a scalable, compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of other video frames, and the video frames of the second frame format being predicted from at least one other video frame, the method comprising

encoding, at an encoder, the video sequence as at least part of a first sub-sequence, at least part of which has been formed by coding video frames of the at least first frame format;

and

encoding, at the encoder, the video sequence as at least a second sub-sequence, at least part of which has been formed by coding at least video frames of the second frame format, and at least one video frame of the second sub-sequence has been predicted from at least one video frame of the first sub-sequence;

determining a dependency between at least the video frames of the second sub-sequence and at least one video frame of the first sub-sequence; and

encoding said dependency into the video sequence.

2. A method according to claim 1, further comprising

coding the video sequence into a plural number of scalability layers;

and

determining the dependency of the video frames of the second sub-sequence such that at least one video frame of the second sub-sequence is predicted from a group comprising

a video frame of a higher scalability layer,

a video frame of another sub-sequence in the same scalability layer.

3. A method according to claim 2, further comprising

determining the dependencies of the video frames of the second sub-sequence on the basis of at least a scalability layer identifier and a sub-sequence identifier.

4. A method according to claim 2, further comprising

coding the first scalability layer of the video sequence to comprise video frames according to one frame format, each one of the frames forming a separate sub-sequence.

5. A method according to claim 2, further comprising

coding the first scalability layer of the video sequence to comprise video frames according to both the first and the second frame format.

6. A method according to claim 2, further comprising

determining a unique identifier for each video frame as a combination of layer number, sub-sequence identifier and image number.

7. A method according to claim 2, further comprising

determining a unique identifier for each video frame on the basis of the image number.

8. A method according to claim 6, further comprising

adding the identifier to the header field of the video sequence or to the header field of the transfer protocol to be used for transferring the video sequence.

9. A method according to claim 6, further comprising

adding the identifier to the Supplemental Enhancement Information (SLI) data structure to be transmitted synchronously with the video sequence.

10. A method according to claim 1, wherein

the video frames of the first frame format are I-frames and the video frames of the second frame format are temporally forward and/or backward predicted P-frames, which have been predicted using at least one reference image.

11. A method according to claim 1, further comprising

coding the sub-sequences in such a way that at least some of the sub-sequence frames are temporally at least partly overlapping.

12. A method according to claim 1, further comprising

coding the video frames in such a way that the temporal predicting taking place between the video frames is block- or macroblock-specific.

13. A video encoder for forming a scalable, compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of other video frames, and the video frames of the second frame format being predicted from at least one other video frame, wherein the video encoder is configured to

form into the video sequence a first sub-sequence, at least part of which is formed by coding at least video frames of the first frame format;

form into the video sequence at least a second sub-sequence, at least part of which is formed by coding at least video frames of the second frame format, at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence;

determine a dependency between at least the video frames of the second sub-sequence and at least one video frame of the first sub-sequence; and

encode said dependency into the video sequence.

14. A method for decoding a scalably compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of the other video frames, and the video frames of the second frame format being predicted from at least one of the other video frames, the method comprising

decoding, at a decoder, a first sub-sequence of the video sequence, at least part of the first sub-sequence being formed by coding at least video frames of the first frame format;

decoding, at the decoder, at least a second sub-sequence of the video sequence, at least part of the second sub-sequence being formed by coding at least video frames of the second frame format, at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence;

determining, at the decoder, dependency data relating at least to the video frames comprised by the second sub-sequence of the video sequence; and

reconstructing, at the decoder, at least part of the video sequence on the basis of the dependencies of the sub-sequences.

15. A method according to claim 14, wherein video frames are entered into a sliding buffer memory in connection with decoding, further comprising

decoding from the video sequence an indication informing that the discontinuities in the numbering of the images of the image frames in the video sequence are intentional;

configuring, in response to the indication, the buffer memory to comprise a number of image frames corresponding to the missing image numbers; and

continuing the entering of the image frames comprised by the video sequence in question into the buffer memory from the correct image frame number after the buffer memory has been configured to comprise the number of image frames corresponding to the missing image numbers.

16. A method according to claim 15, further comprising

entering into the buffer memory a number of filler frames corresponding to the missing image numbers.

17. A method according to claim 14, further comprising

decoding the video sequence by removing at least one independently decodable sub-sequence from the video sequence.

18. A method according to claim 14, further comprising

initiating the decoding from a random point in the video sequence;

determining an independently decodable sub-sequence that is next after the random point, and

setting the value of the image number of the first video frame in the sub-sequence at zero.

19. A method according to claim 14, further comprising

identifying at least partly temporally overlapping sub-sequence frames from the video sequence based on the picture order count information, and

outputting from the decoder the last image frame in the decoding order, said frame being selected from a group of said at least partly temporally overlapping image frames.

20. A video decoder for decoding a scalably compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of the other video frames, and the video frames of the second frame format being predicted from at least one of the other video frames, wherein the video decoder is configured to

decode a first sub-sequence of the video sequence, at least part of which is formed by coding at least video frames of the first frame format;

decode at least a second sub-sequence of the video sequence, at least part of which is formed by coding at least video frames of the second frame format, and at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence;

determine the dependency data of at least the video frames comprised by the second sub-sequence of the video sequence; and

reconstruct at least part of the video sequence on the basis of the dependency of the sub-sequences.

21. A computer program product, stored on a non-transitory computer readable medium and executable in a data processing device, for coding video frames so as to form a scalable, compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of the other video frames, and the video frames of the second frame format being predicted from at least one of the other video frames, wherein the computer program comprises

a program code for forming a first sub-sequence of the video sequence, at least part of the sub-sequence being formed by coding at least video frames of the first frame format;

a program code for forming at least a second sub-sequence of the video sequence, at least part of the sub-sequence being formed by coding at least video frames of the second frame format, and at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence;

a program code for determining a dependency between at least the video frames of the second sub-sequence and at least one video frame of the first sub-sequence; and

a program code for encoding said dependency into the video sequence.

22. A computer program product, stored on a non-transitory computer readable medium and executable in a data processing device, for decoding a scalably compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of the other video frames, and the video frames of the second frame format being predicted from at least one of the other video frames, wherein the computer program comprises

a program code for decoding a first sub-sequence of the video sequence, at least part of the sub-sequence being formed by coding at least video frames of the first frame format;

and

a program code for decoding at least a second sub-sequence of the video sequence, at least part of the sub-sequence being formed by coding at least video frames of the second frame format, and at least one video frame of the second sub-sequence having been predicted from at least one video frame of the first sub-sequence; and

a program code for determining the dependency data of at least the video frames comprised by the second sub-sequence of the video sequence; and

a program code for reconstructing at least part of the video sequence on the basis of the dependency of the sub-sequences.

23. A method for coding video frames for the purpose of forming a scalable, compressed video sequence comprising video frames coded according to at least a first and a second frame format, the video frames of the first frame format being independent of the other video frames, and the video frames of the second frame format being predicted from at least one of the other video frames, the method comprising

encoding, at an encoder, the video sequence as at least part of a first sub-sequence, at least part of which has been formed by coding video frames of the at least first frame format;

encoding, at the encoder, the video sequence as at least a second sub-sequence, at least part of which has been formed by coding at least video frames of the second frame format, and at least one video frame of the second sub-sequence has been predicted from at least one video frame of the first sub-sequence; and

encoding, at the encoder, into the video sequence information indicating which video frames belong to the second sub-sequence, wherein subsequence information is different from picture type information.

24. A method according to claim 23, wherein

a removal of the second subsequence from the bitstream does not prevent decoding of the bitstream correctly.