INTER-PLANE PREDICTION

• US 20130034171A1
• Filed: 10/11/2012
• Published: 02/07/2013
• Est. Priority Date: 04/13/2010
• Status: Active Grant

• Current Assignee: GE Video Compression, LLC (General Electric Company)
• Original Assignee: Fraunhofer-Gesellschaft ZUR Forderung Der Angewandten Forschung EV
• Inventors: Wiegand, Thomas, Schwarz, Heiko, Marpe, Detlev, Winken, Martin, Kirchhoffer, Heiner
• Granted Patent: US 10,873,749 B2
• Time in Patent Office: Days
• US Class Current: 375/240.25
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/11 among a plurality of spatial predictive coding modes
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/96 Tree coding, e.g. quad-tree coding

Abstract

A better rate distortion ratio is achieved by making interrelationships between coding parameters of different planes available for exploitation for the aim of redundancy reduction despite the additional overhead resulting from the need to signal the inter-plane prediction information to the decoder. In particular, the decision to use inter plane prediction or not may be performed for a plurality of planes individually. Additionally or alternatively, the decision may be done on a block basis considering one secondary plane.

Claims

1. A decoder for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, the decoder being configured to

reconstruct each array of information samples by processing payload data associated with simply connected regions into which each array of information samples is sub-divided in a way prescribed by coding parameters associated with the simply connected regions;

derive the coding parameters associated with the simply connected regions of a first array of the arrays of information samples from the data stream;

derive inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples from the data stream,

depending on the inter-plane interchange information for the simply connected regions of the second array, decide, for each simply connected region or a proper subset of the simply connected regions of the second array, to

derive the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, at least partially from the coding parameters of a locally corresponding simply connected region of the first array and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the coding parameters thus derived;

or

ignore the coding parameters for the locally corresponding simply connected region of the first array, derive the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array from the data stream, and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the associated coding parameters derived from the data stream.

2. The decoder according to claim 1 wherein the decoder is configured to derive subdivision information from the data stream and subdivide the arrays of information samples into the simply connected regions depending on the subdivision information.

3. The decoder according to claim 1 wherein the decoder is configured to subdivide the arrays of information samples into the simply connected regions such that the simply connected regions are of varying size.

4. The decoder according to claim 1 wherein the decoder is configured to subdivide the arrays of information samples into the simply connected regions such that the subdivision into the simply connected regions is different for the first and second arrays.

5. The decoder according to claim 1 wherein the decoder is configured to, in processing the payload data associated with the simply connected regions, predict information samples within the simply connected regions in a way prescribed by the coding parameters associated with the simply connected regions.

6. The decoder according to claim 1 wherein the coding parameters associated with the simply connected regions comprise one or more of the following coding parameters:

block prediction modes specifying what prediction is used for the respective simply connected region among, at least, intra prediction and inter prediction,

intra prediction modes specifying how an intra prediction signal is generated,

an identifier specifying how many prediction signals are combined for generating the final prediction signal for the respective simply connected region,

reference indices specifying which reference picture(s) is/are employed for motion-compensated prediction,

motion parameters specifying how the prediction signal(s) is/are generated using the reference picture(s), and

an identifier specifying how the reference picture(s) is/are filtered for generating a prediction signal; and

subdivision information specifying how a respective simply connected region is subdivided into even smaller simply connected regions.

7. The decoder according to claim 1 wherein the decoder is configured to determine the locally corresponding simply connected region of the first array by selecting a predetermined information sample within a current simply connected region or a current proper subset of the simply connected regions of the second array, and appointing that simply connected region among the simply connected regions of the first array as the locally corresponding simply connected region, which comprises an information sample closest to, or within which resides, a location of the predetermined information sample within the first array.

8. The decoder according to claim 1 wherein the decoder is configured to determine the locally corresponding simply connected region of the first array by appointing that simply connected region among the simply connected regions of the first array as the locally corresponding simply connected region, which spatially overlaps to a largest extent with an area occupied by a current simply connected region or a current proper subset of the simply connected regions of the second array within the first array.

9. The decoder according to claim 1 wherein the decoder is configured to, in deriving the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, at least partially from the coding parameters of the locally corresponding simply connected region of the first array, completely infer the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, from the coding parameters of the locally corresponding simply connected region of the first array without deriving any residual data for the respective simply connected region or the proper subset of the simply connected regions of the second array, by simply copying or by adaptation taking into account the difference in spatial resolution and spatial phase relationship between the first and second arrays.

10. The decoder according to claim 1 wherein the decoder is configured to, in deriving the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, at least partially from the coding parameters of the locally corresponding simply connected region of the first array, use the coding parameters of the locally corresponding simply connected region of the first array as a prediction for the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, and combine the prediction with residual data for the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array derived from the data stream.

11. The decoder according to claim 1, wherein

the first array of the arrays of information samples relates to a brightness or color component,

the second array of the arrays of information samples is a depth map.

the coding parameters associated with the simply connected regions comprise one or more of the following coding parameters:

motion parameters used to reconstruct the first and second arrays by motion-compensated prediction, and

subdivision information specifying how respective simply connected regions of the first and second arrays are subdivided into even smaller simply connected regions.

12. The decoder according to claim 1, wherein

each information component is selected from a group comprising different color components, a depth map and transparency data,

the decoder is configured to derive subdivision information from the data stream and subdivide the arrays of information samples into the simply connected regions depending on the subdivision information by use of multi-tree based sub-divisioning such that the simply connected regions are of varying size and the multi-tree based subdivision into the simply connected regions is different for the first and second arrays.

18. The decoder according to claim 1, wherein

the first array of the arrays of information samples relates to a brightness or color component,

the second array of the arrays of information samples is a depth map.

the coding parameters associated with the simply connected regions comprise one or more of the following coding parameters:

motion parameters used to reconstruct the first and second arrays by motion-compensated prediction, and

subdivision information specifying how respective simply connected regions of the first and second arrays are subdivided into even smaller simply connected regions.

13. A decoder for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, wherein at least one of the array of information samples is a primary array of information samples and at least another two thereof are secondary arrays of information samples, the decoder being configured to

reconstruct each array of information samples by processing payload data associated with the respective array of information samples in a way prescribed by coding parameters also associated with the respective array of information samples;

derive the coding parameters associated with the primary array from the data stream;

derive inter-plane interchange information for each secondary array from the data stream,

depending on the inter-plane interchange information for the secondary arrays, decide, for each secondary array individually, to

derive the coding parameters for the respective secondary array, at least partially, from the coding parameters of the primary array and decode the payload data associated with the respective secondary array in a way prescribed by the coding parameters thus derived;

or

ignore the coding parameters of the primary array, derive the coding parameters for the respective secondary array from the data stream and decode the payload data associated with the respective secondary array in a way prescribed by the coding parameters for the respective second array independent from the ignored coding parameters.

14. The decoder according to claim 13 wherein the arrays of information samples represent different brightness components, color components, and/or non-brightness/non-color components such as depth or transparency components of the picture.

15. The decoder according to claim 13 wherein the decoder is configured to derive an plane appointment information from the data stream and to appoint the arrays of information samples primary and secondary array depending on the plane appointment information.

16. The decoder according to claim 15 wherein the scene is time-varying and each plane comprises a sequences of arrays of information samples representing a respective one of the different spatially sampled information components in a time-sampled manner with at least an array of information samples per picture of a sequence of pictures of the scene, and the decoder is configured to perform the derivation of the plane appointment information and appointment in a time granularity corresponding to pictures, or groups of pictures, or to perform the derivation and appointment merely once for initialization.

17. The decoder according to claim 13 wherein the scene is time-varying and each plane comprises a sequences of arrays of information samples representing a respective one of the different spatially sampled information components in a time-sampled manner with at least an array of information samples per picture of a sequence of pictures of the scene, and the decoder is configured to perform the derivation of the inter-plane interchange information and decision in a time granularity corresponding to sub-regions into which the pictures are sub-divided, pictures, or groups of pictures.

19. A method for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, the method comprising

reconstructing each array of information samples by processing payload data associated with simply connected regions into which each array of information samples is sub-divided in a way prescribed by coding parameters associated with the simply connected regions;

deriving the coding parameters associated with the simply connected regions of a first array of the arrays of information samples from the data stream;

deriving inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples from the data stream,

depending on the inter-plane interchange information for the simply connected regions of the second array, decide, for each simply connected region or a proper subset of the simply connected regions of the second array, to

deriving the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, at least partially from the coding parameters of a locally corresponding simply connected region of the first array and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the coding parameters thus derived;

or

ignoring the coding parameters for the locally corresponding simply connected region of the first array, derive the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array from the data stream, and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the associated coding parameters derived from the data stream.

20. A method for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, wherein at least one of the array of information samples is a primary array of information samples and at least another two thereof are secondary arrays of information samples, the method comprising

reconstructing each array of information samples by processing payload data associated with the respective array of information samples in a way prescribed by coding parameters also associated with the respective array of information samples;

deriving the coding parameters associated with the primary array from the data stream;

deriving inter-plane interchange information for each secondary array from the data stream,

depending on the inter-plane interchange information for the secondary arrays, decide, for each secondary array individually, to

deriving the coding parameters for the respective secondary array, at least partially, from the coding parameters of the primary array and decode the payload data associated with the respective secondary array in a way prescribed by the coding parameters thus derived;

or

ignoring the coding parameters of the primary array, derive the coding parameters for the respective secondary array from the data stream and decode the payload data associated with the respective secondary array in a way prescribed by the coding parameters for the respective second array independent from the ignored coding parameters.

21. An encoder for generating a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, the encoder being configured to

determining, for each array of information samples, payload data associated with simply connected regions into which each array of information samples is sub-divided, and coding parameters associated with the simply connected regions and prescribing a way by which the payload data is to be reconstructed to reconstruct each array of information samples, and inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples in a granularity of the simply connected region or of proper subsets of the simply connected regions of the second array;

and

insert the coding parameters associated with the simply connected regions of a first array of the arrays of information samples and the inter-plane interchange information into the data stream;

wherein the decoder is configured to perform the determination such that the inter-plane interchange information for the simply connected regions of the second array indicates as to whether the coding parameters for a respective simply connected region or a proper subset of the simply connected regions of the second array are, at least partially, to be derived from the coding parameters of a locally corresponding simply connected region of the first array, or not, and to, depending on the inter-plane interchange information,

refrain from inserting the coding parameters for a respective simply connected region or a proper subset of the simply connected regions of the second array, into the data stream, or insert a prediction residual for the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array into the data stream enabling a reconstruction based on a prediction from the coding parameters of the locally corresponding simply connected region of the first array;

or

inserting the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array into the data stream as they are.

22. An encoder for generating a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, wherein at least one of the array of information samples is a primary array of information samples and at least another two thereof are secondary arrays of information samples, the encoder being configured to

determine, for each array of information samples, payload data and coding parameters prescribing a way by which the payload data is to be reconstructed to reconstruct the respective array of information samples, and inter-plane interchange information for each secondary array from the data stream;

and

insert the coding parameters associated with the primary array and the inter-plane interchange information into the data stream;

wherein the decoder is configured to perform the determination such that the inter-plane interchange information indicates as to whether the coding parameters for a respective secondary array are, at least partially, to be derived from the coding parameters of the primary array, or not, and to, depending on the inter-plane interchange information,

refrain from inserting the coding parameters for the respective secondary array into the data stream or insert a prediction residual for the coding parameters for the respective secondary array into the data stream enabling a reconstruction based on a prediction from the coding parameters of the primary array;

or

insert the coding parameters for the respective secondary array into the data stream as they are.

23. A method for generating a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, the method comprising

determining, for each array of information samples, payload data associated with simply connected regions into which each array of information samples is sub-divided, and coding parameters associated with the simply connected regions and prescribing a way by which the payload data is to be reconstructed to reconstruct each array of information samples, and inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples in a granularity of the simply connected region or of proper subsets of the simply connected regions of the second array;

and

inserting the coding parameters associated with the simply connected regions of a first array of the arrays of information samples and the inter-plane interchange information into the data stream;

wherein the determination is performed such that the inter-plane interchange information for the simply connected regions of the second array indicates as to whether the coding parameters for a respective simply connected region or a proper subset of the simply connected regions of the second array are, at least partially, to be derived from the coding parameters of a locally corresponding simply connected region of the first array, or not, wherein, depending on the inter-plane interchange information,

no insertion of the coding parameters for a respective simply connected region or a proper subset of the simply connected regions of the second array, into the data stream takes place, or merely a prediction residual for the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array is inserted into the data stream, enabling a reconstruction based on a prediction from the coding parameters of the locally corresponding simply connected region of the first array;

or

the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array are inserted into the data stream as they are.

24. A method for generating a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, wherein at least one of the array of information samples is a primary array of information samples and at least another two thereof are secondary arrays of information samples, the method comprising

determining, for each array of information samples, payload data and coding parameters prescribing a way by which the payload data is to be reconstructed to reconstruct the respective array of information samples, and inter-plane interchange information for each secondary array from the data stream;

and

inserting the coding parameters associated with the primary array and the inter-plane interchange information into the data stream;

wherein the determination is performed such that the inter-plane interchange information indicates as to whether the coding parameters for a respective secondary array are, at least partially, to be derived from the coding parameters of the primary array, or not, wherein, depending on the inter-plane interchange information,

no insertion of the coding parameters for the respective secondary array into the data stream takes place or merely an insertion of a prediction residual for the coding parameters for the respective secondary array into the data stream enabling a reconstruction based on a prediction from the coding parameters of the primary array;

or

the coding parameters for the respective secondary array are inserted into the data stream as they are.

25. A data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, the data stream enabling reconstruction of each array of information samples by processing payload data associated with simply connected regions into which each array of information samples is sub-divided in a way prescribed by coding parameters associated with the simply connected regions, the data stream comprising

the coding parameters associated with the simply connected regions of a first array of the arrays of information samples;

inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples,

depending on the inter-plane interchange information for the simply connected regions of the second array, for each simply connected region or a proper subset of the simply connected regions of the second array,

an absence of the coding parameters for a respective simply connected region or a proper subset of the simply connected regions of the second array, or merely a prediction residual for the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array, enabling a reconstruction based on a prediction from the coding parameters of the locally corresponding simply connected region of the first array;

or

the coding parameters for the respective simply connected region or the respective proper subset of the simply connected regions of the second array as they are.

26. A data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, wherein at least one of the array of information samples is a primary array of information samples and at least another two thereof are secondary arrays of information samples, the data stream enabling reconstruction of each array of information samples by processing payload data associated with the respective array of information samples in a way prescribed by coding parameters also associated with the respective array of information samples, the data stream comprising

the coding parameters associated with the primary array;

inter-plane interchange information for each secondary array;

depending on the inter-plane interchange information for the secondary arrays, for each secondary array individually,

an absence of the coding parameters for the respective secondary array into the data stream or merely a prediction residual for the coding parameters for the respective secondary array, enabling a reconstruction based on a prediction from the coding parameters of the primary array;

or

the coding parameters for the respective secondary array as they are.

27. A computer readable digital storage medium having stored thereon a computer program comprising a program code for performing, when running on a computer, a method for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each plane comprising an array of information samples, said method comprising

reconstructing each array of information samples by processing payload data associated with simply connected regions into which each array of information samples is sub-divided in a way prescribed by coding parameters associated with the simply connected regions;

deriving the coding parameters associated with the simply connected regions of a first array of the arrays of information samples from the data stream;

deriving inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples from the data stream,

depending on the inter-plane interchange information for the simply connected regions of the second array, decide, for each simply connected region or a proper subset of the simply connected regions of the second array, to

deriving the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array, at least partially from the coding parameters of a locally corresponding simply connected region of the first array and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the coding parameters thus derived;

or

ignoring the coding parameters for the locally corresponding simply connected region of the first array, derive the coding parameters for the respective simply connected region or the proper subset of the simply connected regions of the second array from the data stream, and decode the payload data associated with the respective simply connected region or the proper subset of the simply connected regions of the second array in a way prescribed by the associated coding parameters derived from the data stream.

28. A decoder for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each information component being selected from a group comprising different color components, a depth map and transparency data, each plane comprising an array of information samples, the decoder being configured to

subdivide simply connected regions into which each array of information samples is sub-divided, into simply connected sub-regions by use of multi-tree based sub-divisioning using subdivision information associated with the simply connected regions;

reconstruct each array of information samples by processing payload data associated with the simply connected regions in a way prescribed by coding parameters associated with the simply connected sub-regions of the simply connected regions and specifying how these simply connected sub-regions are predicted or a residual coding mode to be used for these simply connected sub-regions;

derive the subdivision information associated with the simply connected regions of a first array of the arrays of information samples and the coding parameters associated with the simply-connected sub-regions of the simply connected regions of the first array from the data stream;

derive inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples from the data stream,

depending on the inter-plane interchange information for the simply connected regions of the second array, decide, for each simply connected region, to

derive the subdivision information for the respective simply connected region from the subdivision information of a locally corresponding simply connected region of the first array by deriving for each simply-connected sub-region of the locally corresponding simply connected region of the first array, or for each simply-connected sub-region of the locally corresponding simply connected region of the first array the size of which fulfills a condition, a syntax element from the data stream, which specifies if a sub-region of the respective simply connected region, corresponding to the respective simply-connected sub-region of the locally corresponding simply connected region of the first array is further decomposed into respective ones of the simply connected sub-regions of the respective simply connected region of the second array, or if the sub-region of the respective simply connected region of the second array, corresponding to the respective simply-connected sub-region of the locally corresponding simply connected region of the first array remains un-decomposed and directly represents one of the simply connected sub-regions of the respective simply connected region, derive the coding parameters associated with the simply-connected sub-regions of the respective simply connected region of the second array from the data stream and decode the payload data associated with the respective simply connected region in a way prescribed by the coding parameters thus derived;

or

ignore the subdivision information for the locally corresponding simply connected region of the first array, and derive the subdivision information for the respective simply connected region of the second array from the data stream, derive the coding parameters associated with the simply-connected sub-regions of the respective simply connected region of the second array from the data stream and decode the payload data associated with the respective simply connected region in a way prescribed by the associated coding parameters derived from the data stream.

29. A method for decoding a data stream representing different spatially sampled information components of a picture of a scene in planes, each information component being selected from a group comprising different color components, a depth map and transparency data, each plane comprising an array of information samples, the method comprising

subdivide simply connected regions into which each array of information samples is sub-divided, into simply connected sub-regions by use of multi-tree based sub-divisioning using subdivision information associated with the simply connected regions;

reconstruct each array of information samples by processing payload data associated with the simply connected regions in a way prescribed by coding parameters associated with the simply connected sub-regions of the simply connected regions and specifying how these simply connected sub-regions are predicted or a residual coding mode to be used for these simply connected sub-regions;

derive the subdivision information associated with the simply connected regions of a first array of the arrays of information samples and the coding parameters associated with the simply-connected sub-regions of the simply connected regions of the first array from the data stream;

derive inter-plane interchange information for the simply connected regions of a second array of the arrays of information samples from the data stream,

depending on the inter-plane interchange information for the simply connected regions of the second array, decide, for each simply connected region, to

derive the subdivision information for the respective simply connected region from the subdivision information of a locally corresponding simply connected region of the first array by deriving for each simply-connected sub-region of the locally corresponding simply connected region of the first array, or for each simply-connected sub-region of the locally corresponding simply connected region of the first array the size of which fulfills a condition, a syntax element from the data stream, which specifies if a sub-region of the respective simply connected region, corresponding to the respective simply-connected sub-region of the locally corresponding simply connected region of the first array is further decomposed into respective ones of the simply connected sub-regions of the respective simply connected region of the second array, or if the sub-region of the respective simply connected region of the second array, corresponding to the respective simply-connected sub-region of the locally corresponding simply connected region of the first array remains un-decomposed and directly represents one of the simply connected sub-regions of the respective simply connected region, derive the coding parameters associated with the simply-connected sub-regions of the respective simply connected region of the second array from the data stream and decode the payload data associated with the respective simply connected region in a way prescribed by the coding parameters thus derived;

or

ignore the subdivision information for the locally corresponding simply connected region of the first array, and derive the subdivision information for the respective simply connected region of the second array from the data stream, derive the coding parameters associated with the simply-connected sub-regions of the respective simply connected region of the second array from the data stream and decode the payload data associated with the respective simply connected region in a way prescribed by the associated coding parameters derived from the data stream.