Region merging and coding parameter reuse via merging

• US 10,672,028 B2
• Filed: 02/05/2019
• Issued: 06/02/2020
• Est. Priority Date: 04/13/2010
• Status: Active Grant

• Current Assignee: GE Video Compression, LLC (General Electric Company)
• Original Assignee: GE Video Compression, LLC (General Electric Company)
• Inventors: Helle, Philipp, Oudin, Simon, Winken, Martin, Marpe, Detlev, Wiegand, Thomas
• Primary Examiner(s): Retallick, Kaitlin A
• Application Number:
  US16/268,136
• Publication Number:
  US 20190197579A1
• Time in Patent Office: 483 Days
• Field of Search: None
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/463 by compressing encoding parameters before transmission
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/96 Tree coding, e.g. quad-tree coding

Abstract

A favorable merging or grouping of simply connected regions into which the array of information samples is sub-divided, is coded with a reduced amount of data. To this end, a predetermined relative locational relationship is defined enabling an identifying, for a predetermined simply connected region, of simply connected regions within the plurality of simply connected regions which have the predetermined relative locational relationship to the predetermined simply connected region. Namely, if the number is zero, a merge indicator for the predetermined simply connected region may be absent within the data stream. In other embodiments, spatial sub-division is performed depending on a first subset of syntax elements, followed by combining spatially neighboring simply connected regions depending on a second subset of syntax elements, to obtain an intermediate sub-division.

Claims

1. A decoder for decoding arrays of information samples encoded in a data stream each array representing a picture, the decoder comprising:

an extractor configured for

extracting, from the data stream, a merge indicator with respect to a region currently being decoded within an array of information samples, wherein the region currently being decoded is one of a plurality of regions obtained by dividing the array of information samples and the merge indicator is indicative of whether the region currently being decoded is to be reconstructed based on at least one coding parameter determined based on one or more coding parameters associated with a selected candidate region, wherein the at least one coding parameter and the one or more coding parameters include motion vectors,

identifying a set of candidate regions with respect to the region currently being decoded,

extracting a candidate identifier from the data stream, and

selecting the selected candidate region from the set of candidate regions in accordance with the candidate identifier; and

a reconstructor configured for

copying, if an indication is to copy, a motion vector of the one or more coding parameters as a motion vector of the at least coding parameter,

computing, if the indication is to compute, the at least one coding parameter based on the one or more coding parameters, the computing including;

determining a prediction of a motion vector of the at least coding parameter based on a motion vector of the one or more coding parameters,

extracting, from the data stream, a prediction residual of the motion vector of the at least one coding parameter, and

combining the prediction and the prediction residual of the motion vector of the at least one coding parameter to obtain the motion vector of the at least one coding parameter, and

reconstructing the region currently being decoded based on the motion vector of the at least one coding parameter.

2. The decoder of claim 1, wherein the extractor is further configured for determining the at least one coding parameter when at least some of the set of candidate regions share at least one identical coding parameter.

3. The decoder of claim 1, wherein the extractor is further configured for extracting the candidate identifier from the data stream when the set of candidate regions do not share an identical coding parameter.

4. The decoder of claim 1, wherein the array of information samples includes depth information.

5. The decoder of claim 1, wherein the picture includes arrays of information samples related to different color planes and the decoder is configured to decode the different color planes of the picture independently.

6. The decoder of claim 5, wherein the different color planes comprise an array of luma samples along with two arrays of chroma samples, wherein a first scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a second scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the vertical direction.

7. The decoder of claim 1, further comprising an entropy decoder configured to entropy decode the merge indicator using context modeling based on a first plurality of contexts.

8. The decoder of claim 7, wherein the entropy decoder is configured to entropy decode the candidate identifier using the context modeling based on a second plurality of contexts.

9. The decoder of claim 1, further comprising a sub-divider configured for obtaining the plurality of regions by:

partitioning the array of information samples into a first set of regions in accordance with a maximum region size extracted from the data stream, and

sub-partitioning at least a subset of the first set of regions into a set of sub-regions based on multi-tree subdivision information associated therewith.

10. The decoder of claim 9, wherein the extractor is configured for processing the plurality of regions in a depth-first traversal order.

11. A method of decoding arrays of information samples encoded in a data stream each representing a picture, the method comprising:

extracting, from the data stream, a merge indicator with respect to a region currently being decoded within an array of information samples, wherein the region currently being decoded is one of a plurality of regions obtained by dividing the array of information samples and the merge indicator is indicative of whether the region currently being decoded is to be reconstructed based on at least one coding parameter determined based on one or more coding parameters associated with a selected candidate region, wherein the at least one coding parameter and the one or more coding parameters include motion vectors;

identifying a set of candidate regions with respect to the region currently being decoded;

extracting a candidate identifier from the data stream;

selecting the selected candidate region from the set of candidate regions in accordance with the candidate identifier;

copying, if an indication is to copy, a motion vector of the one or more coding parameters as a motion vector of the at least coding parameter;

computing, if the indication is to compute, the at least one coding parameter based on the one or more coding parameters, the computing including;

determining a prediction of a motion vector of the at least coding parameter based on a motion vector of the one or more coding parameters,

extracting, from the data stream, a prediction residual of the motion vector of the at least one coding parameter, and

combining the prediction and the prediction residual of the motion vector of the at least one coding parameter to obtain the motion vector of the at least one coding parameter; and

reconstructing the region currently being decoded based on the motion vector of the at least one coding parameter.

12. The method of claim 11, wherein extracting, from the data stream, the candidate identifier is performed when the set of candidate regions do not share an identical coding parameter.

13. The method of claim 11, wherein the array of information samples includes depth information.

14. The method of claim 11, wherein the picture includes arrays of information samples related to different color planes and the decoder is configured to decode the different color planes of the picture independently.

15. The method of claim 14, wherein the different color planes comprise an array of luma samples along with two arrays of chroma samples, wherein a first scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a second scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the vertical direction.

16. The method of claim 11, further comprising determining the at least one coding parameter when at least some of the set of candidate regions share at least one identical coding parameter.

17. The method of claim 11, further comprising obtaining the plurality of regions by:

partitioning the array of information samples into a first set of regions in accordance with a maximum region size extracted from the data stream; and

sub-partitioning at least a subset of the first set of regions into a set of sub-regions based on multi-tree subdivision information associated therewith.

18. The method of claim 17, wherein the plurality of regions is processed in a depth-first traversal order.

19. An encoder for encoding arrays of information samples into a data stream, each array representing a picture into a data stream, the encoder comprising:

a sub-divider configured for dividing the array of information samples into a plurality of regions; and

a data stream generator configured for

encoding, into the data stream, a merge indicator with respect to one of the plurality of regions that is currently being coded, wherein the merge indicator is indicative of whether the region currently being coded is to be reconstructed based on at least one coding parameter determined based on one or more coding parameters associated with a selected candidate region selected from a set of candidate regions, identified with respect to the region currently being coded, wherein the at least one coding parameter and the one or more coding parameters include motion vectors,

encoding, into the data stream, a candidate identifier that identifies a specific one of the set of candidate regions as the selected candidate region,

wherein if an indication is to copy, a motion vector of the one or more coding parameters is copied as a motion vector of the at least coding parameter, and

if the indication is to compute, the at least one coding parameter is computed based on the one or more coding parameters, which includes;

determining a prediction of a motion vector of the at least coding parameter based on a motion vector of the one or more coding parameters, and

combining the prediction and a prediction residual of the motion vector of the at least one coding parameter to obtain the motion vector of the at least one coding parameter, and

encoding, into the data stream, the one or more coding parameters associated with the selected candidate region and the prediction residual of the motion vector of the at least one coding parameter.

20. The encoder of claim 19, wherein the data stream generator is further configured for encoding, into the data stream, the candidate identifier when the set of candidate regions do not share an identical coding parameter.

21. The encoder of claim 19, wherein the array of information samples includes depth information.

22. The encoder of claim 19, wherein the picture includes arrays of information samples related to different color planes and the decoder is configured to decode the different color planes of the picture independently.

23. The encoder of claim 22, wherein the different color planes comprise an array of luma samples along with two arrays of chroma samples, wherein a first scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the horizontal direction differs from a second scaling factor for a spatial resolution of the arrays of chroma samples relative to the array of luma samples in the vertical direction.

24. The encoder of claim 19, wherein the data stream generator is further configured for determining the at least one coding parameter when at least some of the set of candidate regions share at least one identical coding parameter.

25. The encoder of claim 19, wherein the sub-divider is configured for obtaining the plurality of regions by:

partitioning the array of information samples into a first set of regions in accordance with a maximum region size encoded into the data stream, and

sub-partitioning at least a subset of the first set of regions into a set of sub-regions based on multi-tree subdivision information associated therewith.

26. A non-transitory computer-readable medium for storing data associated with a video, comprising:

a data stream stored in the non-transitory computer-readable medium, the data stream comprising encoded information including arrays of information samples each representing a picture, the data stream further comprising,

a merge indicator with respect to a region currently being decoded within an array of information samples, wherein the region currently being decoded is one of a plurality of regions obtained by dividing the array of information samples and the merge indicator is indicative of whether the region currently being decoded is to be reconstructed based on at least one coding parameter determined based on one or more coding parameters associated with a selected candidate region, identified with respect to the region, wherein the at least one coding parameter and the one or more coding parameters include motion vectors;

a candidate identifier that identifies a specific one of the set of candidate regions as the selected candidate region;

a prediction residual of a motion vector of the at least one coding parameter; and

one or more coding parameters associated with the selected candidate region, from which the at least one coding parameter is copied or computed, and used to reconstruct the region,

wherein if an indication is to copy, a motion vector of the one or more coding parameters is copied as a motion vector of the at least coding parameter, and

if the indication is to compute, the at least one coding parameter is computed based on the one or more coding parameters, which includes;

determining a prediction of a motion vector of the at least coding parameter based on a motion vector of the one or more coding parameters,

extracting, from the data stream, the prediction residual of the motion vector of the at least one coding parameter, and

combining the prediction and a prediction residual of the motion vector of the at least one coding parameter to obtain the motion vector of the at least one coding parameter.

27. The computer-readable medium of claim 26, wherein the candidate identifier is encoded into the data stream when the set of candidate regions do not share an identical coding parameter.

28. The computer-readable medium of claim 26, wherein the picture includes arrays of information samples related to different color planes and the decoder is configured to decode the different color planes of the picture independently.

29. The computer-readable medium of claim 26, wherein the at least one coding parameter is determined when at least some of the set of candidate regions share at least one identical coding parameter.

30. The computer-readable medium of claim 26, the data stream further comprising a maximum region size, wherein the plurality of regions is obtained by:

partitioning the array of information samples into a first set of regions in accordance with the maximum region size, and

sub-partitioning at least a subset of the first set of regions into a set of sub-regions based on multi-tree subdivision information associated therewith.