METHOD OF ADAPTIVE FRAME PREDICTION FOR MULTIVIEW VIDEO SEQUENCE CODING

• US 20130294504A1
• Filed: 01/20/2012
• Published: 11/07/2013
• Est. Priority Date: 01/20/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Mishourovsky, Mikhail, Fartukov, Alexey, Kovliga, Igor
• Granted Patent: US 9,538,182 B2
• Time in Patent Office: Days
• US Class Current: 375/240.02
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A process and apparatus for digital compression of multiview video, supplied by additional data of scene depth. The method of coding is offered, including: each frame of the multiview video sequence, encoded again, determined according to the predefined order of coding, is represented as a collection of non-overlapped blocks, such that at least one already encoded frame is detected, corresponding to the given view and designated as reference, the synthesized frames for encoded and reference frames, differing that for each non-overlapped block of pixels of the encoded frame designated as an encoded block the spatial-combined block in the synthesized frame is determined, corresponding to the encoded frame, designated as the virtual block, for which spatial position of the block of pixels is determined in the synthesized frame corresponding to a reference frame.

Claims

1. A method of coding a multiview video sequence, in which a multiview sequence of depth maps is included, the method comprising:

representing as a set of non-overlapping blocks of pixels each new coded frame of the multiview video sequence, determined in accordance with a preset coding order;

detecting at least one already encoded frame, corresponding to a given view and designating it as a reference frame,

generating synthesized frames for encoded and reference frames, wherein for each non-overlapping block of pixels of the encoded frame designated as a block to be encoded, a spatial-combined block of a respective synthesized frame of the synthesized frames is determined, corresponding to the encoded frame, and designated as a virtual block, for which a spatial position of the block of pixels is determined in the respective synthesized frame corresponding to the reference frame, such that the reference virtual block so determined, is a most accurate numerical approximation of the virtual block;

for the reference virtual block determined in such way, the spatial-combined block is determined, belonging to the reference frame designated as the reference block, and

computing an error between the virtual block and the reference virtual block, and an error between the reference virtual block and the reference block;

selecting the minimum of these errors and determining, at least, one mode of difference coding, based on the selected error; and

setting which of the detected blocks are necessary for generation of prediction at the subsequent difference coding of the block to be encoded, and the difference coding of the block to be encoded, is performed in accordance with the selected mode of difference coding.

2. The method of claim 1, wherein when the error between the virtual block and the reference virtual block is less than the error between the reference virtual block and the reference block, the mode of difference coding is selected such that the reference block is used as the prediction;

otherwise the mode of difference coding is selected such that the virtual block is used as the prediction.

3. The method of claim 2, wherein an error between the virtual block and the block to be encoded, is computed, and an error between the reference block and the block to be encoded, is computed, and, when the error between the virtual block and the block to be encoded, is less than an error between the reference block and the block to be encoded, the virtual block is selected as the prediction;

otherwise the reference block is selected;

based on the selected mode of difference coding and the selected block, coding of data of the selected block is performed, and also the difference coding of the block to be encoded is performed.

4. The method of claim 3, wherein, when the error relating to the selected mode of difference coding at which the reference block is selected as a prediction, and the minimum detected error is an error between the virtual block and the encoded block, and the detected absolute difference between these errors does not exceed predefined threshold value, then the reference block is selected as a prediction;

when the error relating to the selected mode of difference coding at which the virtual block is selected as a prediction, and the minimum detected error is an error between the reference block and the encoded block, and the detected absolute difference between these errors does not exceed predefined threshold value, then the virtual block is selected as a prediction;

otherwise the block, according to a specified mode of coding, is selected as a prediction;

coding of data of the selected block is performed, and difference coding of the encoded block is performed.

5. The method of claim 4, wherein the reference virtual block and the reference block are used for computation of the difference block, and the difference block is computed as a pixel-by-pixel difference between the virtual block and a pixel-by-pixel difference between the reference virtual block and the reference block;

an error between the detected difference block and the encoded block is computed;

when the detected error is less than an error between the block, selected on the previous steps as a predictor, and the encoded block, then the detected difference block is used as a predictor; and

coding of data of the selected block is performed, and difference coding of the encoded block is performed.

6. The method of claim 1, wherein at least one of one-sided synthesis and two-sided synthesis is performed in order to generate the synthesized frames.

7. The method of claim 1, further comprising:

determining an optimum mode of prediction, among a temporal mode of prediction and a parallax mode of prediction, based on a comparison of the error between the virtual block and the reference virtual block, and the error between the reference virtual block and the reference block.

8. An apparatus for coding a multiview video sequence, in which a multiview sequence of depth maps is included, the apparatus comprising:

a control unit to receive an input of the multiview video sequence;

a frame synthesis unit to generate synthesized frames, using the multiview sequence of depth maps; and

an adaptive three-dimensional (3D) frame prediction unit to select a prediction, using the generated synthesized frames.