METHOD AND DEVICE FOR ENCODING A DEPTH MAP OF MULTI VIEWPOINT VIDEO DATA, AND METHOD AND DEVICE FOR DECODING THE ENCODED DEPTH MAP

• US 20140192154A1
• Filed: 08/09/2012
• Published: 07/10/2014
• Est. Priority Date: 08/09/2011
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jeong, Seung-soo, Choi, Byeong-doo, Park, Jeong-hoon
• Granted Patent: US 9,402,066 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/96 Tree coding, e.g. quad-tree coding
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format

Abstract

A method and apparatus for decoding the depth map of multi-view video data are provided. The method includes splitting a block of restored multi-view color video frame into a partition based on a pixel value of the block of the prediction-encoded and restored multi-view color video frame; obtaining a parameter indicating correlation between block partitions of the multi-view color video frame and block partitions of the depth map frame using peripheral pixel values of the block partitions of the multi-view color video frame and peripheral pixel values of the block partitions of the depth map frame corresponding to the block partitions of the multi-view color video frame with respect to each of the block partitions of the restored multi-view color video frame; and obtaining prediction values of corresponding block partitions of the depth map frame from the block partitions of the restored multi-view color video frame using the obtained parameter.

Claims

1. A method of encoding a depth map of multi-view video data, the method comprising:

obtaining a multi-view color video frame and a depth map frame corresponding to the multi-view color video frame;

prediction-encoding and restoring the obtained multi-view color video frame;

splitting a block of the restored multi-view color video frame into at least one partition based on a pixel value of the block of the restored multi-view color video frame;

obtaining a parameter indicating a correlation between each of block partitions of the multi-view color video frame and each of block partitions of the depth map frame by using peripheral pixel values of the block partitions of the multi-view color video frame and peripheral pixel values of the block partitions of the depth map frame corresponding to the block partitions of the multi-view color video frame with respect to each of the block partitions of the restored multi-view color video frame; and

obtaining prediction values of the corresponding block partitions of the depth map frame from the block partitions of the restored multi-view color video frame by using the obtained parameter.

2. The method of claim 1, wherein the prediction-encoding of the multi-view color video frame comprises:

determining coding units having a tree structure including coding units of a coded depth from among hierarchical coding units corresponding to depths, for each of at least one maximum coding unit that is split from the multi-view color video frame;

determining partitions for prediction-encoding each of the coding units corresponding to the coded depth; and

determining transformation units having a tree structure by performing transformation based on hierarchical transformation units,

wherein the depths each denote a number of times the at least one maximum coding unit is spatially split.

3. The method of claim 1, wherein the obtaining of the parameter comprises determining the parameter so that differences between the peripheral pixel values of the block partitions of the depth map frame obtained from the peripheral pixel values of the block partitions of the multi-view color video frame and original peripheral pixel values of the block partitions of the depth map frame are minimized by using the correlation.

4. The method of claim 3, wherein the correlation is a linear relationship, and the parameter comprises a weight and an offset value defining the linear relationship between the peripheral pixel values of the block partitions of the multi-view color video frame and the corresponding peripheral pixel values of the block partitions of the depth map frame.

5. The method of claim 4, wherein, if the peripheral pixel values of the block partitions of the multi-view color video frame is Rec_Y′

, prediction values of the corresponding peripheral pixel values of the block partitions of the depth map frame is Pred_D, the weight is a, and the offset value is b, the correlation is defined as an equation;

Pred_—

D=a*Rec_—

Y′

+b.

6. The method of claim 1, wherein the obtaining prediction values of the corresponding block partitions of the depth map frame comprises:

obtaining the prediction values of the corresponding block partitions of the depth map frame by using block partitions of a restored multi-view color video frame having different color components; and

comparing costs of the prediction values obtained from the block partitions of the restored multi-view color video frame having different color components and determining block partitions of a multi-view color video frame to be used to predict the block partitions of the depth map frame.

7. An apparatus for encoding a depth map of multi-view video data, the apparatus comprising:

an image obtainer which is configured to obtain a multi-view color video frame and a depth map frame corresponding to the multi-view color video frame;

a color video frame encoder which is configured to prediction-encode the obtained multi-view color video frame;

a restorer which is configured to restore the encoded color video frame;

a splitter which is configured to split a block of the restored multi-view color video frame into at least one partition based on a pixel value of the block of the restored multi-view color video frame;

a correlation parameter obtainer which is configured to obtain a parameter indicating a correlation between each of block partitions of the multi-view color video frame and each of block partitions of the depth map frame by using peripheral pixel values of the block partitions of the multi-view color video frame and peripheral pixel values of the block partitions of the depth map frame corresponding to the block partitions of the multi-view color video frame with respect to each of the block partitions of the restored multi-view color video frame; and

a depth map frame predictor which is configured to obtain prediction values of the corresponding block partitions of the depth map frame from the block partitions of the restored multi-view color video frame by using the obtained parameter.

8. A method of decoding a depth map of multi-view video data, the method comprising:

receiving a bitstream obtained by encoding a multi-view color video frame and a depth map frame corresponding to the multi-view color video frame;

decoding the multi-view color video frame;

splitting a block of the decoded multi-view color video frame into at least one partition based on a pixel value of the block of the decoded multi-view color video frame;

obtaining a parameter indicating a correlation between each of block partitions of the multi-view color video frame and each of block partitions of the depth map frame by using peripheral pixel values of the block partitions of the multi-view color video frame and peripheral pixel values of the block partitions of the depth map frame corresponding to the block partitions of the multi-view color video frame with respect to each of the block partitions of the decoded multi-view color video frame; and

obtaining prediction values of the corresponding block partitions of the depth map frame from the block partitions of the decoded multi-view color video frame by using the obtained parameter.

9. The method of claim 8, wherein the decoding of the multi-view color video frame comprises:

obtaining information about a size of each of at least one maximum coding unit that is split from the multi-view color video frame, a depth denoting a number of times the at least one maximum coding unit is spatially split, partitions used to prediction-encode coding units hierarchically configured according to the depth, and transformation units having a hierarchical structure, from the bitstream;

determining coding units having a tree structure including coding units corresponding to a coded depth from among hierarchical coding units corresponding to depths, for each of the at least one maximum coding unit that is split from the multi-view color video frame;

determining partitions for prediction-decoding each of the coding units corresponding to the coded depth; and

determining transformation units having a tree structure, based on the obtained information,

wherein the depths each denote the number of times the at least one maximum coding unit is spatially split.

10. The method of claim 8, wherein the obtaining of the parameter comprises:

determining the parameter so that differences between the peripheral pixel values of the block partitions of the depth map frame obtained from the peripheral pixel values of the block partitions of the multi-view color video frame and original peripheral pixel values of the block partitions of the depth map frame are minimized by using the correlation.

11. The method of claim 10, wherein the correlation is a linear relationship, and the parameter comprises a weight and an offset value defining the linear relationship between the peripheral pixel values of the block partitions of the multi-view color video frame and the corresponding peripheral pixel values of the block partitions of the depth map frame.

12. The method of claim 11, wherein, if the peripheral pixel values of the block partitions of the multi-view color video frame is Rec_Y′

, prediction values of the corresponding peripheral pixel values of the block partitions of the depth map frame is Pred_D, the weight is a, and the offset value is b, the correlation is defined as an equation;

Pred_—

D=a*Rec_—

Y′

+b.

13. The method of claim 8, wherein the obtaining prediction values of the corresponding block partitions of the depth map frame comprises:

obtaining the prediction values of the corresponding block partitions of the depth map frame by using block partitions of a restored multi-view color video frame having different color components; and

comparing costs of the prediction values obtained from the block partitions of the restored multi-view color video frame having different color components and determining block partitions of a multi-view color video frame to be used to predict the block partitions of the depth map frame.

14. The method of claim 8, wherein the block of the multi-view color video frame is one of a color video frame block having a same view as the depth map frame block, a color video frame block having a same view scaled to have a same size as that of the depth map frame block, peripheral blocks of a color video frame block having a same view as that of the depth map frame block, peripheral blocks of the color video frame block having a same view scaled to have a same size as that of the depth map frame block, a color video frame block having a different view from that of the depth map frame block determined based on a disparity vector indicating a difference between views of multi-view video, a color video frame block having a different view scaled to have a same size as that of the depth map frame block and determined based on the disparity vector, peripheral blocks of the color video frame block having a different view from that of the depth map frame block determined based on the disparity vector, and peripheral blocks of the color video frame block having a different view scaled to have a same size as that of the depth map frame block and determined based on the disparity vector.

15. An apparatus for decoding a depth map of multi-view video data, the apparatus comprising:

a receiver which is configured to receive a bitstream obtained by encoding a multi-view color video frame and a depth map frame corresponding to the multi-view color video frame;

a color video frame decoder which is configured to decode the encoded multi-view color video frame obtained from the bitstream;

a splitter which is configured to split a block of the decoded multi-view color video frame into at least one partition based on a pixel value of the block of the decoded multi-view color video frame;

a correlation parameter obtainer which is configured to obtain a parameter indicating a correlation between each of block partitions of the multi-view color video frame and each of block partitions of the depth map frame by using peripheral pixel values of the block partitions of the multi-view color video frame and peripheral pixel values of the block partitions of the depth map frame corresponding to the block partitions of the multi-view color video frame with respect to each of the block partitions of the decoded multi-view color video frame; and

a depth map decoder which is configured to obtain prediction values of the corresponding block partitions of the depth map frame from the block partitions of the decoded multi-view color video frame by using the obtained parameter.