Method of estimating disparity vector, and method and apparatus for encoding and decoding multi-view moving picture using the disparity vector estimation method

• US 20070064800A1
• Filed: 09/22/2006
• Published: 03/22/2007
• Est. Priority Date: 09/22/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Ha, Tae-hyeun
• Granted Patent: US 8,644,386 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/52 by predictive encoding
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and apparatus are provided for encoding and decoding a multi-view moving picture. A method of estimating a disparity vector to encode a multi-view moving picture includes: estimating disparity vectors for a predetermined number of encoded macroblocks; and calculating disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors. Therefore, it is possible to quickly perform encoding of a multi-view moving picture and enhance compressibility of the multi-view moving picture.

Claims

1. A method of estimating a disparity vector to encode a multi-view moving picture, the method comprising:

estimating disparity vectors for a plurality of encoded macroblocks; and

calculating disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors.

2. The method of claim 1, wherein the estimating the disparity vectors comprises estimating the disparity vectors for the encoded macroblocks using a full-search method or a fast-search method, and

the calculating the, disparity vectors comprises interpolating and calculating the disparity vectors of the macroblocks adjacent to the encoded macroblocks using an interpolation equation.

3. The method of claim 2, wherein the interpolation equation is selected according to a characteristic of the disparity vectors determined for the encoded macroblocks,

wherein the characteristic of the disparity vectors comprises information indicating whether deviation of the estimated disparity vectors is constant.

4. A multi-view moving picture encoding apparatus comprising:

a direct mode performing unit which receives a current frame and a reference frame, estimates disparity vectors for a plurality of encoded macroblocks, and calculates disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors;

an interpolation equation providing unit which provides an interpolation equation for calculating the disparity vectors of the macroblocks adjacent to the encoded macroblocks to the direct mode performing unit;

a search mode performing unit which receives the current frame and the reference frame and searches for the current frame and the reference frame to estimate disparity vectors;

a cost calculating unit which calculates a direct mode cost according to a disparity vector determining method performed by the direct mode performing unit and a search mode cost according to a disparity vector determining method performed by the search mode performing unit;

a comparator which compares the direct mode cost and the search mode cost with each other;

a mode selector which selects an encoding mode according to a comparison result provided by the comparator; and

an encoder which encodes a multi-view moving picture according to the selected encoding mode.

5. The apparatus of claim 4, wherein the direct mode performing unit estimates the disparity vectors for the encoded macroblocks using a full-search method or a fast-search method, and interpolates and calculates the disparity vectors of the macroblocks adjacent to the encoded macroblocks using an interpolation equation.

6. The apparatus of claim 5, wherein the interpolation equation providing unit stores at least one interpolation equation which can be selected according to a characteristic of disparity vectors estimated for the encoded macroblocks.

7. The apparatus of claim 4, wherein the cost calculating unit comprises:

a direct mode cost calculator which calculates the direct mode cost using a disparity vector DV_d calculated by the direct mode performing unit; and

a search mode cost calculator which calculates the search mode cost using a disparity vector DV_f estimated by the search mode performing unit.

8. The apparatus of claim 4, wherein the cost calculating unit calculates at least one of a bit rate and a peak signal to noise ratio for each of the direct mode and the search mode.

9. The apparatus of claim 7, wherein the comparator compares a difference between the direct mode cost and the search mode cost with a first threshold value, and

the mode selector sets a flag according to the comparison result and selects an encoding mode according to the set flag.

10. The apparatus of claim 9, wherein the comparator compares a difference between the disparity vector DV_d and the disparity vector DV_f, with a second predetermined threshold value, if the difference between the direct mode cost and the search mode cost is greater than the first threshold value, and

the mode selector sets the flag according to the comparison result and selects the encoding mode according to the set flag.

11. The apparatus of claim 4, wherein the encoder

encodes a residual picture, if the selected encoding mode is a direct mode,

encodes a residual picture and a difference between a disparity vector DV_d determined according to the direct mode and a disparity vector DV_f determined according to a search mode, if the selected encoding mode is the search mode and the difference between the disparity vector DV_d determined according to the direct mode and the disparity vector DV_f determined according to the search mode is smaller than a predetermined threshold value, and

encodes the residual picture and the disparity vector DV_f determined according to the search mode, if the selected encoding mode is the search mode and the difference between the disparity vector DV_d determined according to the direct mode and the disparity vector DV_f determined according to the search mode is greater than the predetermined threshold value.

12. A method of encoding a multi-view moving picture, the method comprising:

receiving a current frame and a reference frame;

performing a direct mode operation by estimating disparity vectors for a plurality of encoded macroblocks, and calculating disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors;

performing a search mode operation by searching for the current frame and the reference frame and estimating a disparity vector;

calculating a cost for the direct mode and a cost for the search mode;

comparing the cost for the direct mode with the cost for the search mode;

selecting an encoding mode according to a comparison result of the comparing; and

encoding a multi-view moving picture according to the selected encoding mode.

13. The method of claim 12, wherein the performing the direct mode comprises:

estimating the disparity vectors for the encoded macroblocks using a full-search method or a fast-search method; and

interpolating and calculating the disparity vectors of the macroblocks adjacent to the encoded macroblocks using a predetermined interpolation equation.

14. The method of claim 13, wherein interpolating and calculating the disparity vectors comprises calculating the disparity vectors using at least one equation which can be selected according to a characteristic of disparity vectors estimated for the encoded macroblocks.

15. The method of claim 12, wherein the calculating the cost for the direct mode and the cost for the search mode comprises:

calculating the cost for the direct mode using a disparity vector DV_d estimated according to the direct mode; and

calculating the cost for the search mode using a disparity vector DV_f estimated according to the search mode.

16. The method of claim 12, wherein the calculating the cost for the direct mode and the cost for the search mode comprises calculating a bit rate or a peak signal to noise ratio for the direct mode or the search mode.

17. The method of claim 15, wherein the comparing the cost for the direct mode with the cost for the search mode comprises comparing a difference between the cost for the direct mode and the cost for the search mode with a first threshold value, and

the selecting the encoding mode comprises setting a flag according to the result of the comparing, and selecting the encoding mode according to the set flag.

18. The method of claim 17, wherein the comparing the cost for the direct mode with the cost for the search mode comprises comparing a difference between a disparity vector DV_d determined according to the direct mode and a disparity vector DV_f determined according to the search mode, with a second threshold value, if the difference between the cost for the direct mode and the cost for the search mode is greater than the first threshold value, and

selecting the encoding mode comprises setting a flag according to the result of the comparing and selecting the encoding mode according to the set flag.

19. The method of claim 12, wherein the encoding the multi-view moving picture according to the selected encoding mode comprises:

encoding a residual picture if the selected encoding mode is the direct mode;

encoding the residual picture and a difference between a disparity vector DV_d determined according to the direct mode and a disparity vector DV_f determined according to the search mode if the selected encoding mode is the search mode and the difference between the disparity vector DV_d determined according to the direct mode and the disparity vector DV_f determined according to the search mode is smaller than a threshold value; and

encoding the residual picture and the disparity vector DV_f determined according to the search mode, if the selected encoding mode is the search mode and the difference between the disparity vector DV_d determined according to the direct mode and the disparity vector DV_f determined according to the search mode is greater than the threshold value.

20. An apparatus for decoding a multi-view moving picture, the apparatus comprising:

an encoding mode checking unit which checks a mode information indicating an encoding mode included in a received multi-view moving picture bitstream to determine the encoding mode;

a disparity vector determining unit which determines a disparity vector according to the encoding mode determined by the encoding mode checking unit; and

a decoder which decodes a multi-view moving picture, using a disparity vector determined according to the checked encoding mode and residual picture data included in the multi-view moving picture bitstream,

wherein the disparity vector determining unit calculates the disparity vector using a same disparity vector interpolation equation as that used to encode the received multi-view moving picture.

21. The apparatus of claim 20, wherein, if the encoding mode checked by the encoding mode checking unit is a first mode in which a residual picture is received, the disparity vector determining unit calculates the disparity vector using the disparity interpolation equation, and

if the encoding mode checked by the encoding mode checking unit is a second mode in which the residual picture and a difference between a disparity vector DV_d determined according to a direct mode and a disparity vector DV_f determined according to a search mode are transmitted, the disparity vector determining unit calculates the disparity vector by summing the difference and the disparity vector calculated using the disparity vector interpolation equation.

22. A method of decoding a multi-view moving picture the method comprising:

determining an encoding mode using a mode information indicating an encoding mode included in a received multi-view moving picture bitstream;

determining a disparity vector according to the encoding mode; and

decoding a multi-view moving picture, using the disparity vector determining according to the encoding mode and residual picture data included in the multi-view moving picture bitstream,

wherein the determining the disparity vector comprises calculating the disparity vector using a same disparity vector interpolation equation as that used to encode the received multi-view moving picture.

23. The method of claim 22, wherein the determining the disparity vector comprises:

if the encoding mode is a first mode in which a residual picture is received, calculating the disparity vector using the disparity vector interpolation equation; and

if the encoding mode is a second mode in which the residual picture and a difference between a disparity vector DV_d determined according to a direct mode and a disparity vector DV_f determined according to a search mode are transmitted, calculating the disparity vector by summing the difference and the disparity vector calculated using the disparity vector interpolation equation.

24. A computer-readable recording medium having embodied thereon a computer program for performing a method of estimating a disparity vector to encode a multi-view moving picture, the method comprising:

estimating disparity vectors for a plurality of encoded macroblocks; and

calculating disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors.

25. A computer-readable recording medium having embodied thereon a computer program for performing a method of encoding a multi-view moving picture, the method comprising:

receiving a current frame and a reference frame;

performing a direct mode operation by estimating disparity vectors for a plurality of encoded macroblocks, and calculating disparity vectors of macroblocks adjacent to the encoded macroblocks using the estimated disparity vectors;

performing a search mode operation by searching for the current frame and the reference frame and estimating a disparity vector;

calculating a cost for the direct mode and a cost for the search mode;

comparing the cost for the direct mode with the cost for the search mode;

selecting an encoding mode according to a result of the comparing; and

encoding a multi-view moving picture according to the selected encoding mode.

26. A computer-readable recording medium having embodied thereon a computer program for performing a method of decoding a multi-view moving picture, the method comprising:

determining an encoding mode using a mode information indicating an encoding mode included in a received multi-view moving picture bitstream;

determining a disparity vector according to the encoding mode; and

decoding a multi-view moving picture, using the disparity vector determining according to the encoding mode and residual picture data included in the multi-view moving picture bitstream,

wherein the determining the disparity vector comprises calculating the disparity vector using a same disparity vector interpolation equation as that used to encode the received multi-view moving picture.