METHOD AND DEVICE FOR GENERATING DEPTH IMAGE USING REFERENCE IMAGE, METHOD FOR ENCODING/DECODING DEPTH IMAGE, ENCODER OR DECODER FOR THE SAME, AND RECORDING MEDIUM RECORDING IMAGE GENERATED USING THE METHOD

• US 20090103616A1
• Filed: 10/18/2008
• Published: 04/23/2009
• Est. Priority Date: 10/19/2007
• Status: Abandoned Application

• Current Assignee: Gwangju Institute of Science and Technology, KT Corporation
• Original Assignee: Gwangju Institute of Science and Technology
• Inventors: Na, Sang Tae, Lee, Cheon, Ho, Yo Sung, Oh, Kwan Jung
• Application Number:
  US12/253,959
• Publication Number:
  US 20090103616A1
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

The present invention relates to a method and device for generating a depth image, a method for encoding/decoding the depth image, an encoder/decoder for the same, and a recording medium recording an image generated by the method, which are related to a depth image encoding method that can effectively reduce a bit generation rate using a reference image obtained by at least one camera and improve encoding efficiency. A depth image generating method according to an embodiment of the invention includes a step (a) of obtaining a depth image at a viewpoint and setting the obtained depth image to a reference image; a step (b) of applying a 3D warphing method to the reference image and predicting and generating a depth image at a specific viewpoint; and a step (c) of removing a hole that exists in the predicted and generated depth image.

Claims

1. A depth image generating method comprising:

a step (a) of obtaining a depth image at a viewpoint and setting the obtained depth image to a reference image;

a step (b) of applying a 3D warphing method to the reference image and predicting and generating a depth image at a specific viewpoint; and

a step (c) of removing a hole that exists in the predicted and generated depth image.

2. The depth image generating method of claim 1,

wherein, in the step (a), the reference image is down-sampled.

3. The depth image generating method of claim 1,

wherein the step (b) includes:

a step (b1) of projecting positions of pixel values existing in the reference image onto a three-dimensional space;

a step (b2) of reprojecting the projected position values on the three-dimensional space at predetermined positions of a target image; and

a step (b3) of transmitting the pixel values of the reference image to pixel positions of the target image corresponding to pixel positions of the reference image.

4. The depth image generating method of claim 1,

wherein, in the step (c), when one reference image exists, an intermediate value of available pixel values among the pixel values around the hole is applied to the hole so as to remove the hole.

5. The depth image generating method of claim 1,

wherein, in the step (c), when a plurality of reference images exist, a pixel value of a corresponding portion of another reference image is applied to a hole of a depth image that is predicted and generated from a specific reference image so as to remove the hole.

6. The depth image generating method of claim 5, further comprising:

when the hole is not removed from the predicted and generated depth image,

a step (c1) of applying an intermediate value of available pixel values among pixel values around the hole to the hole; and

a step (c2) of extracting the pixel value applied to the hole and applying the pixel value to the predicted and generated depth image.

21. A computer readable recording medium that stores the image implemented by the method of any one of claims 1 to 6.

7. A depth image generating device comprising:

a depth image storage unit that obtains a depth image at a viewpoint and stores the obtained depth image as a reference image;

a depth image prediction unit that applies a 3D warphing method to the reference image and predicts and generates a depth image at a specific viewpoint; and

a hole removing unit that removes a hole that exists in the depth image predicted and generated by the depth image prediction unit.

8. The depth image generating device of claim 7, further comprising:

a down-sampling unit that down-samples the reference image stored in the depth image storage unit.

9. The depth image generating device of claim 8,

wherein the depth image prediction unit projects positions of pixel values existing in the reference image onto a three-dimensional space, reprojects the projected position values on the three-dimensional space at predetermined positions of a target image, and transmits the pixel values of the reference image to pixel positions of the target image corresponding to pixel positions of the reference image, such that the depth image at the specific viewpoint is predicted and generated.

10. The depth image generating device of claim 7,

wherein, when one reference image exists, the hole removing unit applies an intermediate value of available pixel values among pixel values around the hole to the hole so as to remove the hole.

11. The depth image generating device of claim 7,

wherein, when a plurality of reference images exist, the hole removing unit applies a pixel value of a corresponding portion of another reference image to a hole of a depth image that is predicted and generated from a specific reference image so as to remove the hole.

12. The depth image generating device of claim 11,

wherein, when the hole is not removed from the predicted and generated depth image, the hole removing unit applies an intermediate value of available pixel values among pixel values around the hole to the hole, extracts the pixel value applied to the hole, and applies the pixel value to the predicted and generated depth image, such that the hole is removed.

13. An encoding method using a depth image at a specific viewpoint, the depth image being generated using the following steps:

a step (a) of obtaining a depth image at a viewpoint and setting the obtained depth image to a reference image;

a step (b) of applying a 3D warphing method to the reference image and predicting and generating the depth image at a specific viewpoint; and

a step (c) of removing a hole that exists in the predicted and generated depth image.

14. The encoding method of claim 13,

wherein the step (b) includes:

a step (b1) of projecting positions of pixel values existing in the reference image onto a three-dimensional space;

a step (b2) of reprojecting the projected position values on the three-dimensional space at predetermined positions of a target image; and

a step (b3) of transmitting the pixel values of the reference image to pixel positions of the target image corresponding to pixel positions of the reference image.

15. The encoding method of claim 13,

wherein, in the step (c), when one reference image exists, an intermediate value of available pixel values among pixel values around the hole is applied to the hole so as to remove the hole, and when a plurality of reference images exist, a pixel value of a corresponding portion of another reference image is applied to a hole of a depth image that is predicted and generated from a specific reference image so as to remove the hole.

19. A decoding method that decodes the image encoded by the method of any one of claims 13 to 15.

20. A decoder that decodes the image encoded by the method of any one of claims 13 to 15.

16. An encoder comprising:

an image prediction unit that performs inter-prediction and intra-prediction;

an image T/Q unit that transforms and quantizes a prediction sample that is obtained by the image prediction unit;

an entropy coding unit that encodes image data quantized by the image T/Q unit; and

a depth image generating unit that generates a depth image at a specific viewpoint by the image prediction unit,

wherein the depth image generating unit includes;

a depth image prediction unit that applies a 3D warphing method to a reference image using a depth image at a viewpoint as the reference image and predicts and generates a depth image at a specific viewpoint; and

a hole removing unit that removes a hole that exists in the depth image predicted and generated by the depth image prediction unit.

17. The encoder of claim 16,

wherein the depth image prediction unit projects positions of pixel values existing in the reference image onto a three-dimensional space, reprojects the projected position values on the three-dimensional space at predetermined positions of a target image, and transmits the pixel values of the reference image to pixel positions of the target image corresponding to pixel positions of the reference image, such that the depth image at the specific viewpoint is predicted and generated.

18. The depth image generating device of claim 16,

wherein, when one reference image exists, the hole removing unit applies an intermediate value of available pixel values among pixel values around the hole to the hole so as to remove the hole, and when a plurality of reference images exist, the hole removing unit applies a pixel value of a corresponding portion of another reference image to a hole of a depth image that is predicted and generated from a specific reference image so as to remove the hole, such that the depth image is generated.