Depth Map Coding

• US 20130002816A1
• Filed: 12/29/2011
• Published: 01/03/2013
• Est. Priority Date: 12/29/2010
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: HANNUKSELA, Miska Matias, RUSANOVSKYY, Dmytro
• Granted Patent: US 9,398,313 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/59 involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

The invention relates to coding of depth information for multi-view video coding. Different parameters and/or any features from picture encoding or the encoded and reconstructed pictures may be used in the coding of the depth information, especially in filtering the depth picture using e.g. a loop filter in the depth coding loop. The same principle may be applied in decoding, that is, the decoded (texture) pictures and parameters may be used to control the decoding of the depth data, e.g. to control the filtering of the depth data in a loop filter. Parameters and data that may be used as such control may comprise features extracted from the reconstructed pictures, the encoded video data and parameters, the motion estimation data and others.

Claims

1. A method for encoding picture data, the method comprising:

encoding at least one texture element of said picture data to obtain at least one encoded texture element,

forming at least one texture encoding parameter in said encoding,

encoding at least one depth element related to said at least one texture element to obtain at least one encoded depth element,

applying filtering to depth data in said encoding of at least one depth element, and

controlling said filtering using said at least one texture encoding parameter.

2. A method according to claim 1, further comprising:

encoding said at least one texture element in a texture coding loop,

encoding said at least one depth element in a depth coding loop,

applying said filtering to depth data in a loop filter of said depth coding loop, and

forming said coded texture element and said coded depth element into at least one bitstream.

3. A method according to claim 1, further comprising:

reconstructing encoded depth data of at least one other depth element, and

using said reconstructed depth data as a prediction source in encoding of said at least one depth element.

4. A method according to claim 1, further comprising:

reconstructing at least part of an image using said at least one encoded texture element,

extracting a feature from said at least part of an image using image feature extraction, and

controlling said filtering using said feature.

5. A method according to claim 1, wherein said at least one texture encoding parameter is indicative of at least one of the group of an encoding process, a filtering process, a prediction process, a transform process, a quantization process, a feature of a picture, a coded texture element and a non-coded texture element.

6. A method according to claim 1, the method further comprising

increasing a sample count of the at least one depth element by resampling in said filtering to obtain at least one upsampled depth element, and

encoding said at least one upsampled depth element to obtain at least one encoded depth element.

7. An apparatus comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

encode at least one texture element of said picture data to obtain at least one encoded texture element,

form at least one texture encoding parameter in said encoding,

encode at least one depth element related to said texture element to obtain at least one encoded depth element,

apply filtering to depth data in said encoding of at least one depth element, and

control said filtering using said at least one texture encoding parameter.

8. An apparatus according to claim 7, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

encode said at least one texture element in a texture coding loop,

encode said at least one depth element in a depth coding loop,

apply said filtering to depth data in a loop filter of said depth coding loop, and

form said coded texture element and said coded depth element into at least one bitstream.

9. An apparatus according to claim 7, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

reconstruct encoded depth data of at least one other depth element, and

use said reconstructed depth data as a prediction source in encoding of said at least one depth element.

10. An apparatus according to claim 7, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

reconstruct at least part of an image using said at least one encoded texture element,

extract a feature from said at least part of an image using image feature extraction, and

control said filtering using said feature.

11. An apparatus according to claim 7, wherein said at least one texture encoding parameter is indicative of at least one of the group of an encoding process, a filtering process, a prediction process, a transform process, a quantization process, a feature of a picture, a coded texture element and a non-coded texture element.

12. An apparatus according to claim 7, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

increase a sample count of the at least one depth element by resampling in said filtering to obtain at least one upsampled depth element, and

encode said at least one upsampled depth element to obtain at least one encoded depth element.

13. A method for decoding encoded depth data, the method comprising:

decoding at least one encoded texture element from encoded texture data,

forming at least one texture encoding parameter in said decoding,

decoding at least one depth element related to said at least one texture element from said encoded depth data to obtain at least one decoded depth element,

applying filtering to said at least one decoded depth element in said decoding of at least one depth element, and

controlling said filtering using said at least one texture encoding parameter.

14. A method according to claim 13, further comprising:

receiving said encoded texture data and said encoded depth data in at least one bitstream,

decoding said at least one texture element in a texture coding loop,

decoding said at least one depth element in a depth coding loop, and

applying said filtering to said depth element in a loop filter of said depth coding loop.

15. A method according to claim 13, further comprising:

reconstructing encoded depth data of at least one other depth element, and

using said reconstructed depth data as a prediction source in decoding of said at least one depth element.

16. A method according to claim 13, further comprising:

reconstructing at least part of an image using said at least one texture element,

extracting a feature from said at least part of an image using image feature extraction, and

controlling said filtering using said feature.

17. A method according to claim 13, wherein said at least one texture encoding parameter is indicative of at least one of the group of a decoding process, a filtering process, a prediction process, a transform process, a quantization process, a feature of a picture, a coded texture element and a non-coded texture element.

18. A method according to claim 13, the method further comprising

increasing a sample count of at least one depth element by resampling in said filtering to obtain at least one upsampled depth element.

19. An apparatus comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least:

decode at least one encoded texture element from encoded texture data,

form at least one texture encoding parameter in said decoding,

decode at least one depth element related to said at least one texture element from said encoded depth data to obtain at least one decoded depth element,

filter said at least one decoded depth element in said decoding of at least one depth element, and

control said filtering using said at least one texture encoding parameter.

20. An apparatus according to claim 19, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

receive said encoded texture data and said encoded depth data in at least one bitstream,

decode said at least one texture element in a texture coding loop,

decode said at least one depth element in a depth coding loop, and

apply said filtering to said depth element in a loop filter of said depth coding loop.

21. An apparatus according to claim 19, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

reconstruct encoded depth data of at least one other depth element, and

use said reconstructed depth data as a prediction source in decoding of said at least one depth element.

22. An apparatus according to claim 19, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

reconstruct at least part of an image using said at least one texture element,

extract a feature from said at least part of an image using image feature extraction, and

control said filtering using said feature.

23. An apparatus according to claim 19, wherein said at least one texture encoding parameter is indicative of at least one of the group of an encoding process, a filtering process, a prediction process, a transform process, a quantization process, a feature of a picture, a coded texture element and a non-coded texture element.

24. An apparatus according to claim 19, further comprising computer program code configured to, with the at least one processor, cause the apparatus to at least:

increase a sample count of at least one depth element by resampling in said filtering to obtain at least one upsampled depth element.

25. A method for coding picture data, the method comprising:

coding at least one texture element of said picture data,

forming at least one texture coding parameter in said coding,

coding at least one depth element related to said at least one texture element,

applying filtering to depth data in said coding of at least one depth element, and

controlling said filtering using said at least one texture coding parameter.

26. A computer program embodied on a non-transitory computer readable medium, the computer program comprising instructions causing, when executed on at least one processor, at least one apparatus to:

code at least one texture element of said picture data,

form at least one texture coding parameter in said coding,

code at least one depth element related to said at least one texture element,

apply filtering to depth data in said coding of at least one depth element, and

control said filtering using said at least one texture coding parameter.

27. A system comprising at least one processor, memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the system to at least:

code at least one texture element of said picture data,

form at least one texture coding parameter in said coding,

code at least one depth element related to said at least one texture element,

apply filtering to depth data in said coding of at least one depth element, and

control said filtering using said at least one texture coding parameter.

28. An apparatus comprising:

means for encoding at least one texture element of said picture data to obtain at least one encoded texture element,

means for forming at least one texture encoding parameter in said encoding,

means for encoding at least one depth element related to said texture element to obtain at least one encoded depth element,

means for filtering depth data in said encoding of at least one depth element, and

means for controlling said filtering using said at least one texture encoding parameter.

29. An apparatus comprising:

means for decoding at least one encoded texture element from encoded texture data,

means for forming at least one texture encoding parameter in said decoding,

means for decoding at least one depth element related to said at least one texture element from said encoded depth data to obtain at least one decoded depth element,

means for filtering said at least one decoded depth element in said decoding of at least one depth element, and

means for controlling said filtering using said at least one texture encoding parameter.