DEPTH CODING

• US 20140184744A1
• Filed: 08/26/2011
• Published: 07/03/2014
• Est. Priority Date: 08/26/2011
• Status: Abandoned Application

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Liu, Shujie, Lai, Wang Lin, Tian, Dong
• Application Number:
  US14/239,918
• Publication Number:
  US 20140184744A1
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Various implementations address depth coding and related disciplines. In one particular implementation, a segmentation is determined for a particular portion of a video image in a sequence of video images. The segmentation is determined based on reference depth indicators that are associated with at least a portion of one video image in the sequence of video images. Target depth indicators associated with the particular portion of the video image are processed. The processing is based on the determined segmentation in the particular portion of the video image. In another particular implementation, a segmentation is determined for at least a given portion of a video image based on depth indicators associated with the given portion. The segmentation is extended from the given portion into a target portion of the video image based on pixel values in the given portion and on pixel values in the target portion.

Claims

1. A method comprising:

determining a segmentation for a particular portion of a video image in a sequence of video images based on reference depth indicators, the reference depth indicators being depth indicators associated with at least a portion of one video image in the sequence of video images; and

processing target depth indicators associated with the particular portion of the video image, the processing being based on the determined segmentation in the particular portion of the video image.

2. The method of claim 1 wherein the reference depth indicators comprise reconstructed depth indicators.

3. The method of claim 2 wherein:

the reconstructed depth indicators are associated with at least a portion of the video image, and

the reconstructed depth indicators are in a common image with the target depth indicators.

4. The method of claim 1 wherein determining the segmentation for the particular portion comprises:

determining a reference segmentation of the reference depth indicators; and

using the reference segmentation to determine the segmentation for the particular portion.

5. The method of claim 4 wherein:

the reference depth indicators comprise depth indicators neighboring the target depth indicators, and

using the reference segmentation to determine the segmentation for the particular portion comprises;

applying the reference segmentation to a neighbor portion of the video image that is a neighbor of the particular portion of the video image; and

growing the reference segmentation from the neighbor portion into the particular portion of the video image to produce a grown segmentation.

6. The method of claim 5 wherein growing the reference segmentation comprises:

selecting a given pixel from the particular portion;

determining a value of the given pixel;

determining a pixel from the neighbor portion;

determining a segment of the pixel from the neighbor portion; and

assigning the given pixel to the segment of the pixel from the neighbor portion.

7. The method of claim 6 wherein determining the pixel from the neighbor portion comprises determining a pixel from the neighbor portion that has a closest value, among all pixels in the neighbor portion, to the value of the given pixel.

8. The method of claim 5 wherein growing the reference segmentation comprises:

selecting a given pixel from the particular portion;

determining a metric value for the given pixel with respect to a specific segment of the reference segmentation, wherein the metric value is based on (i) distance from the given pixel to one or more pixels in the specific segment, and (ii) difference in pixel value between the given pixel and the one or more pixels in the specific segment; and

assigning the given pixel to the specific segment based on the determined metric value.

9. The method of claim 5 wherein processing the target depth indicators comprises applying the grown segmentation to the target depth indicators.

10. The method of claim 9 wherein processing the target depth indicators further comprises:

associating (i) a segment of the target depth indicators resulting from applying the grown segmentation to the target depth indicators with (ii) a segment of the reference segmentation of the reference depth indicators;

determining a representative value for the segment of the reference segmentation; and

using the representative value as a predictor for depth values in the segment of the target depth indicators.

11. The method of claim 10 wherein processing target depth indicators further comprises encoding the target depth indicators based on the representative value.

12. The method of claim 10 wherein processing target depth indicators further comprises decoding the target depth indicators based on the representative value.

13. The method of claim 10 wherein determining the representative depth value comprises determining at least one of a mean or a median for the segment of the reference segmentation.

14. The method of claim 1 wherein processing comprises determining one or more predictors for the target depth indicators based on the determined segmentation for the particular portion of the video image.

15. The method of claim 14 wherein determining the predictor based on the determined segmentation comprises:

applying the determined segmentation to the target depth indicators;

determining a representative depth value for a segment of the target depth indicators, the segment resulting from applying the determined segmentation to the target depth indicators; and

using the representative depth value as a predictor for depth values in the segment of the target depth indicators.

16. The method of claim 1 wherein processing target depth indicators comprises coding the target depth indicators based on the determined segmentation.

17. The method of claim 16 wherein coding the target depth indicators based on the determined segmentation comprises applying the determined segmentation to the target depth indicators.

18. The method of claim 17 wherein coding the target depth indicators based on the determined segmentation further comprises:

determining a representative depth value for a segment of the target depth indicators resulting from applying the determined segmentation to the target depth indicators;

using the representative depth value as a predictor for depth values in the segment of the target depth indicators;

determining a residue for the segment of the target depth indicators, based on the predictor; and

coding the residue.

19. The method of claim 1 wherein processing target depth indicators comprises decoding the target depth indicators based on the determined segmentation, to produce reconstructed target depth indicators.

20. The method of claim 19 wherein decoding the target depth indicators based on the determined segmentation comprises applying the determined segmentation to the target depth indicators.

21. The method of claim 20 wherein decoding the target depth indicators based on the determined segmentation further comprises:

determining a representative depth value for a segment of the target depth indicators resulting from applying the determined segmentation to the target depth indicators;

using the representative depth value as a predictor for depth values in the segment of the target depth indicators;

accessing a residue for the segment of the target depth indicators, based on the predictor, and

adding the residue to the predictor to produce the reconstructed target depth indicators.

22. An apparatus configured for performing at least the following operations:

determining a segmentation for a particular portion of a video image in a sequence of video images based on reference depth indicators, the reference depth indicators being depth indicators associated with at least one video image in the sequence of video images; and

processing target depth indicators associated with at least a portion of the particular portion of the video image, the processing being based on the determined segmentation in the particular portion of the video image.

23. The apparatus of claim 22 wherein the apparatus comprises one or more of an encoder or a decoder.

24. The apparatus of claim 22 wherein the apparatus comprises a processor configured to perform the operations of determining the segmentation and processing the target depth indicators.

25. An apparatus comprising:

means for determining a segmentation for a particular portion of a video image in a sequence of video images based on reference depth indicators, the reference depth indicators being depth indicators associated with at least a portion of one video image in the sequence of video images; and

means for processing target depth indicators associated with the particular portion of the video image, the processing being based on the determined segmentation in the particular portion of the video image.

26. A processor readable medium having stored thereon instructions for causing one or more processors to collectively perform:

determining a segmentation for a particular portion of a video image in a sequence of video images based on reference depth indicators, the reference depth indicators being depth indicators associated with at least a portion of one video image in the sequence of video images; and

processing target depth indicators that correspond to the particular portion of the video image, the processing being based on the determined segmentation in the particular portion of the video image.

27. A method comprising:

determining a segmentation for at least a given portion of a video image based on depth indicators associated with the given portion; and

extending the segmentation from the given portion into a target portion of the video image based on pixel values in the given portion and on pixel values in the target portion.

28. The method of claim 27 further comprising applying the extended segmentation to at least a target portion of a depth-indicator image, the target portion of the depth-indicator image including depth indicators associated with the target portion of the video image.

29. The method of claim 28 further comprising encoding the target portion of the depth-indicator image based on the applied segmentation.