METHOD, APPARATUS AND COMPUTER PROGRAM PRODUCT FOR FILTERING OF MEDIA CONTENT

• US 20150269736A1
• Filed: 03/18/2015
• Published: 09/24/2015
• Est. Priority Date: 03/20/2014
• Status: Abandoned Application

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: HANNUKSELA, Miska Matias, AFLAKI BENI, Payman
• Application Number:
  US14/661,792
• Publication Number:
  US 20150269736A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/395 with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

In an example embodiment, a method, apparatus and computer program product are provided. The method includes defining a plurality of depth layers of a depth map. At least one depth layer of the plurality of depth layers is associated with a respective depth limit. The method further includes determining, for the at least one depth layer, a respective texture view layer of a first picture. The method further includes deriving a measure of a respective texture property for the respective texture view layer. Selective filtering is applied to the respective texture view layer based on the measure of the respective texture property and the respective depth limit associated with the at least one depth layer.

Claims

1. A method comprising:

defining a plurality of depth layers of a depth map, at least one depth layer of the plurality of depth layers being associated with a respective depth limit;

determining, for the at least one depth layer, a respective texture view layer of a first texture picture;

deriving, for the respective texture view layer, a measure of a respective texture property; and

applying selective filtering to the respective texture view layer based on the measure of the respective texture property and the respective depth limit associated with the at least one depth layer.

2. The method as claimed in claim 1, wherein defining the plurality of depth layers of the depth map comprises:

determining the respective depth limit for the at least one depth layer of the plurality of depth layers based on at least one depth criteria; and

segmenting the depth map into the plurality of depth layers based at least on the respective depth limit associated with the at least one depth layer of the plurality of depth layers.

3. The method as claimed in claim 2, wherein the at least one depth criteria comprises at least one of a differential value of depth, mean value of depth, median value of depth, and histogram based value of depth.

4. The method as claimed in claim 3, wherein defining the plurality of depth layers of the depth map comprises:

determining depth interval of the depth map based on the differential value of depth, the differential value of depth being determined based on a minimum depth value and a maximum depth value associated with the depth map; and

partitioning at least a portion of the depth interval into a plurality of intervals to define the plurality of depth layers.

5. The method as claimed in claim 3, wherein defining the plurality of depth layers of the depth map comprises:

determining depth interval of the depth map based on a minimum depth value and a maximum depth value associated with the depth map;

determining the mean value of depth of the depth map based on depth values associated with the depth map;

partitioning at least one portion of the depth interval into multiple depth intervals based on the mean value of depth; and

partitioning each of the multiple depth intervals into a plurality of intervals to define the plurality of depth layers.

6. The method as claimed in claim 3, wherein defining the plurality of depth layers of the depth map comprises:

determining depth interval of the depth map based on a minimum depth value and a maximum depth value associated with the depth map;

determining the median value of depth of the depth map based on depth values associated with the depth map;

partitioning at least one portion of the depth interval into multiple depth intervals based on the median value of depth; and

partitioning each of the multiple depth intervals into a plurality of intervals to define the plurality of depth layers.

7. The method as claimed in claim 3, wherein defining the plurality of depth layers of the depth map comprises:

determining a histogram of depth values associated with the depth map, the histogram being representative of a number of pixels associated with a plurality of depth values of the depth map; and

partitioning a depth interval of the depth map into a plurality of intervals comprising substantially equal number of pixels, the plurality of intervals being associated with the plurality of depth layers of the depth map.

8. The method as claimed in claim 1, further comprising receiving a second texture picture, the first texture picture and the second texture picture jointly representing a stereoscopic picture.

9. The method as claimed in claim 8, further comprising:

encoding the second texture picture in a base layer of a scalable bitstream, the second texture picture being generated on applying the selective filtering to the respective texture view layer associated with the first texture picture; and

encoding the first texture picture in an enhancement layer of the scalable bitstream.

10. The method as claimed in claim 9, further comprising:

decoding the encoded second texture picture from the base layer of the scalable bitstream; and

decoding the encoded first texture picture from the enhancement layer of the scalable bitstream.

11. The method as claimed in claim 9, further comprising applying prediction from the encoded second texture picture in at least one of the encoding and the decoding of the first texture picture.

12. The method as claimed in claim 8, further comprising filtering the second texture picture asymmetrically with the filtering of the respective texture view layer of the first texture picture.

13. The method as claimed in claim 1, wherein applying selective filtering to the respective texture view layer comprises manipulating a filter strength for performing filtering of the respective texture view layer based on the measure of the respective texture property and the respective depth limit associated with the at least one depth layer.

14. An apparatus comprising:

at least one processor; and

at least one memory comprising computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform;

define a plurality of depth layers of a depth map, at least one depth layer of the plurality of depth layers being associated with a respective depth limit;

determine, for the at least one depth layer, a respective texture view layer of a first texture picture;

derive, for the respective texture view layer, a measure of a respective texture property; and

apply selective filtering to the respective texture view layer based on the measure of the respective texture property and the respective depth limit associated with the at least one depth layer.

15. The apparatus as claimed in claim 14, wherein for defining the plurality of depth layers of the depth map, the apparatus is further caused, at least in part to:

determine the respective depth limit for the at least one depth layer of the plurality of depth layers based on at least one depth criteria; and

segment the depth map into the plurality of depth layers based at least on the respective depth limit associated with the at least one depth layer of the plurality of depth layers.

16. The apparatus as claimed in claim 15, wherein the at least one depth criteria comprises at least one of a differential value of depth, mean value of depth, median value of depth, and histogram based value of depth.

17. The apparatus as claimed in claim 16, wherein for defining the plurality of depth layers of the depth map, the apparatus is further caused, at least in part to:

determine depth interval of the depth map based on the differential value of depth, the differential value of depth being determined based on a minimum depth value and a maximum depth value associated with the depth map; and

partition at least a portion of the depth interval into a plurality of intervals to define the plurality of depth layers.

18. The apparatus as claimed in claim 16, wherein for defining the plurality of depth layers of the depth map, the apparatus is further caused, at least in part to:

determine depth interval of the depth map based on a minimum depth value and a maximum depth value associated with the depth map;

determine the mean value of depth of the depth map based on depth values associated with the depth map;

partition at least one portion of the depth interval into multiple depth intervals based on the mean value of depth; and

partition each of the multiple depth intervals into a plurality of intervals to define the plurality of depth layers.

19. The apparatus as claimed in claim 16, wherein for defining the plurality of depth layers of the depth map, the apparatus is further caused, at least in part to:

determine depth interval of the depth map based on a minimum depth value and a maximum depth value associated with the depth map;

determine the median value of depth of the depth map based on depth values associated with the depth map;

partition at least one portion of the depth interval into multiple depth intervals based on the median value of depth; and

partition each of the multiple depth intervals into a plurality of intervals to define the plurality of depth layers.

20. The apparatus as claimed in claim 16, wherein for defining the plurality of depth layers of the depth map, the apparatus is further caused, at least in part to:

determine a histogram of depth values associated with the depth map, the histogram being representative of a number of pixels associated with a plurality of depth values of the depth map; and

partition a depth interval of the depth map into a plurality of intervals comprising substantially equal number of pixels, the plurality of intervals being associated with the plurality of depth layers of the depth map.

21. The apparatus as claimed in claim 14, wherein the apparatus is further caused, at least in part to receive a second texture picture, the first texture picture and the second texture picture jointly representing a stereoscopic picture.

22. The apparatus as claimed in claim 21, wherein the apparatus is further caused, at least in part to:

encode the second texture picture in a base layer of a scalable bitstream, the second being generated on applying the selective filtering to the respective texture view layer associated with the first texture picture; and

encode the first texture picture from an enhancement layer of the scalable bitstream.

23. The apparatus as claimed in claim 22, wherein the apparatus is further caused, at least in part to:

decode the encoded second texture picture from the base layer of the scalable bitstream; and

decode the encoded first texture picture in the enhancement layer of the scalable bitstream.

24. The apparatus as claimed in claim 22, wherein the apparatus is further caused, at least in part to apply prediction from the second texture picture in at least one of the encoding and the decoding of the first texture picture.

25. The apparatus as claimed in claim 21, wherein the apparatus is further caused, at least in part to filter the second texture picture asymmetrically with the filtering of the respective texture view layer of the first texture picture.

26. The apparatus as claimed in claim 22, wherein for applying selective filtering to the respective texture view layer, the apparatus is further caused, at least in part to manipulate a filter strength for performing filtering of the respective texture view layer based on the measure of the respective texture property and the respective depth limit associated with the at least one depth layer.