System and method for depth-guided filtering in a video conference environment

• US 9,681,154 B2
• Filed: 12/06/2012
• Issued: 06/13/2017
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Patent Capital Group
• Inventors: Tian, Dihong
• Primary Examiner(s): Dang, Hung
• Application Number:
  US13/707,377
• Publication Number:
  US 20140160239A1
• Time in Patent Office: 1,650 Days
• Field of Search: None
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 21/234327 by decomposing into layers, e.g. base layer and one or more enhancement layers
• CPC Class Codes: H04N 7/152 Multipoint control units therefor

Abstract

A method is provided in one example embodiment that includes generating a depth map that corresponds to a video image and filtering the depth map with the video image to create a filtered depth map. The video image can be filtered with the filtered depth map to create an image. In one example implementation, the video image is filtered using extended depth-guided filtering that is incorporated into a video encoding-decoding loop.

Claims

1. A method, comprising:

generating a depth map that corresponds to a video image;

filtering the depth map with the video image to create a filtered depth map, wherein filtering the depth map includes filling in missing data from the depth map and upsampling when the depth map has a lower resolution than the video image, wherein the video image is a higher resolution image than the depth map, wherein details of objects closer to a viewpoint are preserved preferentially over objects further away;

using the video image to create a pyramid of video images; and

using the depth map to create a pyramid of depth maps, wherein the pyramid of video images are filtered with the pyramid of depth maps, layered into multiple bit streams, and cascaded to create an image.

2. The method of claim 1, wherein the filtering the depth map and filtering the video image are performed sequentially.

3. The method of claim 1, further comprising:

using extended depth-guided filtering that is incorporated into a video encoding-decoding loop, and wherein filtering the depth map includes removing noise from the depth map.

4. The method of claim 1, wherein zero-mean Gaussian function with standard deviations are used for filtering the depth map.

5. The method of claim 1, further comprising:

encoding the multiple bit streams for transmission over a network.

6. The method of claim 5, wherein encoding the multiple bit streams includes using a deblocking filter in H.264.

7. Logic encoded in non-transitory media that includes instructions for execution and when executed by a processor, is operable to perform operations comprising:

generating a depth map that corresponds to a video image, wherein the video image is filtered;

filtering the depth map with the video image to create a filtered depth map, wherein filtering the depth map includes filling in missing data from the depth map and upsampling when the depth map has a lower resolution than the video image, wherein the video image is a higher resolution image than the depth map, wherein details of objects closer to a viewpoint are preserved preferentially over objects further away;

using the video image to create a pyramid of video images; and

using the depth map to create a pyramid of depth maps, wherein the pyramid of video images are filtered with the pyramid of depth maps, layered into multiple bit streams, and cascaded to create an image.

8. The logic of claim 7, the operations further comprising:

encoding the multiple bit streams for transmission over a network.

9. An apparatus, comprising:

a memory element for storing data;

a processor that executes instructions associated with the data;

a manager module configured to interface with the processor and the memory element such that the apparatus is configured to;

generate a depth map that corresponds to a video image, wherein the video image is filtered;

filter the depth map with the video image to create a filtered depth map, wherein filtering the depth map includes filling in missing data from the depth map and upsampling when the depth map has a lower resolution than the video image, wherein the video image is a higher resolution image than the depth map, wherein details of objects closer to a viewpoint are preserved preferentially over objects further away;

using the video image to create a pyramid of video images; and

using the depth map to create a pyramid of depth maps, wherein the pyramid of video images are filtered with the pyramid of depth maps, layered into multiple bit streams, and cascaded to create an image.