System and method for depth-guided image filtering in a video conference environment

• US 8,682,087 B2
• Filed: 12/19/2011
• Issued: 03/25/2014
• Est. Priority Date: 12/19/2011
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Tian, Dihong
• Primary Examiner(s): Do, Anh
• Application Number:
  US13/329,943
• Publication Number:
  US 20130156332A1
• Time in Patent Office: 827 Days
• Field of Search: 382/232, 382/260, 382/103, 382/154, 382/159, 382/111, 348/169, 348/177, 348/E7.085, 345/506
• US Class Current: 382/232
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/182 the unit being a pixel
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/816 involving special video data, e.g 3D video
• CPC Class Codes: H04N 7/148 Interfacing a video terminal to a particular transmission medium, e.g. ISDN

Abstract

A method is provided in one example embodiment that includes receiving a plurality of depth values corresponding to pixels of an image; and filtering the image as a function of a plurality of variations in the depth values between adjacent pixels of a window associated with the image. In more detailed embodiments, the method may include encoding the image into a bit stream for transmission over a network. The filtering can account for a bit rate associated with the encoding of the image.

Claims

1. A method, comprising:

receiving, at a video processing unit having a processor, a plurality of depth values corresponding to pixels of an image; and

filtering, at the video processing unit, the image as a function of a plurality of variations in the depth values between adjacent pixels of a window associated with the image.

2. The method of claim 1, further comprising:

encoding the image into a bit stream for transmission over a network, wherein the filtering includes accounting for a bit rate associated with the encoding of the image.

3. The method of claim 1, further comprising:

receiving intensity values corresponding to the pixels, wherein the filtering is a function of variations in the intensity values between the adjacent pixels.

4. The method of claim 1, wherein filtering the image comprises smoothing certain adjacent pixels having variations of depth values below a threshold value.

5. The method of claim 1, wherein the image is filtered in a loop comprising an inverse transform, an inverse quantization, and a prediction compensation that is based on previous encoding.

6. The method of claim 1, wherein the window comprises pixels from a spatial region.

7. The method of claim 1, wherein the window comprises pixels from a temporal region.

8. The method of claim 1, wherein the filtering preserves pixels corresponding to depth values closer to a viewpoint over pixels corresponding to depth values further away from the viewpoint.

9. Logic encoded in one or more non-transitory media that includes code for execution and when executed by one or more processors is operable to perform operations comprising:

receiving a plurality of depth values corresponding to pixels of an image; and

filtering the image as a function of a plurality of variations in the depth values between adjacent pixels of a window associated with the image.

10. The logic of claim 9, the operations further comprising:

encoding the image into a bit stream for transmission over a network, wherein the filtering includes accounting for a bit rate associated with the encoding of the image.

11. The logic of claim 9, the operations further comprising:

receiving intensity values corresponding to the pixels, wherein the filtering is a function of variations in the intensity values between the adjacent pixels.

12. The logic of claim 9, wherein filtering the image comprises smoothing certain adjacent pixels having variations of depth values below a threshold value.

13. The logic of claim 9, wherein the image is filtered in a loop comprising an inverse transform, an inverse quantization, and a prediction compensation that is based on previous encoding.

14. The logic of claim 9, wherein the window comprises pixels from a spatial region.

15. The logic of claim 9, wherein the window comprises pixels from a temporal region.

16. The logic of claim 9, wherein the filtering preserves pixels corresponding to depth values closer to a viewpoint over pixels corresponding to depth values further away from the viewpoint.

17. An apparatus, comprising:

one or more processors;

a memory; and

a video encoder with a depth-guided filter, wherein the apparatus is configured for;

receiving a plurality of depth values corresponding to pixels of an image; and

filtering the image as a function of a plurality of variations in the depth values between adjacent pixels of a window associated with the image.

18. The apparatus of claim 17, the apparatus being further configured for:

encoding the image into a bit stream for transmission over a network, wherein the filtering includes accounting for a bit rate associated with the encoding of the image.

19. The apparatus of claim 17, the apparatus being further configured for:

receiving intensity values corresponding to the pixels, wherein the filtering is a function of variations in the intensity values between the adjacent pixels.

20. The apparatus of claim 17, wherein the filtering is configured to preserve pixels corresponding to depth values closer to a viewpoint over pixels corresponding to depth values further away from the viewpoint.