Temporal video filtering

• US 8,311,129 B2
• Filed: 12/15/2006
• Issued: 11/13/2012
• Est. Priority Date: 12/16/2005
• Status: Active Grant

• Current Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Original Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Inventors: Jenkins, Michael V.
• Primary Examiner(s): Czekaj, David
• Assistant Examiner(s): TORCHMAN, JONATHAN JOSHUA ELIE
• Application Number:
  US11/611,494
• Publication Number:
  US 20070139517A1
• Time in Patent Office: 2,160 Days
• Field of Search: 375/240.29, 375/240.12, 375/240.14
• US Class Current: 375/240.29
• CPC Class Codes: H04N 7/141 between two video terminals, e.g. videophone telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085

Abstract

In various embodiments, temporal filtering may be used to reduce noise over time in a series of video frames. A temporal filter may use previous filtered frames to reduce the noise in a current frame. For example, a temporal filter may filter noise in still parts of an image more strongly than in moving parts of the image. In some embodiments, a temporal filter requiring less processing power may include a method for detecting motion between a set of pixels and applying the temporal filter based on the detected motion. For example, if substantial motion is not detected, the temporal filter for the block of pixels may be stronger (i.e., reduce more noise) than the temporal filter applied to a block of pixels for which substantial motion has been detected.

Claims

1. A method for performing temporal filtering of video frames, comprising:

storing a first video frame at a first time;

receiving a second video frame at a second time, wherein the second time is after the first time;

determining a difference between a first pixel of the first video frame and a corresponding second pixel of the second video frame;

generating a filtered second video frame based on said determining the difference between the first pixel and the corresponding second pixel, wherein said generating comprises;

if the difference is within a first range of thresholds, adding a first offset to the first pixel to form a filtered pixel;

if the difference is not within the first range of thresholds, and the difference is within a second range of thresholds, adding a second offset to the first pixel to form the filtered pixel, wherein the second offset is greater than the first offset, wherein the second range of thresholds is greater than the first range of thresholds; and

if the difference is greater than the first range of thresholds and the second range of thresholds, adding a value approximately equal to the difference to the first pixel to form the filtered pixel;

providing the filtered second video frame for display.

2. The method of claim 1, wherein the first and second video frames are comprised in a video, wherein the first video frame is a video frame prior to the second video frame in the video.

3. The method of claim 1, wherein the first video frame is a previous filtered frame and wherein the second video frame is a new video frame.

4. The method of claim 1, wherein the first offset is approximately 0.

5. The method of claim 1, wherein the second offset is determined using a look-up table based on the difference.

6. The method of claim 1, wherein the first range of thresholds and the second range of thresholds is approximately centered on 0.

7. The method of claim 1, wherein the first pixel is an ten bit pixel and wherein the first range of thresholds extends from approximately a range of −

5 to +5.

8. The method of claim 1, wherein the first pixel is an ten bit pixel and wherein the second range of thresholds extends from approximately a range of −

5 to −

32 and approximately a range of 5 to 32.

9. The method of claim 1, wherein said generating the filtered second video frame is based on performing said determining for a plurality of pixels of the first video frame and a corresponding plurality of pixels of the second video frame.

10. The method of claim 1, wherein said providing the filtered second video frame for display comprises transmitting the filtered second video frame over a network.

20. The method of claim 1, wherein said providing the filtered second video frame for display comprises transmitting the filtered second video frame over a network.

11. A non-transitory computer accessible memory medium storing program instructions for performing temporal filtering of video frames, wherein the program instructions are executable by a processor to:

store a first video frame at a first time;

receive a second video frame at a second time, wherein the second time is after the first time;

determine a difference between a first pixel of the first video frame and a corresponding second pixel of the second video frame;

generate a filtered second video frame based on said determining the difference between the first pixel and the corresponding second pixel, wherein said generating comprises;

if the difference is within a first range of thresholds, adding a first offset to the first pixel to form a filtered pixel;

if the difference is not within the first range of thresholds, and the difference is within a second range of thresholds, adding a second offset to the first pixel to form the filtered pixel, wherein the second offset is greater than the first offset, wherein the second range of thresholds is greater than the first range of thresholds; and

if the difference is greater than the first range of thresholds and the second range of thresholds, adding a value approximately equal to the difference to the first pixel to form the filtered pixel;

provide the filtered second video frame for display.

12. The non-transitory computer accessible memory medium of claim 11, wherein the first and second video frames are comprised in a video, wherein the first video frame is a video frame prior to the second video frame in the video.

13. The non-transitory computer accessible memory medium of claim 11, wherein the first video frame is a previous filtered frame and wherein the second video frame is a new video frame.

14. The non-transitory computer accessible memory medium of claim 11, wherein the first offset is approximately 0.

15. The non-transitory computer accessible memory medium of claim 11, wherein the second offset is determined using a look-up table based on the difference.

16. The non-transitory computer accessible memory medium of claim 11, wherein the first range of thresholds and the second range of thresholds is approximately centered on 0.

17. The non-transitory computer accessible memory medium of claim 11, wherein the first pixel is an ten bit pixel and wherein the first range of thresholds extends from approximately a range of −

5 to +5.

18. The non-transitory computer accessible memory medium of claim 11, wherein the first pixel is an ten bit pixel and wherein the second range of thresholds extends from approximately a range of −

5 to −

32 and approximately a range of 5 to 32.

19. The non-transitory computer accessible memory medium of claim 11, wherein said generating the filtered second video frame is based on performing said determining for a plurality of pixels of the first video frame and a corresponding plurality of pixels of the second video frame.

21. A video conferencing system, comprising:

a video display;

a video codec coupled to the video display, wherein the video codec is configured to send/receive video frames to/from a remote video conferencing system, wherein the video codec comprises a processor and memory;

wherein the memory is configured to store program instructions executable by the processor to;

store a first video frame at a first time;

receive a second video frame at a second time, wherein the second time is after the first time;

determine a difference between a first pixel of the first video frame and a corresponding second pixel of the second video frame;

generate a filtered second video frame based on said determining the difference between the first pixel and the corresponding second pixel, wherein said generating comprises;

if the difference is within a first range of thresholds, adding a first offset to the first pixel to form a filtered pixel;

if the difference is not within the first range of thresholds, and the difference is within a second range of thresholds, adding a second offset to the first pixel to form the filtered pixel, wherein the second offset is greater than the first offset, wherein the second range of thresholds is greater than the first range of thresholds; and

if the difference is greater than the first range of thresholds and the second range of thresholds, adding a value approximately equal to the difference to the first pixel to form the filtered pixel;

provide the filtered second video frame for display.

22. The video conferencing system of claim 21, wherein the first offset is approximately 0.

23. The video conferencing system of claim 21, wherein the second offset is determined using a non-linear function based on the difference.

24. The video conferencing system of claim 21, wherein the first and second video frames are comprised in a video, wherein the first video frame is a video frame prior to the second video frame in the video.

25. The video conferencing system of claim 21, wherein said providing the filtered second video frame for display comprises transmitting the filtered second video frame over a network.

26. The video conferencing system of claim 21, wherein said providing the filtered second video frame for display comprises providing the filtered second video frame to the video display.

27. A non-transitory computer accessible memory medium storing program instructions for performing temporal filtering of video frames, wherein the program instructions are executable by a processor to:

store a first video frame at a first time;

receive a second video frame at a second time, wherein the second time is after the first time;

determine a difference between a first pixel of the first video frame and a corresponding second pixel of the second video frame;

generate a filtered second video frame based on said determining the difference between the first pixel and the corresponding second pixel, wherein said generating comprises;

if the difference is within a first range of thresholds, adding a first offset to the first pixel to form a filtered pixel;

if the difference is not within the first range of thresholds, and the difference is within a second range of thresholds, adding a second offset to the first pixel to form the filtered pixel, wherein the second offset is greater than the first offset, wherein the second range of thresholds is greater than the first range of thresholds; and

if the difference is greater than the first range of thresholds and the second range of thresholds, using the second pixel without filtering;

provide the filtered second video frame for display.

28. The memory medium of claim 27, wherein the first video frame is a weighted average of a plurality of video frames.