Video preprocessing method and apparatus with selective filtering based on motion detection

• US 6,037,986 A
• Filed: 07/16/1996
• Issued: 03/14/2000
• Est. Priority Date: 07/16/1996
• Status: Expired due to Term

• Current Assignee: Harmonic Incorporated
• Original Assignee: DiviCom, Inc (Harmonic Incorporated)
• Inventors: Gouliardon, Cedric, Zhang, Ji
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US08/683,546
• Time in Patent Office: 1,337 Days
• Field of Search: 348/409, 348/405, 348/411, 348/410, 348/413, 348/415, 348/416, 348/420, 348/421, 348/422, 348/451, 348/448, 348/699, 348/700, 348/701, 348/384, 348/385
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

Video preprocessing methods and apparatus which utilize motion detection to control selective filtering of pels in a sequence of video frames wherein a video preprocessor generates first and second motion metrics for a given pel in a current frame N by taking the difference between the given pel and corresponding pels in a previous frame N-1 and a subsequent frame N+1, respectively. The motion metrics are converted to first and second bitmaps by thresholding the motion metric value against a first threshold. A motion detection metric is then generated for a given pel by summing the values of the first and second bitmaps for a group of pels which includes the given pel, and comparing the result to a predetermined threshold. The group of pels may include five pels on each of two lines above the given pel, and five pels on each of two lines below a given pel. The motion detection metric may be used to determine whether or not temporal median filtering and/or temporal lowpass filtering operations should be applied to the given pel. The motion detection metric may also be used in conjunction with an edge detection metric and a frame-wise motion activity measure to generate an address into a look-up table. The look-up table specifies a set of filter coefficients for use in a spatial lowpass filtering operation applied to the given pel.

Claims

1. A method of processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the method comprising the steps of:

generating a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, and a second motion metric as a difference between the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence;

generating first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold, and another value for the given pel otherwise; and

generating a motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise.

2. The method of claim 1 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.

3. The method of claim 1 wherein the group of pels for which the first and second bitmap values are summed further includes pels on two lines above a line containing the given pel and two lines below the line containing the given pel.

4. The method of claim 1 wherein the group of pels for which the first and second bitmap values are summed further includes five pels on each of two lines above a line containing the given pel, and five pels on each of two lines below the line containing the given pel.

5. The method of claim 1 further including the step of utilizing the motion detection metric to determine if temporal median filtering should be applied to the given pel.

6. The method of claim 1 further including the step of utilizing the motion detection metric to determine if temporal lowpass filtering should be applied to the given pel.

7. The method of claim 1 further including the step of utilizing the motion detection metric to determine if spatial lowpass filtering should be applied to the given pel.

8. The method of claim 1 further including the step of utilizing the motion detection metric to determine an amount of spatial lowpass filtering to be applied to the given pel.

9. The method of claim 8 wherein the step of utilizing the motion detection metric to determine an amount of spatial lowpass filtering to be applied to the given pel further includes the step of utilizing the motion detection metric to generate an address into a look-up table including a plurality of entries each specifying a set of spatial lowpass filter coefficients.

10. An apparatus for processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the apparatus comprising:

a memory for storing one or more frames of the frame sequence; and

a motion detector coupled to the memory and operative to generate a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, to generate a second motion metric as a difference between the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence, to generate first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold, and another value for the given pel otherwise, and to generate a motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise.

11. The apparatus of claim 10 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.

12. The apparatus of claim 10 wherein the group of pels for which the first and second bitmap values are summed further includes pels on two lines above a line containing the given pel and two lines below the line containing the given pel.

13. The apparatus of claim 10 wherein the group of pels for which the first and second bitmap values are summed further includes five pels on each of two lines above a line containing the given pel, and five pels on each of two lines below the line containing the given pel.

14. The apparatus of claim 10 further including a temporal median filter coupled to the motion detector and having an input for receiving the motion detection metric therefrom, such that the value of the motion detection metric determines if temporal median filtering will be applied to the given pel.

15. The apparatus of claim 10 further including a temporal lowpass filter coupled to the motion detector and having an input for receiving the motion detection metric therefrom, such that the value of the motion detection metric determines whether temporal lowpass filtering will be applied to the given pel.

16. The apparatus of claim 10 further including a spatial lowpass filter coupled to the motion detector such that the value of the motion detection metric determines whether spatial lowpass filtering will be applied to the given pel.

17. The apparatus of claim 16 further including a look-up table coupled between the motion detector and the spatial lowpass filter, wherein the motion detection metric is used to generate an address into the look-up table to thereby identify an entry in the table specifying a set of spatial lowpass filter coefficients.

18. A method of processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the method comprising the steps of:

generating a motion detection metric for a given pel of a frame in the frame sequence, wherein said step of generating comprises the steps of;

generating a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, and a second motion metric as a difference between the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence;

generating first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold, and another value for the given pel otherwise, and

generating the motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise; and

determining whether or not to apply a filtering operation to the given pel based on a value of the motion detection metric.

19. The method of claim 18 wherein the filtering operation includes a temporal median filtering operation.

20. The method of claim 18 wherein the filtering operation includes a temporal lowpass filtering operation.

21. The method of claim 18 wherein the filtering operation includes a spatial lowpass filtering operation comprising at least one of a horizontal and a vertical lowpass filtering operation.

22. The method of claim 18 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.

23. An apparatus for processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the apparatus comprising;

a motion detector having an input connected to receive pels from one or more of said frames in said frame sequence, and an output corresponding to a motion detection metric for a given pel of a frame in the frame sequence, wherein the motion detector is operative to generate a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, to generate a second motion metric as a difference been the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence, to generate first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold and another value for the given pel otherwise, and to generate the motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise; and

a processor coupled to the motion detector and having an input for receiving the motion detection metric therefrom, the processor operative to determine whether or not to apply a filtering operation to the given pel based on a value of the motion detection metric.

24. The apparatus of claim 23 wherein the filtering operation includes a temporal median filtering operation.

25. The apparatus of claim 23 wherein the filtering operation includes a temporal lowpass filtering operation.

26. The apparatus of claim 23 wherein the filtering operation includes a spatial lowpass filtering operation comprising at least one of a horizontal and a vertical lowpass filtering operation.

27. The apparatus of claim 23 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.

28. A method of processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the method comprising the steps of:

generating a motion detection metric and an edge detection metric for a given pel of a frame in the frame sequence, wherein said step of generating comprises the steps of;

generating a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, and a second motion metric as a difference between the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence;

generating first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold, and another value for the given pel otherwise; and

generating the motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise; and

determining an amount of spatial lowpass filtering to apply to the given pel based on a value of the motion detection metric and the edge detection metric.

29. The method of claim 28 wherein the spatial lowpass filtering includes at least one of horizontal filtering and vertical filtering.

30. The method of claim 28 wherein the step of determining an amount of spatial lowpass filtering to apply to the given pel further includes the step of utilizing the motion detection metric and edge detection metric to generate an address into a look-up table having a plurality of entries corresponding to different filter coefficients for the spatial lowpass filtering.

31. The method of claim 30 wherein the look-up table includes sixteen different possible entries corresponding to sixteen different sets of filter coefficients for the spatial lowpass filtering.

32. The method of claim 28 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.

33. An apparatus for processing a video signal including a sequence of frames, each of the frames including a plurality of pels, the apparatus comprising:

a motion detector having an input connected to receive pels from one or more of said frames in said frame sequence, and an output corresponding to a motion detection metric for a given pel of a frame in the frame sequence, wherein the motion detector is operative to generate a first motion metric as a difference between a given pel in a current frame of the frame sequence and a corresponding pel in a previous frame of the sequence, to generate a second motion metric as a difference between the given pel of the current frame and a corresponding pel in a subsequent frame of the sequence, to generate first and second bitmaps from the first and second motion metrics, such that each of the first and second bitmaps have one value for the given pel if the first and second motion metrics, respectively, exceed a first predetermined threshold, and another value for the given pel otherwise, and to generate the motion detection metric for the given pel of the current frame such that the motion detection metric for the given pel has one value if a sum of the first and second bitmap values for a group of pels including the given pel exceeds a second predetermined threshold, and another value otherwise;

an edge detector having an input connected to receive pels from one or more of said frames in said frame sequence, and an output corresponding to an edge detection metric for a given pel of a frame in the frame sequence; and

a processor coupled to the motion detector and edge detector and having an input for receiving the motion detection metric and edge detection metric therefrom, the processor operative to determine an amount of spatial lowpass filtering to be applied to the given pel based on values of the motion detection metric and the edge detection metric.

34. The apparatus of claim 33 wherein the spatial lowpass filtering includes at least one of horizontal filtering and vertical filtering.

35. The apparatus of claim 33 further including a look-up table coupled to the motion detector, the edge detector and the processor, such that the motion detection metric and edge detection metric are used to provide an address into the look-up table to specify one of a plurality of table entries corresponding to different filter coefficients for the spatial lowpass filtering.

36. The apparatus of claim 35 wherein the look-up table includes sixteen different possible entries corresponding to sixteen different sets of filter coefficients for the spatial lowpass filtering.

37. The apparatus of claim 33 wherein the group of pels for which the first and second bitmap values are summed further includes pels on one or more lines above a line containing the given pel and one or more lines below the line containing the given pel.