Content-Dependent Scan Rate Converter with Adaptive Noise Reduction

• US 20080218630A1
• Filed: 05/20/2008
• Published: 09/11/2008
• Est. Priority Date: 12/31/2001
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Skoog, Arnold P., Kempf, Jeffrey, Fairbanks, Clifford D.
• Granted Patent: US 7,738,038 B2
• Time in Patent Office: Days
• US Class Current: 348/452
• CPC Class Codes: H04N 5/213 Circuitry for suppressing or minimising impulsive noise for suppressing or minimising disturbance in image signal generation H04N23/81
• CPC Class Codes: H04N 5/7416 involving the use of a spatial light modulator, e.g. a light valve, controlled by a video signal
• CPC Class Codes: H04N 5/7458 the modulator being an array of deformable mirrors, e.g. digital micromirror device [DMD]
• CPC Class Codes: H04N 7/0115 with details on the detection of a particular field or frame pattern in the incoming video signal, e.g. 3:2 pull-down pattern
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0137 dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144

Abstract

A content-dependent scan rate converter with adaptive noise reduction that provides a highly integrated, implementation efficient de-interlacer. By identifying and using redundant information from the image (motion values and edge directions), this scan rate converter is able to perform the tasks of film-mode detection, motion-adaptive scan rate conversion, and content-dependent video noise reduction. Adaptive video noise reduction is incorporated in the process where temporal noise reduction is performed on the still parts of the image, thus preserving high detail spatial information, and data-adaptive spatial noise reduction is performed on the moving parts of the image. A low-pass filter is used in flat fields to smooth out Gaussian noise and a direction-dependent median filter is used in the presence of impulsive noise or an edge. Therefore, the selected spatial filter is optimized for the particular pixel that is being processed to maintain crisp edges.

Claims

1. A method for video content-dependent scan rate conversion with adaptive noise reduction, comprising the steps of:

for the luminance domain of an interlaced source image;

temporal filtering said image to mitigate the possibility of false motion detection;

analyzing the motion patterns of said video source to determine if it originated from a 2;

2 or 3;

2 progressive film mode source or from an inherently interlaced source;

based on said source type, determine the optimal solution for scan rate conversion on a pixel-by-pixel basis;

determining motion intensity values for said interlaced image;

detecting edge directions in said interlaced image;

using said motion values and edge directions to perform motion-adaptive scan rate conversion on a pixel-by-pixel basis for said image;

in parallel with said motion-adaptive scan rate conversion, using said motion values and edge directions to perform content-dependent video noise reduction of said interlaced image;

assembling a new content-dependent progressive scan luminance image depending on the presence of motion in said image, wherein;

adjacent odd and even fields of said image are jammed together in the still portions of said image where motion is not present, said still portions of image being temporally filtered to smooth out Gaussian noise;

interpolating along edges of present filed of said image in the portions of said image where motion is present, said motion portions of image being spatially filtered using a direction-dependent median filter in the presence of impulse noise or and edge;

for the chrominance domain of said source interlaced image;

first performing spatial filtering using a median filter to remove impulse noise;

second, performing a linear interpolation on said chrominance signal by averaging adjacent lines in the current field of said interlaced image; and

performing an up conversion to combine the new luminance and chrominance fields with the current field to provide a progressive image.

2. The method of claim 1, wherein if said video originated from a 2:

2 or 3;

2 film mode source, adjacent interlaced fields are jammed together to form the original progressive video frame.

3. The method of claim 1, wherein if said video originated from an inherently interlaced signal:

if motion is not detected, pixels from the current and the previous fields are jammed together; and

if motion is detected, a new pixel value is interpolated using data from said current field only.

4. The method of claim 2, wherein said film mode detection further comprises the steps of:

summing all the motion intensity values of a single motion image and quantizing this value to 20-bits to provide a motion sum;

taking the absolute value of the difference between two adjacent luminance fields of said motion image;

comparing said quantized motion sum to the previous quantized motion sum;

if said motion sums periodically toggle between large and small values, with a data pattern of 0, 1, 0, 1, a 2;

2 pull-down source is detected indicating a 50 Hz PAL format.

5. The method of claim 2, wherein said film mode detection further comprises the steps of:

quantizing said motion values to a single bit;

comparing the sum of all said quantized motion values for a motion image to a noise threshold;

if the output of the comparator is a binary 1 when the motion sum is greater than the said noise threshold and a binary 0 otherwise, a comparator periodic output data pattern of 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, a 3;

2 pull-down source is detected indicating a 60 Hz NTSC format.

6. The method of claim 1, wherein said motion-adaptive scan rate conversion further comprises the steps of:

performing a temporal noise filtering of said video source;

performing a flat-field low-pass filtering of said video source;

performing a first and second field delay on said source image;

obtaining a 9-bit motion image, consisting of a sign-bit and 8 magnitude bits, by subtracting said second delayed field from said original image;

performing a first non-linear pre-filtering of said motion image;

splitting said pre-filtered motion image along two paths, a 5-tap median filter path to reduce the impulse noise in the motion image and a parallel 3×

3 low-pass filter path to remove Gaussian noise from the motion image;

determining a noise detection parameter for said image;

selecting the signal from said median filter path, by means of a multiplexer, if said noise detection parameter is greater than a preset edge threshold, indicating the presence of a motion edge or impulse noise,

selecting the signal from said low-pass filter path, by means of said multiplexer, if said noise detection parameter value is equal to or less than said preset edge threshold, indicating the absence of a motion edge or impulse noise;

further reducing noise in said signal by zeroing out small positive and negative values, leaving only larger motion values;

converting the filtered motion value to a 4-bit absolute value;

passing said 4-bit value through a 5-pixel by 3-line spatial maximum filter to dilate said motion image, thereby improving spatial motion coverage;

further passing said dilated motion image from the current and previous fields through a 7-pixel by 5-line spatial-temporal maximum filter;

coring the 4-bit motion intensity values using a second non-linear filtering function; and

supplying said non-linear filtered image being supplied to a new field multiplexer.

7. The method of claim 6, wherein said noise detection further comprises the steps of:

using said median filter in said scan rate converter to take the absolute difference between said 9-bit motion image and its 8-bit filtered value for each pixel of said image;

summing said absolute differences over an entire field to obtain a single noise detection parameter; and

using said noise detection parameter value to determine the amount of noise filtering to apply to said video data, where strong filtering is applied when the value of said noise detection parameter is large, and little to no filtering is applied when the value of said noise detection parameter is small.

8. The method of claim 1, wherein said edge detection, determined in parallel with said motion intensity determination, further comprises the steps of:

taking the absolute value of the luminance differences between pixels consisting of the group;

{(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x,y−

1)}, {(x,y)−

(x+1,y−

1)}, {(x−

1,y)−

(x+1,y−

1)}, {(x+1,y)−

(x−

1,y−

1)}, {(x−

1,y)−

(x,y−

1)}, {(x+1,y)−

(x, y−

1)}, {(x,y)−

(x+1,y)}, {(x.y)−

(x+1,y+1)}, {(x,y)−

(x,y+1)}, {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x−

1,y)};

wherein if the said difference between two pixels is greater than a pre-determined threshold, it is assumed that an edge exists between said pixels; and

conversely, where the difference between two pixels is less that said pre-determined threshold, it is assumed that the two pixels lie on an edge.

9. The method of claim 8, wherein two types of edges are identified from a group consisting of:

an edge between two lines within a field to be used for interpolation; and

an edge among three lines within a field to be used for spatial noise reduction.

10. The method of claim 9, wherein the minimum from {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x,y−

1)}, {(x,y)−

(x+1,y−

1)}, {(x−

1,y)−

(x+1,y−

1)}, {(x+1,y)−

(x−

1,y−

1)}, {(x−

1,y)−

(x,y−

1)}, {(x+1,y)−

(x, y−

1)} is used to determine an edge for said interpolation to preserve vertical detail in the image.

11. The method of claim 10, wherein:

the average of the differences {(x,y)−

(x−

1,y−

1)} and {(x+1,y)−

(x, y−

1)} is used to detect an edge having a slope of 63 degrees;

the average of the differences {(x,y)−

(x+1,y−

1)} and {(x+1,y)−

(x−

1,y−

1)} is used to detect an edge having a slope of 117 degrees;

the difference {(x+1,y)−

(x−

1,y−

1)} is used to detect an edge having a slope of 135 degrees;

the difference {(x,y)−

(x,y−

1)} is used to detect an edge having a slope of 90 degrees; and

the difference {(x−

1,y)−

(x+1,y−

1)} is used to detect an edge having a slope of 45 degrees.

12. The method of claim 8, wherein if all said differences are less than said pre-determined threshold, it is assumed that an edge is not present and interpolation is done along the edge defined between pixels (x,y) and (x,y−

1).

13. The method of claim 8, wherein ii all said differences {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x+1,y−

1)}, {(x−

1,y)−

(x+1,y−

1)}, {(x+1,y)−

(x−

1,y−

1)}, {(x−

1,y)−

(x,y−

1)}, {(x+1,y)−

(x, y−

1)} are all less than said pre-determined threshold, it is assumed that correlation is present in opposing directions and interpolation is performed along the edge defined by difference {(x,y)−

(x,y−

1)}.

14. The method of claim 8, wherein:

the minimum among differences {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x,y−

1)}, {(x,y)−

(x+1,y−

1)}, {(x,y)−

(x+1,y)}, {(x.y)−

(x+1,y+1)}, {(x,y)−

(x,y+1)}, {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x−

1,y)} is used to determine if an edge is present for said spatial noise reduction filtering, where the pixel at the center of these differences is the pixel to be filtered;

if all these differences are less than a pre-determined threshold it is assumed that an edge is not present and a two-dimensional low-pass filter is used; and

if a difference is greater than said pre-determine threshold it is assumed that an edge is present and a direction dependent 5-tap median filter is used.

15. The method of claim 14, wherein the location of the taps of said direction dependent median filter is determined by finding the minimum among the differences {(x,y)−

(x−

1,y−

1)}, {(x,y)−

(x,y−

1)}, {(x,y)−

(x+1,y−

1)}, {(x.y)−

(x+1,y+1)}, {(x,y)−

(x,y+1)}, {(x,y)−

(x−

1,y+1)}.

16. The method of claim 6, wherein said 5-tap median filter is used for motion noise reduction to replace the current pixel intensity value with the median value from adjacent pixels in the current row, adjacent pixels in current column, and said current pixel.

17. The method of claim 16, wherein:

the differences between the center tap (x,y) value and the non-center tap values (x−

1,y), (x−

1,y), (x,y+1), (x,y−

1) are calculated;

the absolute value of said differences are used to determine the optimum noise reduction filter to use for the current pixel (x,y) from the equation

Out=median{(x,y−

1),(x−

1,y),(x,y),(x,y+1),(x+1,y)};

in the presence of impulse noise the median filter is selected;

in the presence of an edge the median filter is selected; and

otherwise, the low-pass is selected to filter Gaussian noise.

18. The method of claim 6, wherein said 5-tap median filter is used for direction dependent video noise reduction to replace the current pixel intensity value with the median value from adjacent pixels in the row, correlated pixels in above and below rows, and said current pixel.

19. The method of claim 11, wherein:

the differences between the center tap (x,y) value and the non-center tap values (x−

1,y−

1), (x,y−

1), (x+1,y−

1), (x+1,y+1), (x,y+1), and (x+1,y+1) are calculated; and

the absolute value of said differences are used to determine the optimum taps to use for the median filter, whereby the upper tap location is determined by finding the minimum of the upper differences and the lower tap location is determined by finding the minimum of the lower differences.

20. The method of claim 6, wherein said 3×

3 low-pass filter has coefficients of ¼

for current pixel, ⅛

for adjacent pixels in the row, ⅛

for adjacent pixels in the column, and 1/16 for the four corner pixels.

21-33. -33. (canceled)