Apparatus and method for filtering digital image signal

• US 7,538,822 B2
• Filed: 02/01/2005
• Issued: 05/26/2009
• Est. Priority Date: 05/04/2004
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Hoyoung, Kim, Vladimar, Park, Dusik, Kim, Changyeong
• Primary Examiner(s): Tran; Trang U
• Application Number:
  US11/046,696
• Publication Number:
  US 20050248687A1
• Time in Patent Office: 1,575 Days
• Field of Search: 348606-607, 348/620, 348622-625, 348627-630, 348241-242, 348252-253, 348/533, 348/615, 348699-700, 348/618, 382260-261, 382/107, 382263-264
• US Class Current: 348/606
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

An apparatus and method of filtering a digital image signal. The apparatus includes: a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each of frames of an image as a temporal or spatial filtering value in response to magnitudes of the results of temporal and spatial filtering; and a sharpness enhancement filter which highlights and outputs a high pass component of the temporal or spatial filtering value.

Claims

1. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the noise reduction filter comprises

a temporal filter which temporally filters the pixel values;

a spatial filter which spatially filters the pixel values; and

a filter selector which selectively outputs one of the results of the temporal and spatial filtering as the respective temporal or spatial filtering value in response to a result of a comparison between a local spatial mean value and each of the results of the temporal and spatial filtering, wherein the filter selector selects the temporal or spatial filtering value according to an Equation

${\hat{f}}_s<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)=\{\begin{array}{cc}\hat{f}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right),& \mathrm{if}<br><br><br><br><br><br>f_{\mathrm{\sigma <br><br>}}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)-\stackrel{\_}{f}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)<br><br>><br><br><br><br>\hat{f}<br><br>\left(\hat{x};<br><br>t\right)-\stackrel{\_}{f}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)<br><br>\\ f_{\mathrm{\sigma <br><br>}}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right),& \mathrm{else}\end{array}$

where “

{circumflex over (f)}_s(,t)”

denotes the temporal or spatial filtering value, “

{circumflex over (f)}(,t)”

denotes the results of temporal filtering of the pixel values, “

f₉₄(,t)”

denotes the results of spatial filtering of the pixel values, and “

f(,t)”

denotes the local spatial mean value.

2. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the noise reduction filter comprises

a temporal filter which temporally filters the pixel values;

a spatial filter which spatially filters the pixel values; and

a filter selector which selectively outputs one of the results of the temporal and spatial filtering as the respective temporal or spatial filtering value in response to a result of a comparison between a local spatial mean value and each of the results of the temporal and spatial filtering

wherein the temporal filter comprises

a pixel value controller which compares a result of a subtraction of a pixel value of a previous frame from a pixel value of a current frame with a motion detection threshold value and outputs the comparison result as a pixel value control signal;

a pixel value operator which generates a new pixel value using the pixel values of the previous and current frames; and

a pixel value selector which selectively outputs the pixel value of the current frame or the new pixel value in response to the pixel value control signal.

3. The apparatus of claim 2, wherein the pixel value operator determines a mean of the pixel values of the previous and current frames as the new pixel value.

4. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the noise reduction filter comprises

a temporal filter which temporally filters the pixel values;

a spatial filter which spatially filters the pixel values; and

a filter selector which selectively outputs one of the results of the temporal and spatial filtering as the respective temporal or spatial filtering value in response to a result of a comparison between a local spatial mean value and each of the results of the temporal and spatial filtering

wherein the temporal filter comprises

a pixel value controller which compares a result of a subtraction of a temporal or spatial filtering value of a previous frame from a pixel value of a current frame with a motion detection threshold value and outputs the comparison result as a pixel value control signal;

a pixel value operator which adds a weight on the pixel value of the current frame and the temporal or spatial filtering value of the previous frame to generate at least one new pixel value; and

a pixel value selector which selectively outputs the pixel value of the current frame or the at least one new pixel value in response to the pixel value control signal.

5. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the noise reduction filter comprises

a temporal filter which temporally filters the pixel values;

a spatial filter which spatially filters the pixel values; and

a filter selector which selectively outputs one of the results of the temporal and spatial filtering as the respective temporal or spatial filtering value in response to a result of a comparison between a local spatial mean value and each of the results of the temporal and spatial filtering

wherein the temporal filter temporally filters the pixel values according to an Equation

$\hat{f}<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)=\{\begin{array}{cc}\frac{f<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)+2\times <br><br>{\hat{f}}_s<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t-1\right)}{3},& \mathrm{if}<br><br><br><br><br><br>f<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)-{\hat{f}}_s<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t-1\right)<br><br>\le <br><br>\frac{d}{2}\\ \frac{2\times <br><br>f<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)+{\hat{f}}_s<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t-1\right)}{3},& \mathrm{if}<br><br><br><br>\frac{d}{2}<<br><br><br><br>f<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right)-{\hat{f}}_s<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t-1\right)<br><br>\le <br><br>d\\ f<br><br>\left(\stackrel{\mathrm{\rho <br><br>}}{x},t\right),& \mathrm{else}\end{array}$

where, “

{circumflex over (f)}(,t)”

denotes the results of temporal filtering of the pixel values, “

f(,t)”

denotes the pixel value of a current frame, “

{circumflex over (f)}_s(,t−

1)”

denotes the temporal or spatial filtering value of a previous frame, and d denotes a motion detection threshold value.

6. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the noise reduction filter comprises

a temporal filter which temporally filters the pixel values;

a spatial filter which spatially filters the pixel values; and

a filter selector which selectively outPuts one of the results of the temporal and spatial filtering as the respective temporal or spatial filtering value in response to a result of a comparison between a local spatial mean value and each of the results of the temporal and spatial filtering,

wherein the spatial filter comprises

a line delayer which receives the temporal or spatial filtering value generated for each neighboring pixel belonging to a previous line inside a window mask of predetermined size having a central pixel to delay the received temporal or spatial filtering value by a unit line; and

a first multiplier which multiplies pixel values of the neighboring pixels belonging to the previous line input from the line delayer among pixels inside the window mask or pixel values of neighboring pixels belonging to a subsequent line input from an external source among pixels inside the window mask by corresponding weights, accumulates the multiplication results, and outputs the accumulation result as the result of spatial filtering of a pixel value of the central pixel,

wherein the neighboring pixel neighbors the central pixel, and the previous and subsequent lines are respectively located before and after a current line to which the central pixel belongs.

7. The apparatus of claim 6, wherein the spatial filter comprises:

a subtracter which subtracts the pixel value of the neighboring pixel from the pixel value of the central pixel;

a reference value generator which generates a reference weight value using a noise variance value predicted from the pixel value of the central pixel;

a comparator which compares the subtraction result of the subtracter with the reference weight value; and

a lookup table which receives the comparison result of the comparator as an address, stores the weights as data, and outputs data corresponding to the address to the first multiplier.

8. The apparatus of claim 7, wherein the reference value generator comprises:

a variance value predictor which predicts the noise variance value from the pixel value of the central pixel;

a frame delayer which delays the predicted noise variance value by a unit frame; and

an operator which generates the reference weight value using the delayed predicted noise variance value.

9. The apparatus of claim 8, wherein the operator generates the reference weight value according to an Equation:

$\mathrm{\sigma <br><br>}=\frac{{\mathrm{\sigma <br><br>}}_{\mathrm{in}}}{10}\times <br><br>N_{\mathrm{var}}+0.5$

wherein σ

denotes the reference weight value, σ

_in denotes a predetermined initial value, and N_var denotes the delayed and predicted noise variance value.

10. The apparatus of claim 8, wherein the variance value predictor comprises:

a first pixel value difference calculator which calculates a difference between a pixel value of each pixel inside the window mask and a median pixel value;

a mean value calculator which calculates a mean value of the differences calculated by the first pixel value difference calculator; and

a maximum histogram detector which predicts the mean value having a maximum histogram as the noise variance value.

11. An apparatus filtering a digital image signal, comprising

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering:

and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value

wherein the sharpness enhancement filter comprises

a high pass component extractor which extracts a high pass component of the temporal or spatial filtering value for a central pixel;

a gain determiner which determines a gain using a variation range of a difference between a pixel value of the central pixel and a pixel value of each neighboring pixel of the central pixel inside a filter mask;

a second multiplier which multiplies the determined gain by the extracted high pass component; and

a synthesizer which synthesizes the multiplication result of the second multiplier and the image and outputs the synthesis result as a sharpened result,

wherein the neighboring pixels neighbor the central pixel, and

the gain determiner determines the gain according to an Equation

$\mathrm{Gain}=\frac{G_{\max }\times <br><br>0.3\times <br><br>D_{\mathrm{range}}}{{\mathrm{th}}_1},\mathrm{if}<br><br><br><br>D_{\mathrm{range}}<<br><br>{\mathrm{th}}_2$ $\mathrm{Gain}=\frac{G_{\max }\times <br><br>0.7\times <br><br>\left(D_{\mathrm{range}}-{\mathrm{th}}_2\right)}{{\mathrm{th}}_2}+0.3\times <br><br>G_{\max },<br><br>\mathrm{if}<br><br><br><br>{\mathrm{th}}_2\le <br><br>D_{\mathrm{range}}<<br><br>{\mathrm{th}}_3$ $\mathrm{Gain}=G_{\max },\mathrm{if}<br><br><br><br>{\mathrm{th}}_3\le <br><br>D_{\mathrm{range}}<<br><br>{\mathrm{th}}_4$ $\mathrm{Gain}=\frac{-G_{\max }\times <br><br>D_{\mathrm{range}}}{{\mathrm{th}}_4}+2\times <br><br>G_{\max },\mathrm{if}<br><br><br><br>{\mathrm{th}}_4\le <br><br>D_{\mathrm{range}}$

where th₂, th₃, and th₄ denote second, third, and fourth predetermined threshold values, respectively, D_range denotes the variation range of the difference, Gain denotes the gain, and G_max denotes a maximum value of the gain.

12. The apparatus of claim 11, wherein the gain determiner comprises a gain calculator which determines the gain to be low in a region in which the variation range of the difference is small, determines the gain to be inversely proportional to the variation range of the difference in a region in which the variation range of the difference is large, or determines the gain to be high in a region in which the variation range of the difference is median.

13. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the sharpness enhancement filter comprises

a high pass component extractor which extracts a high pass component of the temporal or spatial filtering value for a central pixel;

a gain determiner which determines a gain using a variation range of a difference between a pixel value of the central pixel and a pixel value of each neighboring pixel of the central pixel inside a filter mask;

a second multiplier which multiplies the determined gain by the extracted high pass component; and

a synthesizer which synthesizes the multiplication result of the second multiplier and the image and outputs the synthesis result as a sharpened result,

wherein the neighboring pixels neighbor the central pixel, and

the high pass component extractor comprises

a second pixel value difference calculator which calculates a pixel difference between the pixel value of the central pixel and the pixel value of each of the neighboring pixels;

a parameter determiner which determines a parameter corresponding to a variation range of the difference calculated by the second pixel value difference calculator;

a correlation calculator which calculates a correlation between the central pixel and the neighboring pixels using the parameter and the pixel difference; and

a high pass component calculator which calculates the high pass component using the correlation and the temporal or spatial filtering value.

14. The apparatus of claim 13, wherein the parameter determiner determines the parameter according to an Equation:

$\mathrm{\Delta <br><br>}=\{\begin{array}{cc}\mathrm{High},& \mathrm{if}<br><br><br><br>D_{\mathrm{range}}\le <br><br>{\mathrm{th}}_1\\ \mathrm{Low},& \mathrm{if}<br><br><br><br>D_{\mathrm{range}}><br><br>{\mathrm{th}}_1\end{array}$

wherein Δ

denotes the parameter, th₁ denotes a first predetermined threshold value, and D_range denotes the variation range of the difference which is determined according to an Equation;

D_range=MAX{D(m,n)}−

MIN{D(m,n)}

wherein D(m,n) denotes the pixel difference calculated by the second pixel value difference calculator which is calculated according to an Equation;

D(m,n)=|z(m,n)−

z(i,j)|

wherein, when the filter mask has a size of M×

N, 1≦

m≦

M and 1≦

n≦

N, z(m,n) denotes the pixel value of the central pixel, and z(i,j) denotes the pixel value of the neighboring pixel.

15. The apparatus of claim 13, wherein the correlation calculator calculates the correlation according to Equations:

$D<br><br>\left(m,n\right)=<br><br>z<br><br>\left(m,n\right)-z<br><br>\left(i,j\right)<br><br><br><br><br><br>\mathrm{and}<br><br><br><br>W_{\mathrm{corr}}<br><br>\left(m,n\right)=\exp <br><br>\left(-\frac{{D<br><br>\left(m,n\right)}^2}{{\mathrm{\Delta <br><br>}}^2}\right)$

wherein D(m,n) denotes the pixel difference calculated by the second pixel value difference calculator, when the filter mask has a size of M×

N, 1≦

m≦

M and 1≦

n≦

N, z(m,n) denotes the pixel value of the central pixel, z(i,j) denotes the pixel value of the neighboring pixel, Δ

denotes the parameter, and W_corr(m,n) denotes the correlation.

16. The apparatus of claim 15, wherein the high pass component calculator calculates the high pass component according to an Equation:

${\mathrm{\gamma <br><br>}}_{\mathrm{HPF}}<br><br>\left(i,j\right)=\underset{\left(m,n\right)\in <br><br>M\times <br><br>N}{\sum <br><br>}<br><br>\left\{\left[W_{\mathrm{corr}}<br><br>\left(m,n\right)-\frac{1}{M\times <br><br>N}<br><br>\underset{\left(m,n\right)\in <br><br>M\times <br><br>N}{\sum <br><br>}<br><br>W_{\mathrm{corr}}<br><br>\left(m,n\right)\right]\times <br><br>z<br><br>\left(m,n\right)\right\}$

wherein “

Y_HPF(i,j)”

denotes the high pass component.

17. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outPuts one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the sharpness enhancement filter comprises

a high pass component extractor which extracts a high pass component of the temporal or spatial filtering value for a central pixel;

a gain determiner which determines a gain using a variation range of a difference between a pixel value of the central pixel and a pixel value of each neighboring pixel of the central pixel inside a filter mask;

a second multiplier which multiplies the determined gain by the extracted high pass component; and

a synthesizer which synthesizes the multiplication result of the second multiplier and the image and outputs the synthesis result as a sharpened result,

wherein the neighboring pixels neighbor the central pixel,

the gain determiner comprises a gain adjuster which detects variation ranges of differences for all pixels belonging to a block having one of the neighboring pixels as a central pixel and adjusts the gain using the detected variation ranges of the differences, and

the gain adjuster comprises

a variation range detector which detects the variation range of the difference between the pixel value of each of the pixels belonging to the block and the pixel value of each of the neighboring pixels;

a variation degree calculator which calculates a difference between maximum and minimum values of the variation range of the difference detected by the variation range detector; and

a gain attenuator which adjusts the determined gain using the variation range of the difference in response to a magnitude of the difference calculated by the variation degree calculator.

18. An apparatus filtering a digital image signal, comprising:

a noise reduction filter which selectively outputs one of results obtained by temporally and spatially filtering pixel values of pixels of each frame of an image as a temporal or spatial filtering value in response to magnitudes of the results of the temporal and spatial filtering; and

a sharpness enhancement filter which highlights and outputs a high pass component of the selected temporal or spatial filtering value,

wherein the sharpness enhancement filter comprises

a high pass component extractor which extracts a high pass component of the temporal or spatial filtering value for a central pixel;

a gain determiner which determines a gain using a variation range of a difference between a pixel value of the central pixel and a pixel value of each neighboring pixel of the central pixel inside a filter mask;

a second multiplier which multiplies the determined gain by the extracted high pass component; and

a synthesizer which synthesizes the multiplication result of the second multiplier and the image and outputs the synthesis result as a sharpened result,

wherein the neighboring pixels neighbor the central pixel,

the gain determiner comprises a gain adjuster which detects variation ranges of differences for all pixels belonging to a block having one of the neighboring pixels as a central pixel and adjusts the gain using the detected variation ranges of the differences and

when the difference calculated by the variation degree calculator is equal to or larger than a fifth predetermined threshold value, the gain attenuator adjusts the determined gain according to an Equation

$G_{\mathrm{final}}=\mathrm{Gain}\times <br><br>\frac{D_{\mathrm{range}}}{2^c}$

where G_final denotes the adjusted gain output from the gain attenuator, Gain denotes the determined gain, D_range denotes the variation range of the difference, and c denotes a number of the neighboring pixels.