Method of and apparatus for eliminating image noise

• US 8,050,509 B2
• Filed: 09/28/2007
• Issued: 11/01/2011
• Est. Priority Date: 11/21/2006
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Ju-hee, Lee, Keum-seok, Jeong, Young-hoon, Hwang, Won-kyu
• Primary Examiner(s): Bella; Matthew
• Assistant Examiner(s): Newman; Michael A
• Application Number:
  US11/863,393
• Publication Number:
  US 20080118179A1
• Time in Patent Office: 1,495 Days
• Field of Search: 382/261, 382/254, 382/260, 382/263, 382/264, 382/266, 382/272, 382/274, 382/275, 358/463
• US Class Current: 382/261
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06V 10/30 Noise filtering

Abstract

A method and apparatus are provided for eliminating image noise to remove spatial-temporal noise and improve visibility. The method includes extracting a spatial-temporal noise level of neighbor pixels around a current pixel, filtering noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the extracted spatial-temporal noise level, and applying a weight to the noise-filtered pixel and a boosted-up pixel based on an edge intensity and summing the weight-applied pixels. The spatial-temporal noise level is extracted based on spatial-temporal information of neighbor pixels around a current pixel in a current frame and spatial-temporal information of neighbor pixels around a current pixel in a previous frame.

Claims

1. A method of eliminating image noise, the method comprising:

extracting a spatial-temporal noise level of neighbor pixels around a current pixel of a current frame, wherein the extracting the spatial-temporal noise level comprises extracting a spatial edge intensity parameter, a temporal correlation parameter and a spatial-temporal brightness parameter, and generating the spatial-temporal noise level as a summation of the spatial edge intensity parameter, the temporal correlation parameter and the spatial-temporal brightness parameter;

filtering noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the extracted spatial-temporal noise level; and

applying a weight to the noise-filtered current pixel and to a boosted-up pixel based on an edge intensity, and summing up the weight-applied pixels.

2. The method of claim 1, wherein the spatial-temporal noise level is extracted based on spatial-temporal information of the neighbor pixels around the current pixel in the current frame and spatial-temporal information of neighbor pixels around a corresponding pixel in a previous frame.

3. The method of claim 1, wherein the spatial-temporal noise level is extracted based on a spatial edge intensity of an image, a temporal correlation between the neighbor pixels in the current frame and neighbor pixels around a corresponding pixel in a previous frame, and a change in spatial-temporal brightness of the neighbor pixels in the current frame and the neighbor pixels in the previous frame.

4. The method of claim 1, wherein the extracting the spatial-temporal noise level comprises:

extracting a spatial edge intensity of an image, a temporal correlation between the neighbor pixels in the current frame and neighbor pixels around a corresponding pixel in a previous frame, and a spatial-temporal brightness of the neighbor pixels in the current frame and the neighbor pixels in the previous frame; and

transforming the spatial edge intensity, the temporal correlation, and the spatial-temporal brightness to the spatial edge intensity parameter, the temporal correlation parameter and the spatial-temporal brightness parameter using function-based equations.

5. The method of claim 4, wherein the extracting the spatial edge intensity comprises:

performing high-pass filtering on an image signal of the current frame in order to extract an edge intensity;

transforming the high-pass filtered signal using a function; and

transforming the transformed signal to the spatial edge intensity parameter using a function-based equation.

6. The method of claim 4, wherein the extracting the temporal correlation comprises:

extracting a difference between the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame;

transforming the difference using a function; and

transforming the transformed difference to the temporal correlation parameter using a function-based equation.

7. The method of claim 4, wherein the extracting the spatial-temporal brightness comprises:

extracting a pixel change around the current pixel in the current frame and the corresponding pixel in the previous frame;

transforming the pixel change using a function; and

transforming the transformed pixel change to the spatial-temporal brightness parameter using a function-based equation.

8. The method of claim 4, wherein the filtering of the noise comprises:

summing the spatial edge intensity parameter, the temporal correlation parameter, and the spatial-temporal brightness parameter;

calculating weights for the current pixel and its neighbor pixels based on a result of the summing and a difference between the current pixel and its neighbor pixels;

calculating weights for the corresponding pixel and its neighbor pixels based on the result of the summing and a difference between the corresponding pixel and its neighbor pixels; and

calculating a weight average for the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame based on the calculated weights and replacing the current pixel with the weight average.

9. The method of claim 1, wherein the applying the weight based on the edge intensity comprises:

boosting up a pixel of an image signal of the current frame; and

applying the weight to the noise-filtered pixel and the boosted-up pixel based on the edge intensity, and summing the weight-applied pixels.

10. The method of claim 9, wherein the weight based on the edge intensity is a parameter determined by an output of a function that receives the edge intensity.

15. The method of claim 1, wherein the extracting the spatial-temporal noise level comprises extracting the spatial edge intensity parameter using a spatial edge intensity of an image, extracting the temporal correlation parameter using a temporal correlation between the current frame and a previous frame, and extracting the spatial-temporal brightness parameter using a spatial-temporal brightness of the neighbor pixels around the current pixel and the neighbor pixels around a corresponding pixel in the previous frame.

17. The method of claim 1, wherein the weight applied to the spatial-temporal pixels around the current pixel is inversely proportional to the extracted spatial-temporal noise level.

11. An image processing apparatus comprising:

a noise level extraction unit which extracts a spatial-temporal noise level of neighbor pixels around a pixel based on spatial-temporal information of neighbor pixels around a current pixel in a current frame and spatial-temporal information of neighbor pixels around a corresponding pixel in a previous frame, wherein the noise level extraction unit extracts the spatial-temporal noise level by extracting a spatial edge intensity parameter, a temporal correlation parameter and a spatial-temporal brightness parameter, and generating the spatial-temporal noise level as a summation of the edge intensity parameter, the temporal correlation parameter and the spatial-temporal brightness parameter;

a spatial-temporal noise elimination unit which filters noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the spatial-temporal noise level extracted by the noise level extraction unit; and

an edge improvement unit which applies a weight to the noise-filtered current pixel obtained by the spatial-temporal noise elimination unit and to a boosted-up pixel based on an edge intensity and sums the weight-applied pixels,

wherein at least one of the noise level extraction unit, the spatial-temporal noise elimination unit and the edge improvement unit is a hardware component.

12. The image processing apparatus of claim 11, wherein the noise level extraction unit comprises:

an edge intensity parameter extraction unit which generates the spatial edge intensity parameter by extracting the edge intensity by performing high-pass filtering on an image signal of the current frame;

a correlation parameter extraction unit which generates the temporal correlation parameter by extracting a difference between the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame; and

a brightness parameter extraction unit which generates a brightness parameter by extracting a pixel change around the current pixel in the current frame and the corresponding pixel in the previous frame.

13. The image processing apparatus of claim 12, wherein the spatial-temporal noise elimination unit comprises:

an addition unit which sums the spatial edge intensity parameter, the temporal correlation parameter, and the spatial-temporal brightness parameter;

a weight extraction unit which extracts weights for the current pixel and its neighbor pixels based on a summing result of the addition unit and a difference between the current pixel and its neighbor pixels, and extracts weights for the corresponding pixel and its neighbor pixels based on the summing result of the addition unit and a difference between the corresponding pixel and its neighbor pixels; and

a noise filtering unit which calculates a weight average for the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame based on the calculated weights and replaces the current pixel with the weight average.

14. The image processing apparatus of claim 11, wherein the edge improvement unit comprises:

a boosting-up unit which boosts up a pixel of an image signal of the current frame; and

an edge visual-improvement unit which applies the weight to the noise-filtered pixel and the boosted-up pixel based on the edge intensity and sums the weight-applied pixels.

16. The image processing apparatus of claim 11, the noise level extraction unit extracts the spatial-temporal noise level by extracting the spatial edge intensity parameter using a spatial edge intensity of an image, extracting the temporal correlation parameter using a temporal correlation between the current frame and a previous frame, and extracting the spatial-temporal brightness parameter using a spatial-temporal brightness of the neighbor pixels around the current pixel and the neighbor pixels around a corresponding pixel in the previous frame.

18. The image processing apparatus of claim 11, wherein the weight applied to the spatial-temporal pixels around the current pixel is inversely proportional to the extracted spatial-temporal noise level.

19. A method of eliminating image noise, the method comprising:

extracting a spatial-temporal noise level of neighbor pixels around a current pixel of a current frame;

filtering noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the extracted spatial-temporal noise level; and

applying a weight to the noise-filtered current pixel and to a boosted-up pixel based on an edge intensity, and summing up the weight-applied pixels,

wherein the extracting the spatial-temporal noise level comprises;

extracting a spatial edge intensity of an image, a temporal correlation between the neighbor pixels, and a spatial-temporal brightness of the neighbor pixels; and

transforming the spatial edge intensity, the temporal correlation, and the spatial-temporal brightness to parameters using function-based equations, and

wherein the filtering of the noise comprises;

summing the spatial edge intensity parameter, the temporal correlation parameter, and the spatial-temporal brightness parameter;

calculating weights for the current pixel and its neighbor pixels based on a result of the summing and a difference between the current pixel and its neighbor pixels;

calculating weights for the corresponding pixel and its neighbor pixels based on the result of the summing and a difference between the corresponding pixel and its neighbor pixels; and

calculating a weight average for the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame based on the calculated weights and replacing the current pixel with the weight average.

20. An image processing apparatus comprising:

a noise level extraction unit which extracts a spatial-temporal noise level of neighbor pixels around a pixel based on spatial-temporal information of neighbor pixels around a current pixel in a current frame and spatial-temporal information of neighbor pixels around a corresponding pixel in a previous frame;

a spatial-temporal noise elimination unit which filters noise of the current pixel by applying a weight to spatial-temporal pixels around the current pixel based on the spatial-temporal noise level extracted by the noise level extraction unit; and

an edge improvement unit which applies a weight to the noise-filtered current pixel obtained by the spatial-temporal noise elimination unit and to a boosted-up pixel based on an edge intensity and sums the weight-applied pixels,

wherein at least one of the noise level extraction unit, the spatial-temporal noise elimination unit and the edge improvement unit is a hardware component,

wherein the noise level extraction unit comprises;

an edge intensity parameter extraction unit which generates an edge intensity parameter by extracting the edge intensity by performing high-pass filtering on an image signal of the current frame;

a correlation parameter extraction unit which generates a correlation parameter by extracting a difference between neighbor pixels around the current pixel in the current frame and neighbor pixels around the corresponding pixel in the previous frame; and

a brightness parameter extraction unit which generates a brightness parameter by extracting a pixel change around the current pixel in the current frame and the corresponding pixel in the previous frame, and

wherein the spatial-temporal noise elimination unit comprises;

an addition unit which sums the edge intensity parameter, the correlation parameter, and the brightness parameter;

a weight extraction unit which extracts weights for the current pixel and its neighbor pixels based on a summing result of the addition unit and a difference between the current pixel and its neighbor pixels, and extracts weights for the corresponding pixel and its neighbor pixels based on the summing result of the addition unit and a difference between the corresponding pixel and its neighbor pixels; and

a noise filtering unit which calculates a weight average for the neighbor pixels around the current pixel in the current frame and the neighbor pixels around the corresponding pixel in the previous frame based on the calculated weights and replaces the current pixel with the weight average.