Method and apparatus for image noise reduction using noise models

• US 20080056607A1
• Filed: 08/30/2006
• Published: 03/06/2008
• Est. Priority Date: 08/30/2006
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Micron Technology, Inc.
• Inventors: Ovsiannikov, Ilia
• Granted Patent: US 8,160,381 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

A method and apparatus for processing imager pixel signals to reduce noise. The processing includes receiving a target pixel signal, receiving at least one neighboring pixel signal, formulating a dynamic noise signal based at least in part on a value of the target pixel signal, and controlling a noise reduction operation using the dynamic noise signal.

Claims

1. A method of processing imager pixel signals to reduce noise, the method comprising:

receiving a target pixel signal;

formulating a dynamic noise signal based at least in part on a value of the target pixel signal; and

controlling a noise reduction operation using the dynamic noise signal.

2. The method of claim 1, wherein controlling a noise reduction operation comprises combining the pixel signal with a noise adjustment signal.

3. The method of claim 1, wherein the dynamic noise signal is determined in part from information on target pixel noise and pixel processing system noise.

4. The method of claim 3, further comprising:

receiving at least one neighboring pixel signal,

subtractively combining the target pixel signal and the at least one neighboring pixel signal and forming a noise adjustment signal based on a relationship of the subtractive combination to the dynamic noise threshold.

5. The method of claim 4, wherein:

the noise reduction operation is provided by aperture correction; and

the noise adjustment signal (Y_AP) is generated according to;

Y_d=Y_Pixel−

Y_Surround, and

Y_AP=(|Y_d|−

T_AP)*k_AP*sgn(Y_d),

where;

Y_Pixel is a luminance component of the target pixel signal,

Y_surround is a luminance component of the at least one neighboring pixel,

Y_d is an intermediate value representing the difference between Y_pixel and Y_Surround,

T_AP is the dynamic noise signal,

k_AP is an aperture correction gain, and

sgn(Y_d) is a value of 1 or −

1 based on the sign of Y_d.

6. The method of claim 5, wherein the dynamic noise signal is a threshold (TAP) determined according to:

T_AP=Y_Pixel*k_N+T_N0,

where Y_Pixel is a luminance component of the target pixel signal, k_N is a pixel model noise slope, and T_N0 is a noise floor offset.

7. The method of claim 4, wherein the dynamic noise signal is a threshold (TAP) determined according to:

T_AP=Y_pixel*k_Npixel+Y_surround*k_Nsurround+T_N0,

where Y_Pixel is a luminance component of the target pixel signal, Y_surround is a luminance component of the at least one neighboring pixel, k_Npixel is a noise estimate for the target pixel, k_Nsurround is a noise estimate for the at least one neighboring pixel, and T_N0 is a noise floor offset.

8. The method of claim 1, wherein the noise reduction operation comprises controlling an aperture correction operation when a subtractive combination of the target pixel signal and a received at least one neighboring pixel signal exceeds the dynamic noise signal.

9. The method of claim 8, wherein the aperture correction forms an aperture correction signal in accordance with a luminance value of the target pixel signal and a luminance value of the at least one neighboring pixel signal, a magnitude of the aperture correction signal being determined by an amount by which the subtractive combination of the target pixel signal and the neighboring pixel signal exceeds the dynamic noise signal.

10. The method of claim 9, wherein the aperture correction signal is a luminance signal which is combined with a luminance signal of the target pixel.

11. The method of claim 1, wherein the noise reduction operation comprises controlling an amount of blurring introduced by a color demosaicing operation in accordance with the dynamic noise signal.

12. The method of claim 11, wherein:

the demosaicing operation includes detecting whether there is an edge transition adjacent to the target pixel; and

the detection is based on the dynamic noise signal.

13. The method of claim 12, wherein:

if an edge transition is detected, the demosaicing operation is based on the target pixel and pixels surrounding the target pixel on a same side of the edge transition as the target pixel; and

if an edge transition is not detected, the demosaicing operation is based on the target pixel and pixels surrounding the target pixel.

14. The method of claim 12, wherein detecting the edge transition comprises:

determining a first estimated area pixel value for an area surrounding the target pixel;

determining a second estimated area pixel value for an area surrounding the at least one neighboring pixel; and

comparing the first and second estimated area pixel values.

15. The method of claim 12, wherein the dynamic noise signal is a threshold (T_Edge) determined according to:

T_Edge=P_PixelA*k_N+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

k_N is a coefficient making up a linear noise model, and

T_N0 is a noise floor offset.

16. The method of claim 15, wherein detecting the edge transition is determined according to:

if T_edge<

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is present; and

if T_edge≧

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is not present,

where P_PixelB is a pixel value for a neighboring pixel signal.

17. The method of claim 12, wherein the dynamic noise signal is a threshold (T_Edge) determined according to:

T_Edge=P_PixelA*k_{N A}+P_PixelB*k_{N B}+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

P_PixelB is a pixel value for a neighboring pixel signal,

k_{N A} and k_{N B} are coefficients making up a linear noise model, and

T_N0 is a noise floor offset.

18. The method of claim 17, wherein detecting the edge transition is determined according to:

if T_edge<

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is present; and

if T_edge≧

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is not present.

19. The method of claim 12, wherein the dynamic noise signal is a threshold (T_Edge) determined according to:

T_edge²=P_PixelA²*k_{N A2}+P_PixelA*k_{N A1}+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

k_{N A1} and k_{N A2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

20. The method of claim 19, wherein detecting the edge transition is determined according to:

if T_edge²<

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is not present,

where P_PixelB is a pixel value for a neighboring pixel signal.

21. The method of claim 12, wherein the dynamic noise signal is a threshold (T_Edge) determined according to:

T_edge²=P_PixelA²*k_{N A2}+P_PixelA*k_{N A1}+P_PixelB²*k_{N B2}+P_PixelB*P_PixelB²*k_{N B2}+P_PixelB*k_{N B1}+T_N0;

where;

P_PixelA is an estimate for area brightness for an area defined by a target pixel A at a location (x,y) and selected pixels surrounding the target pixel A,

P_PixelB is an estimate for area brightness for an area defined by a neighboring pixel B on the other side of the edge being tested,

k_{N A1}, k_{N A2}, k_{N B1}, and k_{N B2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

22. The method of claim 21, wherein detecting the edge transition is determined according to:

if T_edge²<

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is not present.

23. The method of claim 12, wherein the dynamic noise signal is a threshold (T_Edge) determined according to:

T_edge²=P_AreaA²*k_{N A2}+P_AreaA*k_{N A1}+P_AreaB²*k_{N B2}+P_AreaB*P_AreaB²*k_{N B2}+P_AreaB*k_{N B1}+T_N0;

P_AreaA(x,y)=avg(P_i,j|P_i,j adjacent to pixel (x,y) and P_i,j on side A); and

P_AreaB(X,y)=avg(P_i,j|P_i,j adjacent to pixel (x,y) and P_i,j on side B),

where;

P_AreaA is an estimate for area brightness for an area defined by a target pixel A at a location (x,y) and selected pixels surrounding the target pixel A,

P_AreaB is an estimate for area brightness for an area defined by a neighboring pixel B on the other side of the edge being tested,

i and j are coordinates for a central target pixel A,

avg is an averaging function,

k_{N A1}, k_{N A2}, k_{N B1}, and k_{N B2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

24. The method of claim 23, wherein detecting the edge transition is determined according to:

if T_edge²<

(P_AreaA−

P_AreaB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_AreaA−

P_AreaB)², then it is determined that an edge transition is not present.

25. A method of processing pixel image signals to reduce noise, the method comprising:

receiving a target pixel signal;

receiving at least one neighboring pixel signal;

determining a dynamic noise threshold based on at least the target pixel signal and a noise model;

determining if a subtractive combination of the target pixel signal with the neighboring pixel signal exceeds the dynamic noise threshold; and

if it is determined that the combination exceeds the dynamic noise threshold, applying noise reduction processing to the target pixel signal which comprises combining the target pixel signal with an adjustment signal as part of an aperture correction operation.

26. The method of claim 25, further comprising controlling a level of the adjustment signal in accordance with an amount by which the combination exceeds the dynamic noise threshold.

27. The method of claim 26, wherein the aperture correction operation forms a noise corrected luminance component of the target pixel signal.

28. A method of processing pixel image signals to reduce noise, the method comprising:

receiving a target pixel signal;

receiving at least one neighboring pixel signal;

determining a dynamic edge threshold based on at least the target pixel signal and a noise model; and

demosaicing based on detecting an edge transition adjacent to a target pixel using the dynamic edge threshold.

29. The method of claim 28, wherein the demosaicing comprises forming red/green/blue (RGB) pixel values for the target pixel in accordance with the dynamic noise threshold.

30. The method of claim 29, wherein:

if an edge is detected, the demosaicing comprises generating the RGB values based on the target pixel and pixels surrounding the target pixel on a same side of the edge transition as the target pixel; and

if an edge is not detected, the demosaicing comprises generating the RGB values based on the target pixel and pixels surrounding the target pixel.

31. The method of claim 29, wherein the demosaicing comprises:

determining a first estimated area pixel value for an area surrounding the target pixel;

determining a second estimated area pixel value for an area surrounding the at least one neighboring pixel;

comparing the first and second estimated area pixel values; and

detect an edge transition based on the comparison.

32. A method of processing imager pixel signals in an image pipeline to reduce noise, the method comprising:

receiving a target pixel signal in an image pipeline;

formulating a dynamic noise signal based at least in part on a value of the target pixel signal; and

controlling a noise reduction operation using the dynamic noise signal.

33. The method of claim 32, wherein controlling a noise reduction operation comprises combining the pixel signal with a noise adjustment signal.

34. The method of claim 32, wherein the dynamic noise signal is determined in part from information on target pixel noise and pixel processing system noise.

35. The method of claim 32, wherein the noise reduction operation comprises controlling an amount of blurring introduced by a color demosaicing operation in accordance with the dynamic noise signal.

36. An imaging device for processing pixel signals to reduce noise, the device comprising:

a pixel processing unit for processing a received target pixel signal;

a noise unit for providing a dynamic noise signal; and

a pixel processing control unit for controlling the pixel processing unit in accordance with a relationship of at least the target pixel signal to the dynamic noise signal.

37. The imaging device of claim 36, wherein the dynamic noise signal is determined in part from information on target pixel noise and pixel processing system noise.

38. The imaging device of claim 37, wherein:

the noise unit comprises a noise model unit for determining a dynamic noise threshold based on at least the first target pixel signal and a noise model; and

wherein the pixel processing unit comprises;

an aperture correction unit for;

comparing a subtractive combination of the first target pixel signal and at least one neighboring pixel signal to the dynamic noise threshold, and

controlling a noise reduction operation when a level of the subtractive combination exceeds a level of the dynamic noise threshold.

39. The imaging device of claim 38, wherein the aperture correction unit adds an adjusted amount to a level of the first target pixel signal which depends on the amount by which the level of the subtractive combination exceeds the level of the dynamic noise threshold.

40. The imaging device of claim 39, wherein the noise model unit is configured to determine the dynamic noise threshold (T_AP) according to:

T_AP=Y_Pixel*k_N+T_N0,

where Y_Pixel is a luminance component of the target pixel signal, k_N is a pixel model noise slope, and T_N0 is a noise floor offset.

41. The imaging device of claim 40, wherein the aperture correction unit is further configured to generate a noise adjustment signal (Y_AP) according to:

Y_d=Y_Pixel−

Y_Surround, and

Y_AP=(|Y_d|−

T_AP)*k_AP*sgn(Y_d),

where;

Y_Pixel is a luminance component of the target pixel signal,

Y_surround is a luminance component of the at least one neighboring pixel,

Y_d is an intermediate value representing the difference between Y_Pixel and Y_Surround,

T_AP is the dynamic noise signal,

k_AP is an aperture correction gain, and

sgn(Y_d) is a value of 1 or −

1 based on the sign of Y_d.

42. The imaging device of claim 39, wherein the noise model unit is configured to determine the dynamic noise threshold (T_AP) according to:

T_AP=Y_pixel*k_Npixel+Y_surround*k_Nsurround+T_N0,

where Y_Pixel is a luminance component of the target pixel signal, Y_surround is a luminance component of the at least one neighboring pixel, k_Npixel is a noise estimate for the target pixel, k_Nsurround is a noise estimate for the at least one neighboring pixel, and T_N0 is a noise floor offset.

43. The imaging device of claim 42, wherein the aperture correction unit is further configured to generate a noise adjustment signal (YAP) according to:

Y_d=Y_Pixel−

Y_Surround, and

Y_AP=(|Y_d|−

T_AP)*k_AP*sgn(Y_d),

where;

Y_Pixel is a luminance component of the target pixel signal,

Y_surround is a luminance component of the at least one neighboring pixel,

Y_d is an intermediate value representing the difference between Y_Pixel and Y_Surround,

T_AP is the dynamic noise signal,

k_AP is an aperture correction gain, and

sgn(Y_d) is a value of 1 or −

1 based on the sign of Y_d, and where the noise adjustment signal is applied to Y_pixel.

44. The imaging device of claim 38, wherein the aperture correction unit is configured to form a noise corrected aperture correcting signal in accordance with a luminance value of the target pixel signal and a luminance value of the at least one neighboring pixel signal, a magnitude of the aperture correction signal being determined by an amount by which a subtractive combination of the first target pixel signal luminance value and the neighboring pixel signal luminance value exceeds the dynamic noise threshold.

45. The imaging device of claim 44, wherein the aperture correction signal is a luminance signal which is combined with a luminance signal of the target pixel.

46. The imaging device of claim 39, further comprising:

a color correction unit for generating corrected color values for the target pixel, the corrected color values including a luminance value; and

a combining circuit for combining the adjustment amount to the luminance value of the color corrected target pixel.

47. The imaging device of claim 36, wherein:

the noise unit comprises a noise model unit for determining a dynamic edge threshold based on at least the target pixel signal and a noise model, and

the imaging device further comprises;

a demosaicing unit for receiving the target pixel value and at least one neighboring pixel signal, the demosaicing unit configured to control a demosaicing operation based on detecting an edge transition adjacent to a target pixel using the dynamic edge threshold.

48. The imaging device of claim 47, wherein the demosaicing unit is configured to:

if an edge transition is detected, the demosaicing unit is configured to generate red/green/blue (RGB) values based on the target pixel and pixels surrounding the target pixel on a same side of the edge transition as the target pixel; and

if an edge transition is not detected, the demosaicing unit is configured to generate the RGB values based on the target pixel and pixels surrounding the target pixel.

49. The imaging device of claim 47, wherein the demosaicing unit is configured to:

determine a first estimated area pixel value for an area surrounding the target pixel;

determine a second estimated area pixel value for an area surrounding at least one neighboring pixel;

compare the first and second estimated area pixel values; and

detect an edge transition based on the comparison.

50. The imaging device of claim 49, wherein the dynamic edge threshold (T_Edge) is determined according to:

T_Edge=P_PixelA*k_N+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

k_N is a coefficient making up a linear noise model, and

T_N0 is a noise floor offset.

51. The imaging device of claim 50, wherein the demosaicing unit is configured to detect an the edge transition according to:

if T_edge<

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is present; and

if T_edge≧

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is not present,

where P_PixelB is a pixel value for the neighboring pixel signal.

52. The imaging device of claim 49, wherein the dynamic edge threshold (T_Edge) is determined according to:

T_Edge=P_PixelA*k_{N A}+P_PixelB*k_{N B}+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

P_PixelB is a pixel value for the neighboring pixel signal,

k_{N A} and k_{N B} are coefficients making up a linear noise model, and

T_N0 is a noise floor offset.

53. The imaging device of claim 52, wherein the demosaicing unit is configured to detect an edge transition according to:

if T_edge<

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is present; and

if T_edge≧

|P_PixelA−

P_PixelB|, then it is determined that an edge transition is not present.

54. The imaging device of claim 49, wherein the dynamic edge threshold (T_Edge) is determined according to:

T_edge²=P_PixelA²*k_{N A2}+P_PixelA*k_{N A1}+T_N0,

where;

P_PixelA is a pixel value for the target pixel signal,

k_{N A1} and k_{N A2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

55. The imaging device of claim 54, wherein the demosaicing unit is configured to detect an edge transition according to:

if T_edge²<

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is not present,

where P_PixelB is a pixel value for the neighboring pixel signal.

56. The imaging device of claim 49, wherein the dynamic edge threshold (T_Edge) is determined according to:

T_edge²=P_PixelA²*k_{N A2}+P_PixelA*k_{N A1}+P_PixelB²*k_{N B2}+P_PixelB*P_PixelB²*k_{N B2}+P_PixelB*k_{N B1}+T_N0;

where;

P_PixelA is a pixel value for the target pixel signal,

P_PixelB is a pixel value for the neighboring pixel signal,

k_{N A1}, k_{N A2}, k_{N B1}, and k_{N B2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

57. The imaging device of claim 56, wherein the demosaicing unit is configured to detect an edge transition according to:

if T_edge²<

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_PixelA−

P_PixelB)², then it is determined that an edge transition is not present.

58. The imaging device of claim 49, wherein the dynamic edge threshold (T_Edge) is determined according to:

T_edge²=P_AreaA²*k_{N A2}+P_AreaA*k_{N A1}+P_AreaB²*k_{N B2}+P_AreaB*P_AreaB²*k_{N B2}+P_AreaB*k_{N B1}+T_N0;

P_AreaA(x,y)=avg(P_i,j|P_i,j adjacent to pixel (x,y) and P_i,j on side A); and

P_AreaB(X,y)=avg(P_i,j|P_i,j adjacent to pixel (x,y) and P_i,j on side B),

where;

P_AreaA is an estimate for area brightness for an area defined by a target pixel A at a location (x,y) and selected pixels surrounding the target pixel A,

P_AreaB is an estimate for area brightness for an area defined by a neighboring pixel B on the other side of the edge being tested,

i and j are coordinates for a central target pixel A,

avg is an averaging function,

k_{N A1}, k_{N A2}, k_{N B1}, and k_{N B2} are coefficients making up a quadratic noise model, and

T_N0 is a noise floor offset.

59. The imaging device of claim 58, wherein the demosaicing unit is configured to detect an edge transition according to:

if T_edge²<

(P_AreaA−

P_AreaB)², then it is determined that an edge transition is present; and

if T_edge²≧

(P_AreaA−

P_AreaB)², then it is determined that an edge transition is not present.

60. A device for processing pixel signals, the device comprising:

a noise model unit for determining a dynamic noise signal based on at least the first target pixel signal and a noise model; and

a demosaicing unit configured to receive the dynamic edge from the noise model unit, generate red/green/blue pixel values using the target pixel and neighboring pixels based on a relationship of at least the target pixel signal and a neighboring pixel to the dynamic edge threshold.

61. The device of claim 60, wherein the dynamic edge threshold is at least in part determined from information on pixel noise and pixel processing system noise.

62. A device for processing pixel signals, the device comprising:

a noise model unit for determining a dynamic noise threshold based on at least a first target pixel signal and a noise model;

an aperture correction unit for;

comparing a combined subtractive value of the target pixel signal and a neighboring pixel signal to the dynamic noise threshold, and

controlling an aperture correction operation based on the result of the comparison.

63. The device of claim 62, wherein the noise reduction operation further comprises increasing the first target pixel signal level by an amount dependent on the amount by which the subtractive combination exceeds the dynamic noise threshold.

64. A camera system comprising:

a lens; and

an imaging device having a pixel array for receiving an image through the lens, and for generating and processing pixel signals, the imaging device further comprising a threshold unit for providing a dynamic noise threshold.

65. The camera system of claim 64, further comprising a pixel processing unit for processing a received target pixel signal in accordance with the dynamic noise threshold.

66. The camera system of claim 65, further comprising:

a noise model unit for determining a dynamic noise threshold based on at least the first target pixel signal and a noise model; and

an aperture correction unit configured to;

compare a subtractive combination of at least the target pixel signal and a neighboring pixel signal to the dynamic noise threshold, and

control a noise reduction operation based on the result of the comparison.

67. The camera system of claim 65, further comprising:

a color correction unit for generating corrected color values, the corrected color values comprising a second target pixel signal; and

a combining circuit for combining the noise reduction processed first target pixel signal with the second target pixel signal.

68. The camera system of claim 65, further comprising:

a noise model unit for determining a dynamic edge threshold based on at least the first target pixel component and a noise model; and

a demosaicing unit for receiving the target pixel value and at least one neighboring pixel signal, the demosaicing unit configured to generate red/green/blue (RGB) values for the target pixel based on a relationship of at least the target pixel signal and the dynamic edge threshold.

69. A camera system comprising:

a lens; and

an imaging device having a pixel array for receiving an image through the lens, and for generating and processing pixel signals, the imaging device further comprising;

an image pipeline for receiving pixel signals, the image pipeline comprising;

a threshold unit for providing a dynamic noise threshold; and

a pixel processing unit for processing a received target pixel signal in accordance with the dynamic noise threshold.