Methods and apparatuses that reduce noise in image signals

• US 20080298708A1
• Filed: 05/31/2007
• Published: 12/04/2008
• Est. Priority Date: 05/31/2007
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Micron Technology, Inc.
• Inventors: Ovsiannikov, Ilia, Jerdev, Dmitri
• Granted Patent: US 7,876,957 B2
• Time in Patent Office: Days
• US Class Current: 382/261
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Methods and apparatuses for noise reduction of imager pixels signals. A weighted combination based on the presence of edges of two calculated demosaiced signals is used to produce a noise reduced signal. The noise reduced signal may be sharpened based on a calculated luminance of the two demosaiced signals.

Claims

1. A method of adjusting a color imaging pixel signal of an imaging pixel from a pixel array, the method comprising:

determining a first demosaiced value of the imaging pixel signal using a first demosaicing method;

determining a second demosaiced value of the imaging pixel signal using a second demosaicing method employing noise reduction;

determining an edge value; and

determining a noise reduced value of the imaging pixel signal as a function of the first demosaiced value, the second demosaiced value, and the edge value.

2. The method of claim 1, wherein the step of determining the edge value comprises detecting the presence of an edge in at least a portion of the pixel array at an area containing an imaging pixel associated with the signal being adjusted.

3. The method of claim 1, wherein the step of determining the edge value comprises comparing signals of pairs of pixels neighboring the imaging pixel and selecting the maximum difference between pixel signals for one of the pairs.

4. The method of claim 1, wherein the noise reduced value is determined as a weighted combination of the first demosaiced value and the second demosaiced value, wherein the weight is determined as a function of the edge value and at least one of the first demosaiced value and the second demosaiced value.

5. The method of claim 4, wherein the weight is further determined as a function of at least one of a luminance of the first demosaiced value and a luminance of the second demosaiced value.

6. The method of claim 4, wherein:

the second demosaicing method is based on a pixel averaging;

the first demosaiced value has a first associated weight;

the second demosaiced value has a second associated weight; and

the noise reduced value is determined as a weighted sum of the first demosaiced value and the second demosaiced value, the first and second weights being determined such that more or less pixel averaging results.

7. The method of claim 4, wherein the noise reduced value is determined according to:

R_NR[i,j]=w_FFNR[i,j]·

R_D[i,j]+(1−

w_FFNR[i,j])·

R_FFNR[i,j],

G_NR[i,j]=w_FFNR[i,j]·

G_D[i,j]+(1−

w_FFNR[i,j])·

G_FFNR[i,j], and

B_NR[i,j]=w_FFNR[i,j]·

B_D[i,j]+(1−

w_FFNR[i,j])·

B_FFNR[i,j]

where i,j denotes a pixel position;

R_NR[i,j], G_NR[i,j], and B_NR[i,j] are components of the noise reduced value;

w_FFNR[i,j] is the weight;

R_D[i,j], G_D[i,j], and B_D[i,j] are components of the first demosaiced value; and

where R_FFNR[i,j], G_FFNR[i,j], and B_FFNR[i,j] are components of the second demosaiced value.

8. The method of claim 4, wherein the weight is determined according to:

9. The method of claim 1, further comprising determining a sharpened value of the imaging pixel signal as a function of the noise reduced value and at least one of a luminance of the first demosaiced value and a luminance of the second demosaiced value.

10. The method of claim 9, wherein the sharpened value is determined according to:

R_i,j=R_NR[i,j]+G_A[i,j]·

Δ

Y_i,j,

G_i,j=G_NR[i,j]+G_A[i,j]·

Δ

Y_i,j, and

B_i,j=B_NR[i,j]+G_A[i,j]·

Δ

Y_i,j

where i,j denotes a pixel position;

R_i,j, G_i,j, and B_i,j are components of the sharpened value;

R_NR[i,j], G_NR[i,j], and B_NR[i,j] are components of the noise reduced value;

G_A[i,j] is a sharpening gain which controls the amount of sharpening applied to the signal of the imaging pixel; and

Δ

Y_i,j is a luminance based aperture correction value.

11. The method of claim 10, wherein the luminance based aperture correction value is determined according to:

Δ

Y_i,j=max(|Y_D[i,j]−

Y_FFNR[i,j]|−

T_A[i,j,0)·

sgn(Y_D[i,j]−

Y_FFNR[i,j])

where Y_D[i,j] is a pixel luminance of the first demosaiced value;

Y_FFNR[i,j] is a pixel luminance of the second demosaiced value;

sgn(Y_D[i,j]−

Y_FFNR[i,j]) is the sign of Y_D[i,j]−

Y_FFNR[i,j];

T_A[i,j] is a minimum luminance difference value necessary to produce a sharpening effect as determined by;

T_A[i,j]=T_A,min[i,j]+G_A,T[i,j]·

Y_D[i,j]

where T_A,min[i,j] is the minimum value for T_A[i,j] and G_A,T[i,j] is a pixel noise model coefficient allowing for sharpening of brighter pixels.

12. The method of claim 1, wherein the second demosaic method is configured to:

select a kernel having a kernel size and a center pixel location for each viable color pattern for a kernel of the kernel size such that each selected kernel has a unique color pattern;

determine the color pixel configurations of the selected kernels for each color;

select a subset of pixels from the color pixel configurations for each selected kernel for each color; and

assign a pixel weight to each pixel within the subsets, the pixel weight being derived as a function of at least one of;

the color the pixels, an edge crossing the kernels, a shape of the subset of pixels, a location of the pixel within the kernel, a zipper effect, and pixel noise.

13. The method of claim 12, wherein the step of determining the second demosaiced value further comprises:

selecting a kernel wherein the imaging pixel is located at the center pixel location; and

determining the demosaiced signal of the imaging pixel signal as a function of at least a subset of the signals of the kernel and their respective pixel weights.

14. A method of adjusting a color imaging pixel signal of an imaging pixel from a pixel array, the method comprising:

determining a first demosaiced value of the imaging pixel signal using a first demosaicing method;

determining a second demosaiced value of the imaging pixel signal using a second demosaicing method employing noise reduction and using pixel signal averaging;

determining an edge value by detecting the presence of an edge in at least a portion of the array at an area containing an imaging pixel associated with the signal being adjusted;

determining a weight as a function of the edge value, a luminance of the second demosaiced value, and at least a constant;

determining a noise reduced value of the imaging pixel signal as a weighted combination of the first demosaiced value and the second demosaiced value, wherein the weight determines the amount of pixel averaging by weighting the demosaiced values;

determining a minimum luminance value necessary to produce a sharpening effect on the noise reduced value as a function of the luminance of the first demosaiced value;

determining a high-frequency sharpening signal value as a function of the luminance of the first demosaiced value, the second demosaiced value, and the minimum luminance value; and

determining a sharpened value as a function of the noise reduced value and the sharpening signal value.

15. An imaging device comprising:

an imaging sensor comprising an array of imaging pixels;

a signal processing circuit for adjusting signals from the array, the signal processing circuit comprising;

a first demosaicing module configured to determine a first demosaiced value of an imaging pixel signal using a first demosaicing method;

a second demosaicing module configured to determine a second demosaiced value of the imaging pixel signal using a second demosaicing method employing noise reduction;

an edge detection module configured to determine an edge value by detecting the presence of an edge in at least a portion of the array at an area containing an imaging pixel associated with the signal being adjusted; and

a noise reduction module configured to determine a noise reduced value of the imaging pixel signal as a function of the first demosaiced value, the second demosaiced value, and the edge value.

16. The imaging device of claim 15, wherein the edge detection module is configured to compare signals of pairs of pixels neighboring the imaging pixel and select the maximum difference between pixel signals for one of the pairs.

17. The imaging device of claim 15, wherein the noise reduction module is further configured to determine the noise reduced value as a weighted combination of the first demosaiced value and the second demosaiced value, wherein the weight is determined as a function of the edge value and at least one of the first demosaiced value and the second demosaiced value.

18. The imaging device of claim 17, wherein the weight is further determined as a function of at least one of a luminance of the first demosaiced value and a luminance of the second demosaiced value.

19. The imaging device of claim 17, wherein:

the second demosaicing method is based on a pixel averaging;

the first demosaiced value has a first associated weight;

the second demosaiced value has a second associated weight; and

the noise reduced value module is configured to determine the noise reduced value as a weighted sum of the first demosaiced value and the second demosaiced value, the first and second weights being determined such that more or less pixel averaging results.

20. The imaging device of claim 15, further comprising a sharpening module configured to determine a sharpened value of the imaging pixel signal as a function of the noise reduced value and at least one of a luminance of the first demosaiced value and a luminance of the second demosaiced value.

21. The imaging device of claim 15, wherein the imaging device is provided in a camera system.

22. A method of demosaicing a color image signal of an imaging pixel, the method comprising:

selecting a kernel having a kernel size and a center pixel location for each viable color pattern for a kernel of the kernel size such that each selected kernel has a unique color pattern;

determining the color pixel configurations of the selected kernels for each color;

selecting a subset of pixels from the color pixel configurations for each selected kernel for each color; and

assigning a pixel weight to each pixel within the subsets, the pixel weight being derived as a function of at least one of;

the color the pixels, an edge crossing the kernels, a shape of the subset of pixels, a location of the pixel within the kernel, a zipper effect, and pixel noise.

23. The method of claim 22, wherein the step of determining the second demosaiced value further comprises:

selecting a kernel wherein the imaging pixel is located at the center pixel location; and

determining the demosaiced signal of the imaging pixel signal as a function of at least a subset of the signals of the kernel and their respective pixel weights.

24. The method of claim 23, wherein the second demosaiced value has a green component, a red component, and a blue component.

25. The method of claim 24, wherein the green component of the second demosaiced value is determined according to:

G_FFNR[i,j]=a·

(p<sub>i−

1,j−

1</sub>+p_i+1,j+1+p<sub>i+1,j−

1</sub>+p<sub>i−

1,j+1</sub>)+b·

p_i,j

where G_FFNR[i,j] is the green component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

a and b are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are green pixel signals for one of the selected subset of pixels.

26. The method of claim 24, wherein the green component of the second demosaiced value is determined according to:

G_FFNR[i,j]=c·

(p<sub>i,J−

1</sub>+p_i,j+1+p_i+1,j+p<sub>i−

,j</sub>)

where G_FFNR[i,j] is the green component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

c is a pixel weight;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are green signals for one of the selected subset of pixels.

27. The method of claim 24, wherein the red component of the second demosaiced value is determined according to:

R_FFNR[i,j]=h·

(p<sub>i−

1,j−

1</sub>+p<sub>i−

1,j+1</sub>+p<sub>i+1,j−

1</sub>+p_i+1,j+1)

where R_FFNR[i,j] is the red component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

h is a pixel weight;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are red signals for one of the selected subset of pixels.

28. The method of claim 24, wherein the red component of the second demosaiced value is determined according to:

R_FFNR[i,j]=d·

(p<sub>i−

1,j−

2</sub>+p<sub>i−

1,j+2</sub>+p<sub>i+1,j−

2</sub>+p_i+1,j+2)+e·

(p<sub>i−

1,j</sub>+p_i+1,j)

where R_FFNR[i,j] is the red component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

d and e are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are red signals for one of the selected subset of pixels.

29. The method of claim 24, wherein the red component of the second demosaiced value is determined according to:

R_FFNR[i,j]=d·

(p<sub>i−

2,j−

1</sub>+p<sub>i−

2,j+1</sub>+p<sub>i+2,j−

1</sub>+p_i+2,j+1)+e·

(p<sub>i,j−

1</sub>+p_i,j+1)

where R_FFNR[i,j] is the red component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

d and e are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are red signals for one of the selected subset of pixels.

30. The method of claim 24, wherein the red component of the second demosaiced value is determined according to:

R_FFNR[i,j]=g·

(p<sub>i,j−

2</sub>+p_i,j+2+p<sub>i−

2,j</sub>+p_i+2,j)+f·

p_i,j

where R_FFNR[i,j] is the red component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

f and g are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are red signals for one of the selected subset of pixels.

31. The method of claim 24, wherein the blue component of the second demosaiced value is determined according to:

B_FFNR[i,j]=h·

(p<sub>i−

1,j−

1</sub>+p<sub>i−

1,j+1</sub>+p<sub>i+1,j−

1</sub>+p_i+1,j+1)

where B_FFNR[i,j] is the blue component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

h is a pixel weight;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are blue signals for one of the selected subset of pixels.

32. The method of claim 24, wherein the blue component of the second demosaiced value is determined according to:

B_FFNR[i,j]=d·

(p<sub>i−

1,j−

2</sub>+p<sub>i−

1,j+2</sub>+p<sub>i+1,j−

2</sub>+p_i+1,j+2)+e·

(p<sub>i−

1,j</sub>+p_i+1,j)

where B_FFNR[i,j] is the blue component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

d and e are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are blue signals for one of the selected subset of pixels.

33. The method of claim 24, wherein the blue component of the second demosaiced value is determined according to:

B_FFNR[i,j]=d·

(p<sub>i−

2,j−

1</sub>+p<sub>i−

2,j+1</sub>+p<sub>i+2,j−

1</sub>+p_i+2,j+1)+e·

(p<sub>i,j−

1</sub>+p_i,j+1)

where B_FFNR[i,j] is the blue component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

d and e are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are blue signals for one of the selected subset of pixels.

34. The method of claim 24, wherein the blue component of the second demosaiced value is determined according to:

B_FFNR[i,j]=g·

(p<sub>i,j−

2</sub>p_i,j+2+p<sub>i−

2,j</sub>+p_i+2,j)+f·

p_i,j

where B_FFNR[i,j] is the blue component for the imaging pixel signal p_i,j;

i is a pixel row;

j is a pixel column;

an i,j notation denotes a pixel position within the selected kernel;

f and g are pixel weights;

p is a pixel signal value for a pixel located at a pixel position; and

all of the p values are blue signals for one of the selected subset of pixels.

35. The method of claim 22, wherein the pixel weight assigned to each pixel is selected to reduce at least one of a zipper effect and pixel noise.