Calculating interpolation errors for interpolation edge detection

• US 7,961,232 B2
• Filed: 01/21/2009
• Issued: 06/14/2011
• Est. Priority Date: 01/05/2004
• Status: Active Grant

• Current Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Original Assignee: Lifesize, Inc. (Enghouse Systems Limited)
• Inventors: Muresan, Darian
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): Hsu; Amy
• Application Number:
  US12/356,640
• Publication Number:
  US 20090147109A1
• Time in Patent Office: 874 Days
• Field of Search: 348/280, 348/311, 348/246, 348/247
• US Class Current: 348/246
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

An edge directed demosaicing algorithm for determining an edge direction from an input color filter array (CFA) sampled image is disclosed. Aspects of the present invention include calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error. Interpolation errors are also calculated for the current missing green pixel in a North-South (NS) direction at known neighboring green pixels, and the NS interpolation errors are averaged to obtain a NS error. An EW or NS direction indicated by a minimum of the EW error and the NS error is then selected as the edge direction.

Claims

1. A computer-implemented method for determining from an input color filter array (CFA) sampled image, an edge direction, the method comprising:

a computer system implementing;

calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error, wherein averaging the EW interpolation errors comprises summing the EW interpolation errors and dividing a resultant sum by a number of EW interpolation errors summed together to obtain an EW error;

calculating for the current missing green pixel, interpolation errors in a North-South (NS) direction at known neighboring green pixels, and averaging the NS interpolation errors to obtain a NS error; and

selecting a direction indicated by a minimum of the EW error and the NS error as the edge direction;

wherein the edge direction is useable to interpolate pixels in an image, wherein the computer system is configured to display the image on a display of the computer system.

2. The method of claim 1 wherein the edge direction and the sampled image, which includes a green channel (G) of green pixels, a red channel (R) of red pixels, and a blue channel (B) of blue pixels, are used to interpolate missing green pixels at red and blue locations in the green channel by:

for the missing green pixel, interpolating a difference image comprising the G-B if the missing green pixel is in a blue location, or G-R if the missing green pixel is in a red location, in the edge direction;

in the blue channel, estimating missing blue pixels in green pixel locations using linear interpolation of the blue pixels in the blue channel in the edge direction; and

in the red channel, estimating the missing red pixels in green pixel locations using linear interpolation of the red pixels in the red channel in the selected direction, thereby providing an interpolated full green channel in which all missing green pixels have an interpolated value.

3. A non-transitory computer-readable storage medium comprising program instructions for determining from an input color filter array (CFA) sampled image, an edge direction, wherein the program instructions are computer-executable to implement a method for:

calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error, wherein averaging the EW interpolation errors comprises summing the EW interpolation errors and dividing a resultant sum by a number of EW interpolation errors summed together to obtain an EW error;

calculating for the current missing green pixel, interpolation errors in a North-South (NS) direction at known neighboring green pixels, and averaging the NS interpolation errors to obtain a NS error; and

selecting a direction indicated by a minimum of the EW error and the NS error as the edge direction.

4. The non-transitory computer-readable storage medium of claim 3 wherein the program instructions are further executable to implement a method for using the edge direction and the sampled image, which includes a green channel (G) of green pixels, a red channel (R) of red pixels, and a blue channel (B) of blue pixels, to interpolate missing green pixels at red and blue locations in the green channel by:

for the missing green pixel, interpolating a difference image comprising the G-B if the missing green pixel is in a blue location, or G-R if the missing green pixel is in a red location, in the edge direction;

in the blue channel, estimating missing blue pixels in green pixel locations using linear interpolation of the blue pixels in the blue channel in the edge direction; and

in the red channel, estimating the missing red pixels in green pixel locations using linear interpolation of the red pixels in the red channel in the edge direction, thereby providing an interpolated full green channel in which all missing green pixels have an interpolated value.

5. An electronic device, comprising:

an input color filter array (CFA) for filtering light at a time of image capture;

a charge coupled device (CCD) for sensing the filtered light to produce a sampled image;

a processor for processing the sampled image; and

an edge directed demosaicing algorithm executed by the processor for;

calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error, wherein averaging the EW interpolation errors comprises summing the EW interpolation errors to obtain an EW error without dividing a resultant sum by a number of EW interpolation errors summed together;

calculating for the current missing green pixel, interpolation errors in a North-South (NS) direction at known neighboring green pixels, and averaging the NS interpolation errors to obtain a NS error; and

selecting a direction indicated by a minimum of the EW error and the NS error as the edge direction.

6. The electronic device of claim 5 wherein the selected edge direction, the sampled image, which includes a green channel (G) of green pixels, a red channel (R) of red pixels, and a blue channel (B) of blue pixels, are used to interpolate missing green pixels at red and blue locations in the green channel by:

for the missing green pixel, interpolating a difference image comprising the G-B if the missing green pixel is in a blue location, or G-R if the missing green pixel is in a red location, in the selected edge direction;

in the blue channel, estimating missing blue pixels in green pixel locations using linear interpolation of the blue pixels in the blue channel in the selected edge direction; and

in the red channel, estimating the missing red pixels in green pixel locations using linear interpolation of the red pixels in the red channel in the selected direction, thereby providing an interpolated full green channel in which all missing green pixels have an interpolated value.

7. A non-transitory computer-readable storage medium, comprising program instructions, wherein the program instructions are computer-executable to determine an edge direction for a known plurality of pixels in a sampled image, in which each known pixel in the known plurality of pixels represents one of a first type pixel, a second type pixel, or a third type pixel, and wherein the edge direction is operable to be used as an interpolation direction to determine, for a current known second type pixel or third type pixel of the known plurality of pixels, an associated interpolated first type pixel, the program instructions being executable to:

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in an East-West (EW) direction of a known neighboring first type pixel, wherein the interpolation error in the EW direction corresponds to a sum of (a known first type pixel west of the known neighboring first type pixel and a known first type pixel east of the known neighboring first type pixel) minus two times the known neighboring first type pixel;

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in a North-South (NS) direction of a known neighboring first type pixel; and

select an EW direction for the edge direction if the interpolation error in the EW direction is less than the interpolation error in the NS direction or select a NS direction for the edge direction if the interpolation error in the NS direction is less than the interpolation error in the EW direction.

8. The non-transitory computer-readable storage medium of claim 7, wherein the first type pixel is a green pixel, the second type pixel is a blue pixel, and the third type pixel is a red pixel.

9. The non-transitory computer-readable storage medium of claim 7, wherein the interpolation error in the NS direction corresponds to a sum of (a known first type pixel north of the known neighboring first type pixel and a known first type pixel south of the known neighboring first type pixel) minus two times the known neighboring first type pixel.

10. The non-transitory computer-readable storage medium of claim 7,

wherein calculating an interpolation error in the EW direction for the current known pixel comprises calculating interpolation errors in the EW direction for at least two known neighboring first type pixels and summing the calculated interpolation errors in the EW direction to derive a cumulative interpolation error in the EW direction;

wherein calculating an interpolation error in the NS direction for the current known pixel comprises calculating interpolation errors in the NS direction for at least two known neighboring first type pixels and summing the calculated interpolation errors in the NS direction to derive a cumulative interpolation error in the NS direction; and

wherein selecting the edge direction comprises selecting an EW direction for the edge direction if the cumulative interpolation error in the EW direction is less than the cumulative interpolation error in the NS direction or selecting a NS direction for the edge direction if the cumulative interpolation error in the NS direction is less than the cumulative interpolation error in the EW direction.

11. The non-transitory computer-readable storage medium of claim 7,

wherein calculating an interpolation error in the EW direction for the known current pixel comprises calculating interpolation errors in the EW direction for at least two known neighboring first type pixels and averaging the calculated interpolation errors in the EW direction to derive an averaged interpolation error in the EW direction;

wherein calculating an interpolation error in the NS direction for the known current pixel comprises calculating interpolation errors in the NS direction for at least two known neighboring first type pixels and averaging the calculated interpolation errors in the NS direction to derive an averaged interpolation error in the NS direction; and

wherein selecting the edge direction comprises selecting an EW direction for the edge direction if the averaged interpolation error in the EW direction is less than the averaged interpolation error in the NS direction or selecting a NS direction for the edge direction if the averaged interpolation error in the NS direction is less than the averaged interpolation error in the EW direction.

12. A computer-implemented method for determining from an input color filter array (CFA) sampled image, an edge direction, the method comprising:

a computer system implementing;

calculating for a current missing green pixel, interpolation errors in an East-West (EW) direction at known neighboring green pixels, and averaging the EW interpolation errors to obtain an EW error, wherein averaging the EW interpolation errors comprises summing the EW interpolation errors to obtain an EW error without dividing a resultant sum by a number of EW interpolation errors summed together;

calculating for the current missing green pixel, interpolation errors in a North-South (NS) direction at known neighboring green pixels, and averaging the NS interpolation errors to obtain a NS error; and

selecting a direction indicated by a minimum of the EW error and the NS error as the edge direction;

wherein the edge direction is useable to interpolate pixels in an image, wherein the computer system is configured to display the image on a display of the computer system.

13. A non-transitory computer-readable storage medium, comprising program instructions, wherein the program instructions are computer-executable to determine an edge direction for a known plurality of pixels in a sampled image, in which each known pixel in the known plurality of pixels represents one of a first type pixel, a second type pixel, or a third type pixel, and wherein the edge direction is operable to be used as an interpolation direction to determine, for a current known second type pixel or third type pixel of the known plurality of pixels, an associated interpolated first type pixel, the program instructions being executable to:

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in an East-West (EW) direction of a known neighboring first type pixel;

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in a North-South (NS) direction of a known neighboring first type pixel, wherein the interpolation error in the NS direction corresponds to a sum of (a known first type pixel north of the known neighboring first type pixel and a known first type pixel south of the known neighboring first type pixel) minus two times the known neighboring first type pixel; and

select an EW direction for the edge direction if the interpolation error in the EW direction is less than the interpolation error in the NS direction or select a NS direction for the edge direction if the interpolation error in the NS direction is less than the interpolation error in the EW direction.

14. A non-transitory computer-readable storage medium, comprising program instructions, wherein the program instructions are computer-executable to determine an edge direction for a known plurality of pixels in a sampled image, in which each known pixel in the known plurality of pixels represents one of a first type pixel, a second type pixel, or a third type pixel, and wherein the edge direction is operable to be used as an interpolation direction to determine, for a current known second type pixel or third type pixel of the known plurality of pixels, an associated interpolated first type pixel, the program instructions being executable to:

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in an East-West (EW) direction of a known neighboring first type pixel, wherein calculating an interpolation error in the EW direction for the current known pixel comprises calculating interpolation errors in the EW direction for at least two known neighboring first type pixels and summing the calculated interpolation errors in the EW direction to derive a cumulative interpolation error in the EW direction;

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in a North-South (NS) direction of a known neighboring first type pixel, wherein calculating an interpolation error in the NS direction for the current known pixel comprises calculating interpolation errors in the NS direction for at least two known neighboring first type pixels and summing the calculated interpolation errors in the NS direction to derive a cumulative interpolation error in the NS direction; and

select an EW direction for the edge direction if the interpolation error in the EW direction is less than the interpolation error in the NS direction or select a NS direction for the edge direction if the interpolation error in the NS direction is less than the interpolation error in the EW direction, wherein selecting the edge direction comprises selecting an EW direction for the edge direction if the cumulative interpolation error in the EW direction is less than the cumulative interpolation error in the NS direction or selecting a NS direction for the edge direction if the cumulative interpolation error in the NS direction is less than the cumulative interpolation error in the EW direction.

15. A non-transitory computer-readable storage medium, comprising program instructions, wherein the program instructions are computer-executable to determine an edge direction for a known plurality of pixels in a sampled image, in which each known pixel in the known plurality of pixels represents one of a first type pixel, a second type pixel, or a third type pixel, and wherein the edge direction is operable to be used as an interpolation direction to determine, for a current known second type pixel or third type pixel of the known plurality of pixels, an associated interpolated first type pixel, the program instructions being executable to:

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in an East-West (EW) direction of a known neighboring first type pixel, wherein calculating an interpolation error in the EW direction for the known current pixel comprises calculating interpolation errors in the EW direction for at least two known neighboring first type pixels and averaging the calculated interpolation errors in the EW direction to derive an averaged interpolation error in the EW direction;

calculate, for the current known pixel of the known plurality of pixels, an interpolation error in a North-South (NS) direction of a known neighboring first type pixel, wherein calculating an interpolation error in the NS direction for the known current pixel comprises calculating interpolation errors in the NS direction for at least two known neighboring first type pixels and averaging the calculated interpolation errors in the NS direction to derive an averaged interpolation error in the NS direction; and

select an EW direction for the edge direction if the interpolation error in the EW direction is less than the interpolation error in the NS direction or select a NS direction for the edge direction if the interpolation error in the NS direction is less than the interpolation error in the EW direction, wherein selecting the edge direction comprises selecting an EW direction for the edge direction if the averaged interpolation error in the EW direction is less than the averaged interpolation error in the NS direction or selecting a NS direction for the edge direction if the averaged interpolation error in the NS direction is less than the averaged interpolation error in the EW direction.