System and method for image demosaicing

• US 8,340,407 B2
• Filed: 01/14/2009
• Issued: 12/25/2012
• Est. Priority Date: 01/14/2009
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Kalman, Mark
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US12/353,539
• Publication Number:
  US 20100177961A1
• Time in Patent Office: 1,441 Days
• Field of Search: 382162-167, 382/260, 382/266, 358/525, 348/252, 348/272
• US Class Current: 382/162
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

A method for generating an interpolated output image, including generating a first interpolation array including RGB values, at least one of which is interpolated, based on a first interpolation region, and generating a first homogeneity array based on the first interpolation array including homogeneity values calculated based on a first homogeneity neighborhood having the same pixel locations as the first interpolation region. The method includes generating a second interpolation array based on a second interpolation region, and generating a second homogeneity array based on the second interpolation array and a second homogeneity neighborhood, where the second homogeneity neighborhood includes the same pixel locations as the second interpolation region. The method includes determining, for a first pixel location, that the homogeneity value in the first homogeneity array is greater than the homogeneity value in the second homogeneity array, and outputting the RGB color values in the first interpolation array.

Claims

1. A computer-implemented method for generating an interpolated output image, comprising:

generating a first interpolation array, using one or more processors, for a portion of an input image based on a first interpolation region, wherein each pixel location in the portion includes a red color value, a green color value, or a blue color value, and the first interpolation array includes, at each pixel location in the portion, red, green, and blue (RGB) color values, at least one of which is interpolated;

generating a second interpolation array based on a second interpolation region and including, at each pixel location in the portion, RGB color values, at least one of which is interpolated;

generating a first homogeneity array based on the first interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a first homogeneity neighborhood that includes the same pixel locations as the first interpolation region;

generating a second homogeneity array based on the second interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a second homogeneity neighborhood that includes the same pixel locations as the second interpolation region;

determining that, for a first pixel location in the portion, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the second homogeneity array; and

outputting to a display or to a memory the RGB color values at the first pixel location in the first interpolation array.

2. The method of claim 1, wherein each of the first interpolation region and the second interpolation region is selected from a group of at least three different interpolation regions.

3. The method of claim 2, wherein each of the first interpolation region and the second interpolation region comprises one of a horizontal interpolation region, a vertical interpolation region, a first diagonal interpolation region, a second diagonal interpolation region, and a bilinear interpolation region.

4. The method of claim 1, further comprising:

converting each of the RGB color values included in the first interpolation array to another color space that includes a luminance value, a first chrominance value, and a second chrominance value; and

calculating, at each pixel location in the portion, a set of luminance distances, wherein each luminance distance indicates a difference between the luminance value at the pixel location and the luminance value at another pixel location in the first homogeneity neighborhood associated with the pixel location.

5. The method of claim 4, further comprising calculating, at each pixel location in the portion, a set of color distances, wherein each color distance indicates a difference between the chrominance values at the pixel location and the chrominance values at another pixel location in the first homogeneity neighborhood associated with the pixel location.

6. The method of claim 5, further comprising computing a luminance threshold based on the calculated luminance distances that indicates a similarity of luminance values at two different pixel locations within one of the homogeneity neighborhoods.

7. The method of claim 6, further comprising computing a color threshold based on the calculated color distances that indicates a similarity of chrominance values at two different pixel locations within one of the homogeneity neighborhoods.

8. The method of claim 7, wherein generating the first homogeneity array comprises, at each pixel location in the portion, determining a number of pixel locations within the first homogeneity neighborhood having a luminance distance below the luminance threshold and a color distance below the color threshold.

9. The method of claim 1, further comprising:

generating a third homogeneity array based on a third interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a third homogeneity neighborhood having the same pixel locations as a third interpolation region used to generate the third interpolation array;

generating a fourth homogeneity array based on a fourth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fourth homogeneity neighborhood having the same pixel locations as a fourth interpolation region used to generate the fourth interpolation array; and

generating a fifth homogeneity array based on a fifth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fifth homogeneity neighborhood having the same pixel locations as a fifth interpolation region used to generate the fifth interpolation array.

10. The method of claim 9, further comprising determining that, at the first pixel location, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the third homogeneity array, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fourth homogeneity array, and the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fifth homogeneity array.

11. The method of claim 1, further comprising generating a smoothed homogeneity array by applying a smoothing filter to the first homogeneity array, wherein the smoothed homogeneity array includes, at each pixel location in the portion, a smoothed homogeneity value.

12. The method of claim 11, further comprising generating a binary array based on the first interpolation array that includes, at each pixel location in the portion, a binary value indicating whether a red variance, a green variance, or a red variance is outside a particular threshold.

13. The method of claim 12, further comprising selecting, at each pixel location in the portion, either the homogeneity value from the homogeneity array corresponding to the pixel location or the smoothed homogeneity value from the smoothed homogeneity array corresponding to the pixel location based on the value in binary array at the pixel location.

14. The method of claim 1, wherein the portion is equivalent to the entire input image.

15. The method of claim 1, wherein each of the green color values in the first interpolation array that is interpolated is derived directly from the first interpolation region, and wherein each blue color value in the first interpolation array that is interpolated and each red color value in the first interpolation array that is interpolated is computed based on at least one of the green color values in the first interpolation array.

16. A computer-readable medium storing instructions that, when executed by a processor, cause the processor to generate an interpolated output image, by performing the steps of:

generating a first interpolation array for a portion of an input image based on a first interpolation region, wherein each pixel location in the portion includes a red color value, a green color value, or a blue color value, and the first interpolation array includes, at each pixel location in the portion, red, green, and blue (RGB) color values, at least one of which is interpolated;

generating a second interpolation array based on a second interpolation region and including, at each pixel location in the portion, RGB color values, at least one of which is interpolated;

generating a first homogeneity array based on the first interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a first homogeneity neighborhood that includes the same pixel locations as the first interpolation region;

generating a second homogeneity array based on the second interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a second homogeneity neighborhood that includes the same pixel locations as the second interpolation region;

determining that, for a first pixel location in the portion, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the second homogeneity array; and

outputting to a display or to a memory the RGB color values at the first pixel location in the first interpolation array.

17. The computer-readable medium of claim 16, wherein each of the first interpolation region and the second interpolation region is selected from a group of at least three different interpolation regions.

18. The computer-readable medium of claim 17, wherein each of the first interpolation region and the second interpolation region comprises one of a horizontal interpolation region, a vertical interpolation region, a first diagonal interpolation region, a second diagonal interpolation region, and a bilinear interpolation region.

19. The computer-readable medium of claim 16, further comprising:

converting each of the RGB color values included in the first interpolation array to another color space that includes a luminance value, a first chrominance value, and a second chrominance value; and

calculating, at each pixel location in the portion, a set of luminance distances, wherein each luminance distance indicates a difference between the luminance value at the pixel location and the luminance value at another pixel location in the first homogeneity neighborhood associated with the pixel location.

20. The computer-readable medium of claim 19, further comprising calculating, at each pixel location in the portion, a set of color distances, wherein each color distance indicates a difference between the chrominance values at the pixel location and the chrominance values at another pixel location in the first homogeneity neighborhood associated with the pixel location.

21. The computer-readable medium of claim 20, further comprising computing a luminance threshold based on the calculated luminance distances that indicates a similarity of luminance values at two different pixel locations within one of the homogeneity neighborhoods.

22. The computer-readable medium of claim 21, further comprising computing a color threshold based on the calculated color distances that indicates a similarity of chrominance values at two different pixel locations within one of the homogeneity neighborhoods.

23. The computer-readable medium of claim 22, wherein generating the first homogeneity array comprises, at each pixel location in the portion, determining a number of pixel locations within the first homogeneity neighborhood having a luminance distance below the luminance threshold and a color distance below the color threshold.

24. The computer-readable medium of claim 16, further comprising:

generating a third homogeneity array based on a third interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a third homogeneity neighborhood having the same pixel locations as a third interpolation region used to generate the third interpolation array;

generating a fourth homogeneity array based on a fourth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fourth homogeneity neighborhood having the same pixel locations as a fourth interpolation region used to generate the fourth interpolation array; and

generating a fifth homogeneity array based on a fifth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fifth homogeneity neighborhood having the same pixel locations as a fifth interpolation region used to generate the fifth interpolation array.

25. The computer-readable medium of claim 24, further comprising determining that, at the first pixel location, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the third homogeneity array, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fourth homogeneity array, and the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fifth homogeneity array.

26. The computer-readable medium of claim 16, further comprising generating a smoothed homogeneity array by applying a smoothing filter to the first homogeneity array, wherein the smoothed homogeneity array includes, at each pixel location in the portion, a smoothed homogeneity value.

27. The computer-readable medium of claim 26, further comprising generating a binary array based on the first interpolation array that includes, at each pixel location in the portion, a binary value indicating whether a red variance, a green variance, or a red variance is outside a particular threshold.

28. The computer-readable medium of claim 27, further comprising selecting, at each pixel location in the portion, either the homogeneity value from the homogeneity array corresponding to the pixel location or the smoothed homogeneity value from the smoothed homogeneity array corresponding to the pixel location based on the value in binary array at the pixel location.

29. The computer-readable medium of claim 16, wherein the portion is equivalent to the entire input image.

30. The computer-readable medium of claim 16, wherein each of the green color values in the first interpolation array that is interpolated is derived directly from the first interpolation region, and wherein each blue color value in the first interpolation array that is interpolated and each red color value in the first interpolation array that is interpolated is computed based on at least one of the green color values in the first interpolation array.

31. A system configured to generate an interpolated output image, the system comprising:

a memory; and

one or more processors coupled to the memory and configured to;

generate a first interpolation array for a portion of an input image based on a first interpolation region, wherein each pixel location in the portion includes a red color value, a green color value, or a blue color value, and the first interpolation array includes, at each pixel location in the portion, red, green, and blue (RGB) color values, at least one of which is interpolated,

generate a second interpolation array based on a second interpolation region and including, at each pixel location in the portion, RGB color values, at least one of which is interpolated,

generate a first homogeneity array based on the first interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a first homogeneity neighborhood that includes the same pixel locations as the first interpolation region,

generate a second homogeneity array based on the second interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a second homogeneity neighborhood that includes the same pixel locations as the second interpolation region,

determine that, for a first pixel location in the portion, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the second homogeneity array, and

output to a display or to a memory the RGB color values at the first pixel location in the first interpolation array.

32. The system of claim 31, wherein each of the first interpolation region and the second interpolation region is selected from a group of at least three different interpolation regions.

33. The system of claim 32, wherein each of the first interpolation region and the second interpolation region comprises one of a horizontal interpolation region, a vertical interpolation region, a first diagonal interpolation region, a second diagonal interpolation region, and a bilinear interpolation region.

34. The system of claim 31, wherein the one or more processors are further configured to:

convert each of the RGB color values included in the first interpolation array to another color space that includes a luminance value, a first chrominance value, and a second chrominance value; and

calculate, at each pixel location in the portion, a set of luminance distances, wherein each luminance distance indicates a difference between the luminance value at the pixel location and the luminance value at another pixel location in the first homogeneity neighborhood associated with the pixel location.

35. The system of claim 34, wherein the one or more processors are further configured to calculate, at each pixel location in the portion, a set of color distances, wherein each color distance indicates a difference between the chrominance values at the pixel location and the chrominance values at another pixel location in the first homogeneity neighborhood associated with the pixel location.

36. The system of claim 35, wherein the one or more processors are further configured to compute a luminance threshold based on the calculated luminance distances that indicates a similarity of luminance values at two different pixel locations within one of the homogeneity neighborhoods.

37. The system of claim 36, wherein the one or more processors are further configured to compute a color threshold based on the calculated color distances that indicates a similarity of chrominance values at two different pixel locations within one of the homogeneity neighborhoods.

38. The system of claim 37, wherein generating the first homogeneity array comprises, at each pixel location in the portion, determining a number of pixel locations within the first homogeneity neighborhood having a luminance distance below the luminance threshold and a color distance below the color threshold.

39. The system of claim 31, wherein the one or more processors are further configured to:

generate a third homogeneity array based on a third interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a third homogeneity neighborhood having the same pixel locations as a third interpolation region used to generate the third interpolation array;

generate a fourth homogeneity array based on a fourth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fourth homogeneity neighborhood having the same pixel locations as a fourth interpolation region used to generate the fourth interpolation array; and

generate a fifth homogeneity array based on a fifth interpolation array that includes, at each pixel location in the portion, a homogeneity value calculated based on a fifth homogeneity neighborhood having the same pixel locations as a fifth interpolation region used to generate the fifth interpolation array.

40. The system of claim 39, wherein the one or more processors are further configured to determine that, at the first pixel location, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the third homogeneity array, the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fourth homogeneity array, and the homogeneity value in the first homogeneity array is greater than the homogeneity value in the fifth homogeneity array.

41. The system of claim 31, wherein the one or more processors are further configured to generate a smoothed homogeneity array by applying a smoothing filter to the first homogeneity array, wherein the smoothed homogeneity array includes, at each pixel location in the portion, a smoothed homogeneity value.

42. The system of claim 31, wherein the one or more processors are further configured to generate a binary array based on the first interpolation array that includes, at each pixel location in the portion, a binary value indicating whether a red variance, a green variance, or a red variance is outside a particular threshold.

43. The system of claim 42, where the one or more processors are further configured to select, at each pixel location in the portion, either the homogeneity value from the homogeneity array corresponding to the pixel location or the smoothed homogeneity value from the smoothed homogeneity array corresponding to the pixel location based on the value in binary array at the pixel location.

44. The system of claim 31, wherein the portion is equivalent to the entire input image.

45. The system of claim 31, wherein each of the green color values in the first interpolation array that is interpolated is derived directly from the first interpolation region, and wherein each blue color value in the first interpolation array that is interpolated and each red color value in the first interpolation array that is interpolated is computed based on at least one of the green color values in the first interpolation array.