System and method of color interpolation

• US 6,781,626 B1
• Filed: 01/13/2000
• Issued: 08/24/2004
• Est. Priority Date: 01/13/2000
• Status: Expired due to Term

• Current Assignee: Biomorphic Vlsi, Inc.
• Original Assignee: Biomorphic Vlsi, Inc.
• Inventors: Wang, H. Taichi
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): Rosendale, Matthew L
• Application Number:
  US09/482,844
• Time in Patent Office: 1,685 Days
• Field of Search: 348/273, 348/280, 348/252, 348/278, 348/276, 348/277, 348/272, 348/222.1, 382/300, 382/173, 382/180, 382/199, 358/525
• US Class Current: 348/273
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours

Abstract

A system and method of processing data associated with an array of pixels arranged in two dimensions are disclosed. The data includes a value representative of an intensity of photoexposure in one of a plurality of distinct spectral regions for each pixel location in the array. For at least one pixel location being associated with a first one of the plurality of spectral regions, a direction for interpolation is selected based upon an intensity gradient at the pixel location and an intensity continuity bias representative of a trend in changes in intensity in a neighborhood in the array of pixels about the pixel location. An interpolation among values representative of an intensity of photoexposure at two or more neighboring pixel locations in at least one set of neighboring pixels aligned along the selected direction to provides a value associated with the at least one pixel location which is representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region.

Claims

1. A method of processing data associated with an array of pixels arranged in two dimensions, each pixel corresponding with one of a plurality of distinct spectral regions and being associated with a value representative of an intensity of photoexposure at a pixel location in the array, the method comprising:

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first column adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second column adjacent to the at least one pixel location, the neighboring pixels in the first column and the neighboring pixels of the second column being aligned along the selected direction, and the first and second columns being separated by the at least one pixel location.

2. The method of claim 1, the method further comprising determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first column and the second column.

3. A method of processing data associated with an array of pixels arranged in two dimensions, each pixel corresponding with one of a plurality of distinct spectral regions and being associated with a value representative of an intensity of photoexposure at a pixel location in the array, the method comprising:

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first row adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second row adjacent to the at least one pixel location, the neighboring pixels in the first row and the neighboring pixels of the second row being aligned along the selected direction, and the first and second rows being separated by the at least one pixel location.

4. The method of claim 3, the method further comprising determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first row and the second row.

5. A method of processing data associated with an array of pixels arranged in two dimensions, each pixel corresponding with one of a plurality of distinct spectral regions and being associated with a value representative of an intensity of photoexposure at a pixel location in the array, the method comprising:

maintaining a continuity measure at each pixel location representative of an intensity gradient in neighboring pixel locations based upon changes in values representative of an intensity of photoexposure in a selected one of the spectral regions in each of the two dimensions at about the pixel location;

determining a continuity bias as a weighted sum of continuity measures associated with the neighboring pixel locations;

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and the continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region at two or more neighboring pixel locations in at least one set of neighboring pixels, the neighboring pixel locations being aligned along the selected direction.

6. The method of claim 5, wherein the continuity bias represents a degree of continuity of intensity changes in pixels neighboring the at least one pixel location in a first one of the plurality of directions in the array and the step of selecting to interpolate in exactly one of the plurality of directions in the array further includes selecting to interpolate in the first direction unless the intensity gradient at the pixel location indicates a degree of change in a second one of the plurality of directions which overrides the continuity bias.

7. The method of claim 5, wherein the step of selecting to interpolate in exactly one of the plurality of interpolation directions includes selecting the interpolation direction from between two substantially orthogonal interpolation directions in the array.

8. The method of claim 5, wherein the step of selecting to interpolate in exactly on of the plurality of interpolation directions includes selecting the interpolation direction from among at least three interpolation directions in the array.

9. In a camera, the camera having an imaging array of pixels and an optical system for projecting an image onto the imaging array, the pixels in the imaging array being arranged in two dimensions, each pixel being at a pixel location in the array and corresponding with one of a plurality of distinct spectral regions providing a signal representative of an intensity of photoexposure at the pixel location, the improvement comprising:

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first column adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second column adjacent to the at least one pixel location, the neighboring pixels in the first column and the neighboring pixels of the second column being aligned along the selected direction, and the first and second columns being separated by the at least one pixel location.

10. The camera of claim 9, the improvement further including logic for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first column and the second column.

11. In a camera, the camera having an imaging array of pixels and an optical system for projecting an image onto the imaging array, the pixels in the imaging array being arranged in two dimensions, each pixel being at a pixel location in the array and corresponding with one of a plurality of distinct spectral regions providing a signal representative of an intensity of photoexposure at the pixel location, the improvement comprising:

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first row adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second row adjacent to the at least one pixel location, the neighboring pixels in the first row and the neighboring pixels of the second row being aligned along the selected direction, and the first and second rows being separated by the at least one pixel location.

12. The camera of 11, the improvement further comprising logic for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first row and the second row.

13. In a camera, the camera having an imaging array of pixels and an optical system for projecting an image onto the imaging array, the pixels in the imaging array being arranged in two dimensions, each pixel being at a pixel location in the array and corresponding with one of a plurality of distinct spectral regions providing a signal representative of an intensity of photoexposure at the pixel location, the improvement comprising:

logic for maintaining a continuity measure at each pixel location representative of an intensity gradient in neighboring pixel locations based upon changes in values representative of an intensity of photoexposure in a selected one of the spectral regions in each of the two dimensions at about the pixel location;

logic for determining a continuity bias as a weighted sum of continuity measures associated with the neighboring pixel locations;

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one of directions in the array based upon an intensity gradient at the pixel location and the continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region at two or more neighboring pixel locations in at least one set of neighboring pixels, the neighboring pixel locations being aligned along the selected direction.

14. The camera of claim 13, wherein the continuity bias represents a degree of continuity of intensity changes in pixels neighboring the at least one pixel location in a first one of the plurality of directions in the array and the step of selecting to interpolate in exactly one of the plurality of directions further includes selecting to interpolate in the first direction unless the intensity gradient at the pixel location indicates a degree of change in a second one of the plurality of directions which overrides the continuity bias.

15. The camera of claim 13, wherein the logic for selecting to interpolate in exactly one of the plurality of interpolation directions selects the interpolation direction from between two substantially orthogonal interpolation directions in the array.

16. The camera of claim 13, wherein the logic for selecting to interpolate in exactly one of the plurality of interpolation directions selects the interpolation direction from among at least three interpolation directions in the array.

17. A computer readable medium for use in conjunction with image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the computer readable medium including computer readable instructions encoded thereon for performing the following:

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first column adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second column adjacent to the at least one pixel location, the neighboring pixels in the first column and the neighboring pixels of the second column being aligned along the selected direction, and the first and second columns being separated by the at least one pixel location.

18. The computer readable medium of claim 17, the computer readable medium further including computer readable instructions encoded thereon for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first column and the second column.

19. A computer readable medium for use in conjunction with image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the computer readable medium including computer readable instructions encoded thereon for performing the following:

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first row adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second row adjacent to the at least one pixel location, the neighboring pixels in the first row and the neighboring pixels of the second row being aligned along the selected direction, and the first and second rows being separated by the at least one pixel location.

20. The computer readable medium of claim 19, the computer readable medium further including computer readable instructions encoded thereon for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first row and the second row.

21. A computer readable medium for use in conjunction with image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the computer readable medium including computer readable instructions encoded thereon for performing the following:

maintaining a continuity measure at each pixel location representative of an intensity gradient in neighboring pixel locations based upon changes in values representative of an intensity of photoexposure in a selected one of the spectral regions in each of the two dimensions at about the pixel location;

determining a continuity bias as a weighted sum of continuity measures associated with the neighboring pixel locations;

for at least one pixel location being associated with a first one of the plurality of spectral regions, selecting to interpolate in exactly one of a plurality of directions in the array based upon an intensity gradient at the pixel location and the continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region at two or more neighboring pixel locations in at least one set of neighboring pixels, the neighboring pixel locations being aligned along the selected direction.

22. The computer readable medium of claim 21, wherein the continuity bias represents a degree of continuity of intensity changes in pixels neighboring the at least one pixel location in a first one of the plurality of directions in the array and the step of selecting to interpolate in exactly one of the plurality of directions further includes selecting to interpolate in the first direction unless the intensity gradient at the pixel location indicates a degree of change in a second one of the plurality of directions which overrides the continuity bias.

23. The computer readable medium of claim 21, the computer readable medium further including computer readable instructions encoded thereon for selecting the interpolation direction from between two substantially orthogonal interpolation directions in the array.

24. The computer readable medium of claim 21, the computer readable medium further including computer readable instructions encoded thereon for selecting the interpolation direction from among at least three interpolation directions in the array.

25. An image processor for processing image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the image processor comprising:

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first column adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second column adjacent to the at least one pixel location, the neighboring pixels in the first column and the neighboring pixels of the second column being aligned along the selected direction, and the first and second columns being separated by the at least one pixel location.

26. The image processor of claim 25, the image processor further including logic for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first column and the second column.

27. An image processor for processing image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the image processor comprising:

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one direction of a plurality of directions in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels in a first row adjacent to the at least one pixel location, and interpolating among values representative of an intensity of photoexposure in the second spectral region in two or more neighboring pixels of a second row adjacent to the at least one pixel location, the neighboring pixels in the first row and the neighboring pixels of the second row being aligned along the selected direction, and the first and second rows being separated by the at least one pixel location.

28. The image processor claim 27, the image processor further comprising logic for determining the value representative of the intensity in the second spectral region based upon a weighted combination of the interpolation of values in the first row and the second row.

29. An image processor for processing image data collected at an imaging array of pixels arranged in two dimensions, the image data including for each pixel location in the array a value representative of an intensity of photoexposure at the pixel location in one of a plurality of distinct spectral regions, the image processor comprising:

logic for maintaining a continuity measure at each pixel location representative of an intensity gradient in neighboring pixel locations based upon changes in values representative of an intensity of photoexposure in a selected one of the spectral regions in each of the two dimensions at about the pixel location;

logic for determining a continuity bias as a weighted sum of continuity measures associated with the neighboring pixel locations;

logic for selecting to interpolate, for at least one pixel location being associated with a first one of the plurality of spectral regions, in exactly one direction in the array based upon an intensity gradient at the pixel location and a continuity bias representative of a trend in changes of intensity in pixels neighboring the at least one pixel location; and

logic for determining a value representative of an intensity in a second one of the plurality of spectral regions distinct from the first spectral region to be associated with the at least one pixel location by interpolating among values representative of an intensity of photoexposure in the second spectral region at two or more neighboring pixel locations in at least one set of neighboring pixels, the neighboring pixel locations being aligned alone the selected direction.

30. The image processor of claim 29, wherein the continuity bias represents a degree of continuity of intensity changes in pixels neighboring the at least one pixel location in a first one of the plurality of directions and the step of selecting to interpolate in exactly one of the plurality of directions includes selecting to interpolate in the first direction unless the intensity gradient at the pixel location indicates a degree of change in a second one of the plurality of directions which overrides the continuity bias.

31. The image processor of claim 29, wherein the logic for selecting to interpolate in exactly one of the plurality of interpolation directions selects the interpolation direction from between two substantially orthogonal interpolation directions in the array.

32. The image processor of claim 29, wherein the logic for selecting to interpolate in exactly one of the plurality of interpolation directions selects the interpolation direction from among at least three interpolation directions in the array.