Subtractive color processing system for digital imaging

• US 6,366,319 B1
• Filed: 07/03/1997
• Issued: 04/02/2002
• Est. Priority Date: 07/03/1997
• Status: Expired due to Term

• Current Assignee: DRS Training & Control Systems LLC (Finmeccanica SPA)
• Original Assignee: Phototronics Incorporated
• Inventors: Bills, Richard
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): MOE, AUNG SOE
• Application Number:
  US08/888,007
• Time in Patent Office: 1,734 Days
• Field of Search: 348/273, 348/275, 348/280, 348/266, 348/268, 348/222, 348/230, 348/267, 348/264, 348/211, 382/162, 382/166, 358/426, 358/505, 358/539, 250/208.1
• US Class Current: 348/273
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Improved methods and apparatus for digital focal plane array (FPA)—based color imaging include the step of sampling an image, using an FPA camera equipped with a color filter array (CFA) to obtain a value at each pixel position, thereby creating a set of values sampled on a per-pixel basis. Single-color cyan, yellow and white (CYW) color planes are generated from the set of sampled pixel values. Interpolates among and between the color plane values is executed to “fill in” the “gaps” in the individual color planes, and the interpolated CYW values from the single-color planes are transformed into RGB values which can be stored, transmitted, or utilized to drive a conventional video monitor or color printing device. Filtering and other post-interpolation steps are executed to reduce color aliasing, and the white plane is used to introduce high frequency content back into the filtered color image. In one practice of the invention, the CYW values are transformed into CIELAB color space values. The interpolated W plane is used to calculate L* values, and C, Y, and W planes are used to calculate a* and b* values. The a* and b* planes are filtered using a window with weighting coefficients, and the CIELAB color space values are then transformed to red, green, and blue color planes using a color look-up table (LUT). In this case, the luminance of each pixel is directly related to the W plane value at that pixel location.

Claims

1. A solid-state, digital color image capture and processing system for capturing a color image, generating a compressed set of digital values representative thereof, and reconstructing the image from the compressed set of digital values, the system comprising:

a multiple-pixel focal plane array (FPA) for sampling the image, said FPA having a set of pixels, each pixel being disposed along an image plane to sample a respective portion of the image, said FPA also having a color filter array disposed on said image plane, said color filter array having subtractive color filter elements of different predetermined colors, arranged such that at each pixel of the FPA, the FPA samples a digital value corresponding to a respective one of the different predetermined colors, whereby the set of digital values sampled by the complete set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete color planes for each of the different predetermined colors, the subtractive color filter elements being selected from the group consisting of any of (i) cyan, yellow and white, (ii) cyan, magenta and white, or (iii) yellow, magenta and white,

a memory element in electrical communication with the FPA to store the digital values sampled by the FPA, the set of sampled digital values stored in the memory element being representative of the color image, and

a digital image signal processing circuit adapted to receive the values stored in the memory element and to generate, from sampled digital values stored in the memory element, a decompressed set of values constituting a reconstructed digital representation of the sampled image, the subtractive color filter elements being arranged in a mosaic pattern consisting of 50% white pixels, 25% cyan pixels, and 25% yellow pixels, the digital image signal processing circuit comprising a circuit output at which the digital image signal processing circuit rapidly generates, from the sampled white values stored in the memory element, a half-resolution black and white image by selecting and transmitting only the sampled white values stored in the memory element.

2. A system according to claim 1, wherein the digital signal processing circuit further comprises

a red/green/blue (RGB) transforming processor in electrical communication with the output of the interpolating circuit, for converting the values constituting full cyan, yellow and white color planes corresponding to the sampled image into red, green and blue values constituting full red, green and blue planes representative of the sampled image.

3. A system according to claim 2, wherein the RGB transforming processor comprises

lookup tables which are addressable by the cyan, white and yellow values to generate red, green and blue values constituting full red, green and blue planes representative of the sampled image.

4. A system according to claim 3, wherein the digital image signal processing circuit further comprises

a white plane multiplication circuit, adapted to (i) receive the red, green and blue values output by the lookup tables, (ii) multiply each such value in the fully-generated red, green and blue planes by the corresponding white plane value for each respective pixel, normalized by the sum of the red, green and blue values at that pixel position, and (iii) adjust the luminance at each pixel so that it is equal to the white plane value at that pixel position.

5. A system according to claim 4, wherein the digital signal processing circuit further comprises:

a spatial filtering circuit interposed between the interpolator and the RGB transforming processor, in electrical communication with an output of the interpolator circuit, for reducing the spatial bandwidth of the cyan, yellow and white planes prior to transformation into RGB planes.

6. A system according to claim 5, wherein the spatial filtering circuit comprises

arithmetic logic circuitry to apply a two dimensional filtering window with selected weighting coefficients to the values of the respective cyan, yellow and white planes, thereby to reduce the spatial bandwidth of the cyan, yellow and white planes.

7. A system according to claim 4, wherein the digital signal processing circuit further comprises:

a spatial filtering circuit in electrical communication with an output of the RGB transforming processor, for reducing, following transformation into RGB planes, the spatial bandwidth of the RGB planes.

8. A system according to claim 1, wherein the memory element further comprises a non-volatile storage device to store the digital values sampled by the FPA.

9. A system according to claim 1, further comprising

telemetry link circuitry, in electrical communication with the memory element and in telemetric communication with the digital signal processing circuit, for transmitting the digital values sampled by the FPA to the digital signal processing circuit via a telemetry link.

10. A solid-state, digital color image capture and processing system for capturing a color image, generating a compressed set of digital values representative thereof, and reconstructing the image from the compressed set of digital values, the system comprising:

a multiple-pixel focal plane array (FPA) for sampling the image, said FPA having a set of pixels, each pixel being disposed along an image plane to sample a respective portion of the image, said FPA also having a color filter array disposed on said image plane, said color filter array having subtractive color filter elements of different predetermined colors, arranged such that at each pixel of the FPA, the FPA samples a digital value corresponding to a respective one of the different predetermined colors, whereby the set of digital values sampled by the complete set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete color planes for each of the different predetermined colors, the subtractive color filter elements being selected from the group consisting of any of (i) cyan, yellow and white, (ii) cyan, magenta and white, or (iii) yellow, magenta and white,

a memory element in electrical communication with the FPA to store the digital values sampled by the FPA, the set of sampled digital values stored in the memory element being representative of the color image,

a digital image signal processing circuit adapted to receive the values stored in the memory element, and to generate, from sampled digital values stored in the memory element, a decompressed set of values constituting a reconstructed digital representation of the sampled image, the digital signal processing circuit comprising;

a digital interpolating circuit which generates, from the sampled values stored in the memory element, a set of cyan, yellow and white values constituting complete cyan, yellow, and white color planes corresponding to the sampled image, the digital interpolating circuit comprising;

a circuit output at which the digital interpolating circuit rapidly generates, from the sampled white values stored in the memory element, a full resolution black and white image by interpolating only the sampled white values stored in the memory element.

11. A decoding circuit for generating a reconstructed color image from a compressed digital dataset representative of the color image, the compressed digital dataset being generated by a focal plane array (FPA) having a subtractive color filter array thereon, said color filter array having subtractive color filter elements of cyan, yellow and white, arranged such that at each pixel of the FPA, the FPA samples a digital cyan value, a digital yellow value, or a digital white value, whereby the set of digital cyan, yellow and white values sampled by the full set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete cyan, yellow and white color planes representative of the sampled image, the decoding circuit comprising:

a digital image signal processor adapted to receive the values generated by the FPA and to generate, from sampled digital values stored in a memory element, a set of values constituting a reconstructed digital representation of the sampled image, the digital image signal processor including;

a digital interpolating circuit which generates, from the sampled values stored in the memory element, values constituting a complete set of cyan, yellow and white color planes corresponding to the sampled image, and

a circuit output at which the digital image signal processor rapidly generates, from the sampled white values stored in the memory element, a half-resolution black and white image by selecting and transmitting only the sampled white values stored in the memory element.

12. The circuit of claim 11, wherein the digital signal processor further comprises

a red/green/blue (RGB) transforming processor in electrical communication with the output of the interpolating circuit, for converting the values constituting full cyan, yellow and white color planes corresponding to the sampled image into values constituting full red, green and blue planes representative of the sampled image.

13. The circuit of claim 12, wherein the RGB transforming processor comprises

lookup tables which are addressable by ones of the cyan, white and yellow values to generate red, green and blue values constituting full red, green and blue planes representative of the sampled image.

14. The circuit of claim 13, wherein the digital image signal processor further comprises

a white plane multiplication circuit, adapted to (i) receive the red, green and blue values output by the lookup tables, (ii) multiply each such value in the fully-generated red, green and blue planes by the corresponding white plane value for each respective pixel, normalized by a weighted sum of the red, green and blue values at that pixel position, and (iii) adjust the luminance at each pixel so that it is equal to the white plane value at that pixel position.

15. The circuit of claim 14, wherein the digital signal processor further comprises:

a spatial filtering circuit interposed between the interpolator and the RGB transforming processor, in electrical communication with an output of the interpolator circuit, for reducing the spatial bandwidth of the cyan, yellow and white planes prior to transformation into RGB planes.

16. The circuit of claim 15, wherein the spatial filtering circuit comprises

arithmetic logic circuitry to apply a two dimensional filtering window with selected weighting coefficients to the values of the respective cyan, yellow and white planes, thereby to reduce the spatial bandwidth of the cyan, yellow and white planes.

17. The circuit of claim 14, wherein the digital signal processor further comprises:

a spatial filtering circuit in electrical communication with an output of the RGB transforming processor, for reducing, following transformation into RGB planes, the spatial bandwidth of the RGB planes.

18. A decoding circuit for generating a reconstructed color image from a compressed digital dataset representative of the color image, the compressed digital dataset being generated by a focal plane array (FPA) having a subtractive color filter array thereon, said color filter array having subtractive color filter elements of cyan, yellow and white, arranged such that at each pixel of the FPA, the FPA samples a digital cyan value, a digital yellow value, or a digital white value, whereby the set of digital cyan, yellow and white values sampled by the full set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete cyan, yellow and white color planes representative of the sampled image, the decoding circuit comprising:

a digital image signal processor adapted to receive the values generated by the FPA and to generate, from sampled digital values stored in a memory element, a set of values constituting a reconstructed digital representation of the sampled image, the digital image signal processor including;

a digital interpolating circuit which generates, from the sampled values stored in the memory element, values constituting a complete set of cyan, yellow and white color planes corresponding to the sampled image, the digital interpolating circuit comprising;

a circuit output at which the interpolating circuit rapidly generates, from the sampled white values stored in the memory element, a full resolution black and white image by interpolating only the sampled white values stored in the memory element.

19. A method of capturing a color image, generating a compressed set of digital values representative thereof, and reconstructing the image from the compressed set of digital values, the method comprising the steps of:

providing a multiple-pixel focal plane array (FPA) for sampling the image, said FPA having a set of pixels, each pixel being disposed along an image plane to sample a respective portion of the image,

disposing a subtractive color filter array on an image plane of the FPA, said color filter array having subtractive color filter elements of different predetermined colors, the arrangement of subtractive color filter elements forming a subtractive filter mosaic in which subtractive color filter elements each overlie a respective pixel of the FPA,

configuring the subtractive color filter array to have subtractive color filter elements of cyan, yellow and white,

sampling the image with the multiple-pixel FPA upon which the subtractive color filter array is disposed, such that each pixel of the FPA samples a digital value corresponding to a respective one of the different predetermined colors, whereby the set of digital values sampled by the complete set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete color planes for each of the different predetermined colors,

storing the digital values sampled by the FPA in a memory element, the set of sampled digital values stored in the memory element being representative of the color image, and

generating, from the sampled digital values stored in the memory element, a decompressed set of values constituting a reconstructed digital representation of the sampled image,

said generating step including the step of utilizing a digital signal processing circuit,

the step of configuring the subtractive color filter array to have subtractive color filter elements of cyan, yellow and white including the step of configuring the subtractive color filter array to have a mosaic pattern consisting of 50% white pixels, 25% cyan pixels, and 25% yellow pixels,

the step of generating a decompressed set of values constituting a reconstructed digital representation of the sampled image comprising the step of interpolating across the set of sampled cyan, yellow and white values stored in the memory element to generate a set of cyan, yellow and white values constituting complete cyan, yellow, and white color planes corresponding to the sampled image,

the step of interpolating comprising;

generating, from the sampled white values stored in the memory element, a full resolution black and white image, by interpolating only the sampled white values stored in the memory element.

20. The method of claim 19, comprising the further step of

converting the cyan, yellow and white values constituting complete cyan, yellow and white color planes corresponding to the sampled image into red, green and blue values constituting complete red, green and blue planes representative of the sampled image.

21. The method of claim 20, wherein the step of converting the cyan, yellow and white values constituting complete cyan, yellow and white color planes into red, green and blue values comprises

addressing a lookup table with ones of the cyan, white and yellow values to generate red, green and blue values constituting complete full red, green and blue planes representative of the sampled image.

22. The method of claim 21, wherein the step of generating a decompressed set of values constituting a reconstructed digital representation of the sampled image further comprises the steps of

multiplying each respective red, green and blue value in the fully-generated red, green and blue planes by the corresponding white plane value for each respective pixel, normalized by the sum of the red, green and blue values at that pixel position, and

adjusting the luminance at each pixel so that it is equal to the white plane value at that pixel position.

23. The method of claim 22, wherein the step of generating a decompressed set of values constituting a reconstructed digital representation of the sampled image comprises the further step of

filtering the cyan, yellow and white values contained in the cyan, yellow and white planes to reduce the spatial bandwidth of the cyan, yellow and white planes prior to transformation into RGB planes.

24. The method of claim 22, wherein the step of generating a decompressed set of values constituting a reconstructed digital representation of the sampled image comprises the further step of

filtering, following transformation into RGB planes, the red, green and blue values contained in the red, green and blue planes to reduce the spatial bandwidth thereof.

25. The method of claim 24, wherein the step of generating a decompressed set of values constituting a reconstructed digital representation of the sampled image comprises the further step of

generating, from the sampled white values stored in the memory element, a half-resolution black and white image, by selecting and transmitting only the sampled white values stored in the memory element.

26. The method of claim 19, wherein the step of storing values in a memory element further comprises the step of

utilizing a non-volatile storage device to store the digital values sampled by the FPA.

27. The method of claim 19, further comprising

utilizing a telemetry link to transmit the digital values sampled by the FPA to the digital signal processing circuit.

28. A method of generating a reconstructed color image from a compressed digital dataset representative of the color image, the compressed digital dataset being generated by a focal plane array (FPA) having a subtractive color filter array thereon, said color filter array having subtractive color filter elements of cyan, yellow and white, arranged such that at each pixel of the FPA, the FPA samples a digital cyan value, a digital yellow value, or a digital white value, whereby the set of digital cyan, yellow and white values sampled by the full set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete cyan, yellow and white color planes representative of the sampled image, the method comprising the steps of:

generating, from digital values sampled by the FPA, a set of values constituting a reconstructed digital representation of the sampled image, the step of generating a set of values constituting a reconstructed digital representation of the sampled image including the step of

interpolating across the digital cyan, yellow and white values sampled by the FPA to generate respective cyan, yellow and white values constituting complete cyan, yellow and white color planes corresponding to the sampled image,

the step of generating a set of values constituting a reconstructed digital representation of the sampled image, and

generating, from the sampled white values stored in the memory element, a half-resolution black and white image by selecting and transmitting only the sampled white values stored in the memory element.

29. The method of claim 28, wherein the step of generating a set of values constituting a reconstructed digital representation of the sampled image further comprises the step of

converting the cyan, yellow and white values constituting complete cyan, yellow and white color planes corresponding to the sampled image into red, green and blue values constituting complete red, green and blue planes representative of the sampled image.

30. The method of claim 29, wherein the converting step comprises the further step of

addressing lookup tables with ones of the cyan, white and yellow values, to generate red, green and blue values constituting full red, green and blue planes representative of the sampled image.

31. The method of claim 30, wherein the step of generating a set of values constituting a reconstructed digital representation of the sampled image further comprises the steps of

multiplying each respective red, green and blue value in the fully-generated red, green and blue planes by the corresponding white plane value for each respective pixel, normalized by the sum of the red, green and blue values at that pixel position, and

adjusting the luminance at each pixel so that it is equal to the white plane value at that pixel position.

32. The method of claim 31, wherein the step of generating a set of values constituting a reconstructed digital representation of the sampled image comprises the further step of

reducing the spatial bandwidth of the cyan, yellow and white planes prior to transformation into RGB planes.

33. The method of claim 32, wherein the step of reducing the spatial bandwidth of the cyan, yellow and white planes includes the step of

filtering the values of the respective cyan, yellow and white planes with a two dimensional filtering window with selected weighting coefficients, to reduce the spatial bandwidth of the cyan, yellow and white planes.

34. A method of generating a reconstructed color image from a compressed digital dataset representative of the color image, the compressed digital dataset being generated by a focal plane array (FPA) having a subtractive color filter array thereon, said color filter array having subtractive color filter elements of cyan, yellow and white, arranged such that at each pixel of the FPA, the FPA samples a digital cyan value, a digital yellow value, or a digital white value, whereby the set of digital cyan, yellow and white values sampled by the full set of pixels of the FPA is a compressed digital dataset representative of the color image, said compressed digital dataset containing fewer values than necessary to constitute complete cyan, yellow and white color planes representative of the sampled image, the method comprising the steps of:

generating, from digital values sampled by the FPA, a set of values constituting a reconstructed digital representation of the sampled image, the step of generating a set of values constituting a reconstructed digital representation of the sampled image including the step of

interpolating across the digital cyan, yellow and white values sampled by the FPA to generate respective cyan, yellow and white values constituting complete cyan, yellow and white color planes corresponding to the sampled image,

the interpolating step includes the further step of

generating, from the sampled white values stored in the memory element, a full resolution black and white image, by interpolating only the sampled white values stored in the memory element.