Using a minimal number of parameters for correcting the response of color image sensors

• US 6,430,313 B1
• Filed: 09/10/1998
• Issued: 08/06/2002
• Est. Priority Date: 09/10/1998
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Smith, Ronald D., Bawolek, Edward J.
• Primary Examiner(s): Tran, Phuoc
• Assistant Examiner(s): Alavi, Amir
• Application Number:
  US09/151,367
• Time in Patent Office: 1,426 Days
• Field of Search: 382/162, 382/163, 382/164, 382/165, 382/166, 382/167, 382/168, 382/169, 382/170, 382/171, 348/221, 348/222, 348/265, 348/266, 348/267, 348/268, 348/272, 348/273, 348/274, 348/275, 348/280, 348/281, 348/282, 348/223, 358/513, 358/514, 358/518, 358/523, 358/530
• US Class Current: 382/167
• CPC Class Codes: H04N 1/6033 using test pattern analysis H04N1/6055 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 9/67 for matrixing camera processing pipelines for matrixing of colour signals H04N23/85

Abstract

Methods for generating a set of parameters and an associated methodology that model a portion of the variation in a number of sensor-specific correction matrices. The parameters are fewer than the number of elements in one of the sensor-specific correction matrices yet adequately model variations in the IC manufacturing process used to fabricate color image sensors. Storage circuitry aboard an IC die containing the sensor can be configured to store the parameters.

Claims

1. A method comprising:

generating a first matrix to correct the response of a reference image sensor associated with an image sensor manufacturing process, based on measured responses to color stimuli of a plurality of image sensors built using the process;

generating a plurality of sensor-specific correction matrices that correct for deviations in the responses of the plurality of image sensors, respectively, from that of the reference image sensor, each sensor-specific correction matrix being a product of a separate second matrix and an inverse of the first matrix; and

generating a set of parameters that model variation among corresponding elements of the plurality of sensor-specific correction matrixes.

2. The method of claim 1 wherein the first matrix is a base color transformation matrix K_base, and wherein each separate second matrix is a color transformation matrix K_sensor that is specific to each of the plurality of image sensors.

3. The method of claim 2 further comprising determining the number of parameters in the set required to model a predefined portion of the variation as being fewer than the number of elements in a sensor-specific correction matrix.

4. The method of claim 2 wherein the step of generating the plurality of sensor-specific correction matrices comprises

computing each of the sensor-specific correction matrices using {tilde over (T)}={tilde over (K)}_sensor{tilde over (K)}_base^{− 1}.

5. An imaging device comprising:

storage circuitry for storing a plurality of parameters for modeling the elements of a correction matrix {tilde over (T)}, {tilde over (T)} and a base matrix {tilde over (K)}_base being factors of a color transformation matrix {tilde over (K)}_sensor used to adjust pixel data derived from an image sensor built using an IC manufacturing process, wherein {tilde over (T)} is proportional to a product of {tilde over (K)}_sensor and {tilde over (K)}_base<sup>−

1</sup>, and the parameters are fewer than the number of matrix elements in {tilde over (K)}_sensor.

6. The imaging device of claim 5 further comprising

a plurality of sensor cells being electrically responsive to light to provide signals from which the pixel data are derived, the sensor cells and the storage circuitry being disposed in the same integrated circuit (IC) die.

7. The imaging device of claim 5 wherein the parameters are obtained by performing principal components analysis (PCA) on statistical sensor response data obtained from the image sensor and a plurality of other sensors built using the IC manufacturing process.

8. The imaging device of claim 5 wherein the storage circuitry provides for non-volatile storage of the parameters.

9. The imaging device of claim 5 wherein the parameters comprise gain and tilt values for the image sensor.

10. The imaging device of claim 9 wherein the pixel data are derived from the image sensor having an R, G, and B color filter array, and wherein the color transformation matrix {tilde over (K)}_sensor is further configured to translate the pixel data into XYZ pixel data.

11. A method of encoding correction information for a color image sensor, comprising:

a) illuminating a first image sensor with known color stimuli to obtain in response thereto a first plurality of measured pixel values, the first sensor being manufactured as part of an integrated circuit (IC) die using a semiconductor fabrication process;

b) computing a color transformation matrix {tilde over (K)}_sensor associated with the first sensor;

c) computing a base matrix {tilde over (K)}_base based on responses to color stimuli of a plurality of image sensors built using the process;

d) computing a correction matrix {tilde over (T)} based on {tilde over (K)}_base and {tilde over (K)}_sensor, where {tilde over (T)}={tilde over (K)}_sensor*{tilde over (K)}_base<sup>−

1</sup>; and

e) computing a plurality of coefficients k_n which minimize the error in an estimate {circumflex over (T)}_ij for an element T_ij of the matrix {tilde over (T)}, the estimate {circumflex over (T)}_ij being obtained by a linear combination of k_n and P_n, where P_n are related to the first measured pixel values.

12. The method of claim 11 further comprising:

storing information corresponding to the plurality of P_n in the IC die.

13. The method of claim 12 wherein the step of storing comprises storing a plurality of pointers in the IC die to the plurality of P_n.

14. A method as in claim 12 wherein the step of storing comprises storing the information in a non-volatile storage in the IC die.

15. A method of recovering correction information for a color image sensor, comprising:

computing a first matrix based on a plurality of parameters and based on information that characterizes the IC manufacturing process used for fabricating the color image sensor, the parameters modeling a portion of the variation in a plurality of sensor-specific correction matrices for a population of color image sensors built using the manufacturing process, the parameters being fewer than the number of elements in one of the sensor-specific correction matrices;

computing a second matrix based on the first matrix and based on said information that characterizes the IC manufacturing process, wherein the second matrix is proportional to a product of the first matrix and a third matrix that incorporates some of said information; and

applying the second matrix for correcting image data captured by the image sensor.

16. The method of claim 15 wherein the second matrix is equal to a product of the first and third matrices.

17. The method of claim 15 wherein the parameters are obtained by performing Principal Components Analysis (PCA) upon the plurality of sensor-specific correction matrices, the information that characterizes the manufacturing process comprising a methodology and set of eigenvectors associated for computing the first matrix.

18. The method of claim 15 wherein the parameters include the gain and tilt of the color image sensor, and the information that characterizes the manufacturing process comprises a plurality of correlation coefficients for computing the first matrix.

19. An article comprising:

a machine-readable medium having instructions that when executed by a processor cause

computing a first matrix based on a plurality of parameters and based on information that characterizes the IC manufacturing process used for fabricating the color image sensor, the parameters modeling a portion of the variation in a plurality of sensor-specific color correction matrices, the parameters being fewer than the number of elements in one of the sensor-specific matrices;

computing a second matrix based on the first matrix and based on said information that characterizes the IC manufacturing process, wherein the second matrix is proportional to a product of the first matrix and a third matrix that incorporates some of said information; and

applying {tilde over (K)}_est for correcting image data captured by the image sensor.

20. The article of claim 19 wherein the machine-readable medium comprises instructions that when executed by the processor cause computing the second matrix as equal to a product of the first and third matrices.

21. The article of claim 19 wherein the machine-readable medium comprises instructions that when executed cause the parameters to be obtained by performing Principal Components Analysis (PCA) upon the plurality of sensor-specific correction matrices, the information that characterizes the manufacturing process comprising a methodology and set of eigenvectors associated for computing the first matrix.

22. The article of claim 19 wherein the parameters include the gain and tilt of the color image sensor, and the information that characterizes the manufacturing process comprises a plurality of correlation coefficients for computing the first matrix.