Color signal processing

• US 7,227,586 B2
• Filed: 01/12/2001
• Issued: 06/05/2007
• Est. Priority Date: 01/12/2000
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: University of East Anglia
• Inventors: Finlayson, Graham, Hordley, Stephen
• Primary Examiner(s): Ometz; David
• Assistant Examiner(s): DESIR, JEAN WICEL
• Application Number:
  US10/181,089
• Publication Number:
  US 20030142222A1
• Time in Patent Office: 2,335 Days
• Field of Search: 348/655, 348/630, 348/650, 348/652, 348/675, 348/679, 348/223.1, 348/225.1, 348/227.1, 348/603, 348/621, 348/627, 358/518, 358/516, 358/447, 358/448, 358/3.27, 382/254, 382/167
• US Class Current: 348/655
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

A method of adjusting the N components of an N component color signal (such as the 3 components of an RGB color signal) with respect to color temperature of illuminating light when imaging a colored subject, comprising the step of computing the difference between each of the components of the color signal for different temperatures of illuminating light, using an algorithm based on the spectral sensitivity of the imaging means and Planck'"'"'s equation defining black body illumination to obtain N components whose values are independent of the color temperature of the illuminating light. Normalisation of the adjusted values renders the values independent of intensity of illumination and can also reduce the processing time for further processing of the signals. A technique for compensating for gamma correction which is commonly built in cameras, is disclosed.

Claims

1. A method of processing an N-component color signal from a color signal source so as to eliminate dependency on color temperature and/or intensity variation by using a mathematical algorithm derived from the source sensitivities to different wavelengths of light in a signal processing computer supplied with the color signal, to provide new values for the components making-up the color signal which are independent of the color temperature of the light illuminating, or deemed to illuminate, the subject described by the color signal, wherein the N components, such as RGB signals of a 3 component color signal, define an image of an illuminated colored subject, the processing serving to produce modified signal components defining an image of the same subject as if illuminated under light of different color temperature, comprising the steps of:

(1) generating the N-(RGB etc.) components of a color signal corresponding to an illumination of the subject using illumination of color temperature T₁;

(2) forming the N log values (R′

G′

B′

etc.) of the N-RGB etc. components of the color signal so obtained; and

(3) computing the N log values (R*, G*, B* etc.) of the N-RGB etc. components of the color signal for a different illumination color temperature T₂ using the equation;

(R*G*B* . . . )=(R′

G′

B′

. . . )+X(u,v,w, . . . )

where X is proportional to (T1–

T2) and u, v, w, etc. are constants that depend only on the spectral sensitivities of the signal source and Planck'"'"'s equation defining black body illumination.

2. A method as claimed in claim 1 further comprising the step of converting the log values (R*, G*, B*, . . . ) to N different signals, typically appropriate RGB etc. values, for processing and/or display and/or storing.

3. A method as claimed in claim 1 wherein the color signal is derived from a camera, multispectral imaging device, or a scanner, or is a computer generated signal.

4. A method as claimed in claim 1 in which the method is applied to the N-RGB etc. values of each pixel making up a multi-pixel image.

5. A corrected RGB signal when obtained by the method of claim 1, comprising three parts namely in which the two non-zero log values are independent of the illumination color temperature.

6. A corrected RGB signal when obtained by the method of claim 1, in the form of a vector (R*, G*, B*)−

G*/v(u, v, w) which can be written in the form (χ

, , γ

) in which the non-zero components χ and

γ

are two values that are independent of the color temperature of the illuminant.

7. A method as claimed in claim 1, wherein the N values are normalized with respect to one of the N values to remove intensity dependence and reduce the number of values to (N−

1).

8. A method as claimed in claim 7 wherein there are three components R G and B in the original signal and as a result of normalisation these are reduced in number to R/G and B/G, and color intensity independence is obtained by converting these two components, one to zero and the other to (B′

−

R′

b/a) where B′

is log B/G and R′

is log R/G, and the values of a and b are derived from the values u, v, w for the signal source.

9. A method as claimed in claim 1, comprising the step of calibrating the source in advance to determine the (u, v, w etc. (or a, b, c etc.)) values for the source.

10. A computer when programmed so as to perform the method of claim 1.

11. A method as claimed in claim 1, further comprising the step of converting corrected component signal values as aforesaid into a grey scale or pseudo color signal for processing and/or comparison and/or storage and/or for forming a grey scale or pseudo color image in place of a true-color image in a display.

12. A method as claimed in claim 11 wherein a 1-dimensional color co-ordinate is expressed as a function of chromaticity, so as to remove dependence on the illumination color temperature by converting an 2 component color signal to a single component grey-scale signal or pseudo color signal, and a comparison is performed by the step of mapping the grey-scale values (or pseudo color values) to grey-scale values (or pseudo color values of a similar scene observed under reference lighting conditions.

13. A method of color-based object recognition employing color signals as processed and/or normalised as claimed in claim 11 wherein objects are represented by the distributions of invariant co-ordinates calculated from color images of the objects obtained by the processing to produce grey-scale histograms (since the invariant co-ordinates are coded as grey-scale), and recognition is performed by distribution comparison whereby the distribution of an unknown object is compared with known object distributions stored in a database, and the closest match identifies the unknown object.

14. A method of image analysis comprising the steps of comparing illumination color temperature and/or intensity corrected signals obtained by the method of claim 1 or summations or other computational combinations of some or all of the intensity corrected signals, with stored values of such signals and/or similar summations or other computational combinations thereof, to permit object recognition.

15. A computer programmed to perform the comparison set forth in claim 14.

16. A method as claimed in claim 1 in which the source includes RGB etc. correction in each of the N channels for the N components to compensate for the expected non-linearity of a display device such as CRT based display or monitor to which the signal could be supplied, and a linear relationship between the N-component color signal and the resulting image is obtained by dividing the log camera responses by the average of the log color responses in a local neighbourhood of the pixel of interest in the image prior to calculating the invariant co-ordinates i.e. corrected component values.

17. A method as claimed in claim 1 in which after the invariant co-ordinate(s) is/are calculated, the value(s) is/are normalised by the average invariant co-ordinate in a local neighbourhood of a pixel of interest in the image.

18. A method of processing an N-component color signal (N>

1) from a color signal source so as to eliminate dependency on color temperature and/or intensity variation by using a mathematical algorithm derived from the source sensitivities to different wavelengths of light in a signal processing computer supplied with the color signal, to provide new values for the components making up the color signal which are independent of the color temperature of the light illuminating, or deemed to illuminate, the subject described by the color signal, the method comprising the step of adjusting the N components of the signal so as to produce component values which when using an imaging means to image a colored subject are independent of the color temperature of the light illuminating the subject wherein the adjustment is effected by the step of computing for each of (N−

1) components a new value using the original component value, the Nth component value, and an algorithm incorporating a mathematical operator whose value is related to the spectral sensitivity of the imaging means to different color temperature illuminants.

19. A method of processing an N-component color signal (N>

1) as claimed in claim 18, wherein the N values are normalized with respect to one of the N values to remove intensity dependence and reduce the number of values to (N−

1).

20. A method as claimed in claim 19, wherein there are three components R G and B in the original signal and as a result of normalization these are reduced in number to R/G and B/G, and color intensity independence is obtained by converting these two components, one to zero and the other to (B′

−

R′

.b/a) where B′

is log B/G and R′

is log R/G, and the values of a and b are derived from the values u, v, w for the signal source.

21. A method of image enhancement when processing a color signal in a system in which the log of the color response is composed of three parts, a vector due to the intensity of the illumination, a vector due to the illumination color temperature, and a vector due to reflective properties of the surface illuminated for the purpose of producing the image, comprising the step of adjusting the magnitude of the intensity vector, or the magnitude of the illumination color temperature vector for each pixel so as to change the effective illumination for each pixel in a display of the image using the processed signal.

22. A method of enhancement as claimed in claim 21 further comprising the step of increasing or decreasing the color warmness or color depth by altering the magnitude of the illumination color temperature vector of the signal.

23. A method as claimed in claim 22 further comprising the step of altering the magnitude by scaling all illumination color temperature vectors for all pixels in the image.

24. A method as claimed in claim 22 further comprising the step of increasing or decreasing the magnitude of the illumination intensity vector for all pixels in an image so as to brighten or darken the overall picture, to allow enhancement of a final image without altering the original illumination.

25. A method as claimed in claim 22 wherein said step of adjusting the magnitude of the intensity vector is performed on a frame basis, or on selected regions of a frame made up of many pixels, or on a pixel by pixel basis.

26. A method of image enhancement applied to N-component color signal, such as the RGB signals of a 3 component color signal, defining an image of an illuminated colored subject, wherein the components are processed to produce modified signal components defining an image of the same subject as if illuminated by a different color temperature (T), comprising the steps of:

(1) generating the N-(RGB etc.) components of a color signal corresponding to an illumination of the subject using illumination of color temperature T;

(2) normalising the N components of the color signal with respect to one of the components (e.g. G) to provide (N−

1) co-ordinates, such as R/G, B/G in a 3 component RGB signal, so that each of the resulting components is independent of illumination intensity,

(3) computing the log values of these (N−

1) co-ordinates, such that log(R/G)=R′ and

log (B/G)=B′

, in the case of an RGB signal, and

(4) adjusting the values of R′

, B′

, etc., to produce new values R*, B*, etc., corresponding to the values which would apply if the color temperature of the illumination is T₂ using the equation (R*, B*, etc.)=(R′

, B′

, etc.)+X(a, b, etc.) where X is proportional to (T₁–

T₂) and a and b etc. are constants that depend only on the spectral sensitivities of the source and Planack'"'"'s equation defining black-body illumination.

27. A method as claimed in claim 26 further comprising the step of converting the values R*, B*, etc., to new RGB etc. values, and using the new values for processing and/or comparison and/or storage and/or display.

28. A method as claimed in claim 26, wherein said signal has a source and wherein said source is a camera, a multispectral imaging device, a scanner or a computer programmed to produce a color signal.