Method of estimating chromaticity of illumination using neural networks

• US 5,907,629 A
• Filed: 11/15/1996
• Issued: 05/25/1999
• Est. Priority Date: 11/15/1996
• Status: Expired due to Fees

• Current Assignee: Brian Vicent Funt, Jacobus Joubert Barnard, Vlad Constantin Cardei
• Original Assignee: Brian Vicent Funt, Jacobus Joubert Barnard, Vlad Constantin Cardei
• Inventors: Cardei, Vlad Constantin, Funt, Brian Vicent, Barnard, Jacobus Joubert
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): NOVIK, DMITRY A
• Application Number:
  US08/746,725
• Time in Patent Office: 921 Days
• Field of Search: 382/162, 382/167, 382/156, 382/158, 382/157, 358/516, 358/518, 358/520, 358/521, 358/523, 358/515, 348/223, 348/225, 348/228
• US Class Current: 382/162
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: H04N 1/60 Colour correction or control H04N1/54 takes precedence
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature

Abstract

A method of estimating the chromaticity of illumination of a colored image consisting of a plurality of color-encoded pixels. The image colors are first mapped into an intensity-independent chromaticity space which is then divided into a plurality of separate regions. For each region, a first binary value is assigned to the region if the region contains no chromaticity value; or, a second binary value is assigned to the region if it does contain a chromaticity value. The assigned values are then applied as inputs to a pre-trained neural network having two output ports and at least one intermediate layer containing a plurality rality of ports connectible between selected input ports and the output ports. The chromaticity space values which characterize the input image'"'"'s chromaticity of illumination are then derived at the output ports. The network is pretrained trained by initially connecting an arbitrary number of the intermediate layer ports to selected input layer ports. A weight value is associated with each connection. The weight values, which have the effect of altering signals transmitted along each connection by a selected amount, are initialized with random values. Each one of a plurality of pre-stored data sets, each containing values characterizing presence or absence of color in selected regions of one of a corresponding plurality of known colored images, are sequentially presented as inputs to the network and the chromaticity space values derived at the output ports are compared with known chromaticity space values characterizing illumination of the known colored image to derive an error value representative of difference therebetween. The weight values are adjusted in response to the inputs in accordance with the well known back propagation algorithm. After the weights are adjusted the intermediate layer ports are adaptively reconnected to the input layer ports to eliminate connections to input layer ports which repeatedly receive zero value inputs. The training process continues until the error value is less than a selected threshold.

Claims

1. A method of estimating the chromaticity of illumination of a colored image consisting of a plurality of color-encoded pixels which define a corresponding plurality of image colors, said method comprising the steps of:

a. mapping said image colors into an intensity-independent chromaticity space;

b. dividing said chromaticity space into a plurality of separate regions;

c. for each one of said regions;

i. assigning a first value to said region if said region contains no chromaticity value corresponding to any of said image colors;

ii. assigning a second value to said region if said region contains a chromaticity value corresponding to any of said image colors;

d. applying each one of said assigned values to a different one of a plurality of input ports in an input layer of a pre-trained neural network, said neural network having;

i. an output layer containing two output ports;

ii. at least one intermediate layer containing a plurality of ports connectible between selected ports in layers adjacent to said intermediate layer; and

,

e. reading, at said respective output ports, chromaticity space values which characterize said chromaticity of illumination.

2. A method as defined in claim 1, wherein said step of mapping said image into an intensity-independent chromaticity space further comprises mapping said image into a two dimensional chromaticity space.

3. A method as defined in claim 2, wherein:

a. said two dimensional chromaticity space is one of;

i. an rg chromaticity space;

ii. an rb chromaticity space;

iii. a bg chromaticity space;

iv. a perspective chromaticity space;

V. a vector-norm chromaticity space;

vi. a u^* v^* chromaticity space;

vii. an a^* b^* chromaticity space;

b. said color-encoded pixels are RGB-encoded pixels, where R, G and B are first, second and third color channels respectively, and said step of mapping said image into said two dimensional chromaticity space further comprises;

i. if said two dimensional chromaticity space is said rg chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where p is defined to be R/(R+G+B) and q is defined to be G/(R+G+B);

ii. if said two dimensional chromaticity space is said rb chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where p is defined to be R/(R+G+B) and q is defined to be B/(R+G+B);

iii. if said two dimensional chromaticity space is said bg chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where p is defined to be B/(R+G+B) and q is defined to be G/(R+G+B);

iv. if said two dimensional chromaticity space is said perspective chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where;

(1) p is defined to be R/B and q is defined to be G/B;

or,

(2) p is defined to be R/G and q is defined to be B/G;

or,

(3) p is defined to be B/R and q is defined to be G/R;

v. if said two dimensional chromaticity space is said vector-norm chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where;

(1) p is defined to be R/(R² +G² +B²)^1/2 and q is defined to be B/(R² +G² +B²)^1/2 ;

or,

(2) p is defined to be R/(R² +G² +B²)^1/2 and q is defined to be G/(R² +G² +B²)^1/2 ;

or,

(3) p is defined to be B/(R² +G² +B²)^1/2 and q is defined to be G/(R² +G² +B²)^1/2 ;

or,

vi. if said two dimensional chromaticity space is said u^* v^* chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where p is defined to be the u^* component of a CIE 1976 (L^* u^* v^*) chromaticity space and q is defined to be the v^* component of said CIE 1976 (L^* u^* v^*) chromaticity space; and

,

vii. if said two dimensional chromaticity space is said a^* b^* chromaticity space, transforming each one of said RGB pixels to yield (p,q) components, where p is defined to be the a^* component of a CIE 1976 (L^* a^* b^*) chromaticity space and q is defined to be the b^* component of said CIE 1976 (L^* a^* b^*) chromaticity space.

4. A method as defined in claim 1, further comprising, before said step of applying said assigned values to said neural network input layer input ports, mapping said assigned values into a one-dimensional input space containing a plurality of input values corresponding to said plurality of input ports.

5. A method as defined in claim 1, further comprising pre-training said neural network by:

a. connecting an arbitrary number of said input layer ports to selected ports in one of said intermediate layers adjacent to said input layer;

b. connecting said one intermediate layer'"'"'s ports to ports in another of said layers adjacent to said one intermediate layer, and further connecting any remaining ones of said intermediate layers to a next adjacent one of said layers until all of said adjacent layers are connected;

c. assigning, to each one of said connections, a selected weight value for altering signals transmitted by each of said connections by a selected amount;

d. for each one of a plurality of pre-stored data sets, each of said data sets containing values characterizing presence or absence of color in selected regions of one of a corresponding plurality of known colored images;

i. selecting a different one of said pre-stored data sets;

ii. applying to said input layer said color presence/absence values contained in said data set;

iii. comparing said two chromaticity space values derived at said respective output ports with known chromaticity space values characterizing illumination of said known colored image to derive an error value representative of difference therebetween;

iv. selectively reassigning said weight values to reduce said error value while minimizing changes to minimized error values previously obtained in performing this step (d) in respect of data sets other than said one data set; and

,

e. repeating step (d) until said error value is minimized to a value less than a selected threshold.

6. A method as defined in claim 1, wherein:

a. said input layer has about 900 to 2,500 input ports; and

,

b. said intermediate layer has about 100 to 150 input ports.

7. A method as defined in claim 1, wherein:

a. said input layer has about 1,000 input ports;

b. said intermediate layer has about 100 to 150 input ports; and

,

c. said neural network further comprises a second intermediate layer having about 20-50 ports, said second intermediate layer connectible between selected ports in said at least one intermediate layer and in said output layer.

8. A method as defined in claim 1, wherein said color-encoded pixels are RGB-encoded pixels, where R, G and B are first, second and third color channel values respectively, said method further comprising, before said claim 1(a) step:

a. dividing said image into a plurality of separate portions;

b. for each one of said portions;

i. disregarding all pixels in said portion having R, B or G values less than a selected minimum threshold;

ii. disregarding all pixels in said portion having R, B or G values greater than a selected maximum threshold;

iii. assigning, as said portion'"'"'s R value, the average R value of all pixels in said portion which have not been disregarded;

iv. assigning, as said portion'"'"'s G value, the average G value of all pixels in said portion which have not been disregarded; and

,

v. assigning, as said portion'"'"'s B value, the average B value of all pixels in said portion which have not been disregarded.

9. A method as defined in claim 1, wherein said color-encoded pixels are RGB-encoded pixels, where R, G and B are first, second and third color channels respectively, said method further comprising, before said claim 1(a) step, mapping said color-encoded pixels into a color-encoding spectrally sharpened color space by combining said channels.

10. A method as defined in claim 1, wherein said color-encoded pixels are RGB-encoded pixels, where R, G and B are first, second and third color channels respectively, said method further comprising, before said claim 1(a) step:

a. mapping said color-encoded pixels into a color-encoding spectrally sharpened color space by combining said channels;

b. dividing said image into a plurality of separate portions;

c. for each one of said portions;

i. disregarding all pixels in said portion having R, B or G values less than a selected minimum threshold;

ii. disregarding all pixels in said portion having R, B or G values greater than a selected maximum threshold;

iii. assigning, as said portion'"'"'s R value, the average R value of all pixels in said portion which have not been disregarded;

iv. assigning, as said portion'"'"'s G value, the average G value of all pixels in said portion which have not been disregarded; and

,

v. assigning, as said portion'"'"'s B value, the average B value of all pixels in said portion which have not been disregarded.

11. A method as defined in claim 5, wherein said selectively reassignment of said weight values further comprises calculation of said weight values by backpropagation.

12. A method as defined in claim 5, further comprising, after said step 5(d) and before said step 5(e), adaptively reconnecting said intermediate layer ports to said input layer ports to selectively eliminate connections to input layer ports which repeatedly receive zero value inputs.

13. A method as defined in claim 5, wherein at least some of said known colored images include surfaces having a significant specular reflectance component.