Automatic illuminant estimation that incorporates apparatus setting and intrinsic color casting information

• US 9,013,596 B2
• Filed: 09/24/2008
• Issued: 04/21/2015
• Est. Priority Date: 09/24/2008
• Status: Active Grant

• Current Assignee: Nikon Corporation
• Original Assignee: Nikon Corporation
• Inventors: Hong, Li
• Primary Examiner(s): Haskins, Twyler
• Assistant Examiner(s): Chon, Peter
• Application Number:
  US13/119,574
• Publication Number:
  US 20110164150A1
• Time in Patent Office: 2,400 Days
• Field of Search: 348/222.1, 348/223.1, 348/224.1, 348/225.1, 348/241, 382/162, 382/163, 382/164, 382/165, 382/166, 382/167, 382/168, 382/169, 382/170, 382/171, 382/172, 382/254
• US Class Current: 348/223.1
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

An image apparatus (10) for providing an estimated final illuminant of a scene (12) and subsequently providing an adjusted image (218) of a scene (12) includes a capturing system (226) and a control system (232). The capturing system (226) captures information for an input image (416) of the scene (12) while at an apparatus setting. The input image (416) is defined by a plurality of input colors. The control system (226) can compare at least one of the input colors to an illuminant database (245) that includes an illuminant gamut of observable colors for a plurality of possible illuminants, and can evaluate the apparatus setting to select one of the possible illuminants to be the estimated final illuminant. Examples of possible apparatus settings include an aperture size and an exposure time. With this design, the present invention is better able to distinguish between some possible outdoor illuminants and some possible indoor illuminants with improved accuracy. Additionally, the control system (226) can evaluate a consistency matching error for the plurality of possible illuminants to determine if the scene (12) has intrinsic color casting.

Claims

1. An image apparatus for providing an estimated final illuminant of a scene, the image apparatus comprising:

a capturing system that captures an input image of the scene, the input image including a plurality of input colors; and

a control system that compares at least one of the input colors to an illuminant database that includes an illuminant gamut of observable colors for a plurality of possible illuminants, evaluates if the scene has intrinsic color casting, and, if the scene is determined to have intrinsic color casting, generates a color distribution of the input image to determine a color casting of the input image.

2. The image apparatus of claim 1 wherein the control system determines a separation distance between the at least one input color and a closest observable color for each of the possible illuminants in the illuminant database to obtain a consistency matching error for each of the plurality of possible illuminants, and the control system evaluates the consistency matching error for each of the possible illuminants to determine if the scene has intrinsic color casting.

3. The image apparatus of claim 1 wherein the control system selects the possible illuminant with the best consistency matching error as an estimated initial illuminant and wherein if the control system determines that the scene does not have intrinsic color casting, the control system selects the estimated initial illuminant as the estimated final illuminant.

4. The image apparatus of claim 1 wherein the control system selects the possible illuminant with the best consistency matching error as an estimated initial illuminant and wherein if the control system determines that the scene has intrinsic color casting, the control system selects one of the possible illuminants near the estimated initial illuminant as the estimated final illuminant.

5. The image apparatus of claim 1 wherein the control system evaluates an exposure time of the capturing system to select one of the possible illuminants to be the estimated final illuminant.

6. A method for providing an estimated final illuminant of a scene, the method comprising the steps of:

capturing an input image of the scene with a capturing system, the input image including a plurality of input colors;

comparing at least one of the input colors to an illuminant database that includes an illuminant gamut of observable colors for a plurality of possible illuminants with a control system, and

evaluating if the scene has intrinsic color casting with the control system and, if the scene is determined to have intrinsic color casting, generating a color distribution of the input image to determine a color casting of the input image with the control system.

7. The method of claim 6 wherein the step of comparing includes the step of determining a separation distance between the at least one input color and a closest observable color for each of the possible illuminants in the illuminant database to obtain a consistency matching error for each of the plurality of possible illuminants, and evaluating the consistency matching error for each of the possible illuminants to determine if the scene has intrinsic color casting.

8. The method of claim 7 wherein the step of comparing includes the step of selecting the possible illuminant with the best consistency matching error as an estimated initial illuminant and if the control system determines that the scene does not have intrinsic color casting, the control system selects the estimated initial illuminant as the estimated final illuminant.

9. The method of claim 7 wherein the step of comparing includes the step of selecting the possible illuminant with the best consistency matching error as an estimated initial illuminant and if the control system determines that the scene has intrinsic color casting, the control system selects one of the possible illuminants near the estimated initial illuminant as the estimated final illuminant.

10. An image apparatus for providing an estimated final illuminant of a scene, the image apparatus comprising:

a capturing system that captures an input image of the scene, the input image including a plurality of input colors including a first input color and a second input color that is different from the first input color; and

a control system including a processor that accesses an illuminant database that includes a first illuminant gamut for a first illuminant and a second illuminant gamut for a second illuminant, the first illuminant gamut being different from the second illuminant gamut, the first illuminant gamut including a plurality of different first observable colors in the visible spectrum for the first illuminant, and the second illuminant gamut including a plurality of different second observable colors in the visible spectrum for the second illuminant;

wherein the control system (i) compares the first input color to the plurality of different first observable colors to determine a first, closest first observable color;

(ii) determines a first separation distance between the first input color and the first, closest first observable color;

(iii) compares the second input color to the plurality of different first observable colors to determine a second, closest first observable color that is different from the first, closest first observable color;

(iv) determines a second separation distance between the second input color and the second, closest first observable color;

(v) compares the first input color to the plurality of different second observable colors to determine a first, closest second observable color;

(vi) determines a third separation distance between the first input color and the first, closest second observable color;

(vii) compares the second input color to the plurality of different second observable colors to determine a second, closest second observable color that is different from the first, closest second observable color; and

(viii) determines a fourth separation distance between the second input color and the second, closest second observable color to select one of the first illuminant or the second illuminant to be the estimated final illuminant.

11. The image apparatus of claim 10 wherein the control system (i) calculates a first consistency matching error for the first illuminant using the first separation distance and the second separation distance, and (ii) calculates a second consistency matching error for the second illuminant using the third separation distance and the fourth separation distance.

12. The image apparatus of claim 10 wherein the plurality of possible illuminants includes a plurality of possible indoor illuminants and a plurality of possible outdoor illuminants, and wherein the control system evaluates an apparatus setting of the capturing system when the input image is captured to determine if the apparatus setting indicates an outdoor illuminant or an indoor illuminant.

13. An image apparatus for providing an estimated final illuminant of a scene, the image apparatus comprising:

a capturing system that captures an input image of the scene, the input image including a plurality of input colors including a first input color and a second input color that is different from the first input color; and

a control system including a processor that accesses an illuminant database that includes a first illuminant gamut for a first illuminant and a second illuminant gamut for a second illuminant, the first illuminant gamut being different from the second illuminant gamut, the first illuminant gamut including a plurality of different first observable colors in the visible spectrum for the first illuminant, and the second illuminant gamut including a plurality of different second observable colors in the visible spectrum for the second illuminant;

wherein the control system (i) evaluates the plurality of input colors from the input image to create a list of non-redundant input colors, (ii) compares each of the input colors in the list of non-redundant input colors to each of the plurality of different first observable colors in the first illuminant gamut, and (iii) compares each of the input colors in the list of non-redundant input colors to each of the plurality of different second observable colors in the second illuminant gamut to select one of the first illuminant or the second illuminant to be the estimated final illuminant.

14. The image apparatus of claim 13 wherein the control system (i) identifies a separate, closest first observable color from the plurality of different first observable colors in the first illuminant gamut for at least two of the input colors in the list of non-redundant input colors, and (ii) identifies a separate closest second observable color from the plurality of different second observable colors in the second illuminant gamut for at least two of the input colors in the list of non-redundant input colors.

15. The image apparatus of claim 14 wherein the control system (i) determines a separate, first separation distance between each closest first observable color and its corresponding input color;

and (ii) determines a separate, second separation distance between each closest second observable color and its corresponding input color.

16. The image apparatus of claim 13 wherein the control system (i) compares the first input color to the plurality of different first observable colors to determine a first, closest first observable color;

(ii) compares the second input color to the plurality of different first observable colors to determine a second, closest first observable color that is different from the first, closest first observable color;

(iii) compares the first input color to the plurality of different second observable colors to determine a first, closest second observable color; and

(iv) compares the second input color to the plurality of different second observable colors to determine a second, closest second observable color that is different from the first, closest second observable color.

17. The image apparatus of claim 16 wherein the control system (i) determines a first separation distance between the first input color and the first, closest first observable color;

(ii) determines a second separation distance between the second input color and the second, closest first observable color;

(iii) determines a third separation distance between the first input color and the first, closest second observable color; and

(iv) determines a fourth separation distance between the second input color and the second, closest second observable color.

18. The image apparatus of claim 17 wherein the control system (i) calculates a first consistency matching error for the first illuminant using the first separation distance and the second separation distance, and (ii) calculates a second consistency matching error for the second illuminant using the third separation distance and the fourth separation distance.

19. The image apparatus of claim 18 wherein the control system evaluates at least one of the consistency matching errors to determine if the scene has intrinsic color casting.

20. A method for providing an estimated final illuminant of a scene, the method comprising the steps of:

capturing an input image of the scene with a capturing system, the entire input image including a plurality of input colors including a first input color and a second input color that is different from the first input color;

accessing an illuminant database that includes a first illuminant gamut for a first illuminant and a second illuminant gamut for a second illuminant using a control system having a processor, the first illuminant gamut being different from the second illuminant gamut, the first illuminant gamut including a plurality of different first observable colors in the visible spectrum for the first illuminant, and the second illuminant gamut including a plurality of different second observable colors in the visible spectrum for the second illuminant;

evaluating the plurality of input colors including the first and second input colors from the input image with the control system to create a list of non-redundant input colors; and

comparing each of the input colors including the first and second input colors in the list of non-redundant input colors to each of the plurality of different first observable colors in the first illuminant gamut with the control system, and comparing each of the input colors including the first and second input colors in the list of non-redundant input colors to each of the plurality of different second observable colors in the second illuminant gamut with the control system to select one of the first illuminant or the second illuminant to be the estimated final illuminant.

21. A method for providing an estimated final illuminant of a scene, the method comprising the steps of:

capturing an input image of the scene with a capturing system, the entire input image including a plurality of input colors including a first input color and a second input color that is different from the first input color;

accessing an illuminant database that includes a first illuminant gamut for a first illuminant and a second illuminant gamut for a second illuminant using a control system having a processor, the first illuminant gamut being different from the second illuminant gamut, the first illuminant gamut including a plurality of different first observable colors in the visible spectrum for the first illuminant, and the second illuminant gamut including a plurality of different second observable colors in the visible spectrum for the second illuminant; and

selecting one of the first illuminant or the second illuminant to be the estimated final illuminant by (i) comparing the first input color to the plurality of different first observable colors to determine a first, closest first observable color with the control system;

(ii) determining a first separation distance between the first input color and the first, closest first observable color with the control system;

(iii) comparing the second input color to the plurality of different first observable colors to determine a second, closest first observable color that is different from the first, closest first observable color with the control system;

(iv) determining a second separation distance between the second input color and the second, closest first observable color with the control system;

(v) comparing the first input color to the plurality of different second observable colors to determine a first, closest second observable color with the control system;

(vi) determining a third separation distance between the first input color and the first, closest second observable color with the control system;

(vii) comparing the second input color to the plurality of different second observable colors to determine a second, closest second observable color that is different from the first, closest second observable color with the control system; and

(viii) determining a fourth separation distance between the second input color and the second, closest second observable color with the control system.

22. The method of claim 21 further comprising the steps of (i) calculating a first consistency matching error for the first illuminant with the control system using the first separation distance and the second separation distance, and (ii) calculating a second consistency matching error for the second illuminant with the control system using the third separation distance and the fourth separation distance.