Efficient Determination of an Illuminant of a Scene

• US 20080278601A1
• Filed: 05/07/2007
• Published: 11/13/2008
• Est. Priority Date: 05/07/2007
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: Goel, Anurag, Muninder, Veldandi
• Granted Patent: US 8,564,687 B2
• Time in Patent Office: Days
• US Class Current: 348/223.1
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

An aspect of the present invention reduces computational complexity in determining a illuminant of a scene of interest by selecting only a subset of illuminants from several more potential illuminants, and searching for a current illuminant for a present image frame in only the subset of illuminants. Computational complexity may be reduced due to the searching in fewer illuminants. The subset of illuminants are selected according to various aspects to enhance the probability that the closest matching potential illuminant is accurately determined. The features can be used in image capture devices (ICDs) such as video cameras.

Claims

1. A method of determining a scene illuminant illuminating a scene of interest, said method comprising:

selecting a subset of illuminants from a plurality of potential illuminants;

receiving a present image frame representing said scene of interest and information representing how each of a set of colors would be manifested under each of said plurality of potential illuminants;

determining a current illuminant from said subset of illuminants by comparing at least a portion of said present image frame with said information corresponding to said subset of illuminants for a match; and

providing said current illuminant as said scene illuminant.

2. The method of claim 1, wherein said selecting selects either an odd group or an even group as said subset of illuminants, wherein said odd group represents those of said plurality of potential illuminants in odd positions when ordered by correlated color temperature and said even group represents those of said plurality of potential illuminants in even positions when ordered by correlated color temperature.

3. The method of claim 2, further comprising:

receiving a next image frame following said present image frame; and

identifying a scene illuminant for said next image frame by comparing for match at least a portion of said next image frame with said information corresponding to the potential illuminants in odd group or even group not used in relation to said present image frame.

4. The method of claim 3, wherein said identifying includes said scene illuminant for said present image frame also in the potential illuminants for match for said next image frame.

5. The method of claim 4, wherein said determining comprises:

setting a present group to either said odd group or said even group;

comparing said information corresponding to each potential illuminant in said present group with at least a portion of said present image frame to ascertain a closest matching illuminant as an intermediate illuminant,

further comparing said information corresponding to said intermediate illuminant and a set of adjacent illuminants to said intermediate illuminants with at least a portion of said present image frame to ascertain a closest matching illuminant as said scene illuminant for said present image frame.

6. The method of claim 5, wherein said identifying does not consider said set of adjacent illuminants in identifying said scene illuminant for said next image frame.

7. The method of claim 6, further comprising:

checking a status of a history flag;

wherein said identifying is performed only if said history flag is set to one logical value; and

wherein said determining is performed treating said next image frame as said present image frame if said history flag is set to another logical value.

8. The method of claim 6, further comprising:

receiving a sequence of frames following said next image frame, wherein said sequence of frames contain a fourth frame and a fifth frame received consecutively in said sequence;

setting a present group for said fourth frame to one of said odd group or said even group not selected for an immediately preceding frame to said fourth frame, adding the closest matching illuminant identified for the immediate preceding frame and removing one or more adjacent illuminants to the intermediate illuminant of the immediate preceding frame from the present group of the fourth frame;

examining said present group to select an intermediate illuminant for said fourth frame;

setting said intermediate illuminant as the closest matching illuminant for said fourth frame if a scene or actual illuminant of the scene is not deemed to have changed;

searching said intermediate illuminant and one or more adjacent illuminants to said intermediate illuminant to select the closest matching illuminant for said fourth frame if said scene or said actual illuminant is deemed to have changed.

9. The method of claim 8, further comprising treating said fifth frame as said next image frame and processing said fifth frame to determine the closest matching illuminant for said fifth frame.

10. The method of claim 1, further comprising:

receiving a second image frame;

identifying an current illuminant by comparing said information corresponding to said scene illuminant of said present image frame and a set of adjacent illuminants to said scene illuminant of said present image frame with at least a portion of said second image frame to ascertain a closest matching illuminant for said second image frame;

checking whether said scene or actual illuminant illuminating said scene has changed at time instances at which said present image frame and said second image frame were generated; and

providing said current illuminant as said scene illuminant for said second image frame if neither of said scene nor said actual illuminant has changed.

11. The method of claim 10, further comprising treating said second image frame as said present image frame and performing said determining to determine said scene illuminant for said second image frame if said checking indicates that said scene has changed.

12. The method of claim 10, wherein said checking checks whether a difference of a first error metric and a second error metric is within a threshold, wherein said first error metric represents a deviation of content of at least a portion of said present image frame from information corresponding to the scene illuminant determined for said present image frame, wherein said second error metric represents a deviation of content of at least a portion of said second image frame from information corresponding to said the scene illuminant for said present image frame.

13. The method of claim 1, said method further comprising:

receiving a fourth image frame following said present image frame;

checking whether said scene or actual illuminant illuminating said scene has changed at time instances at which said present image frame and said fourth image frame were generated; and

providing said scene illuminant for said present image frame as a scene illuminant for said fourth image frame also if neither said scene nor said actual illuminant has changed.

14. The method of claim 1, further comprising:

partitioning said plurality of potential illuminants into a sequence of subsets;

receiving a sequence of image frames including said present image frame;

for each image frame in said sequence of image frames, examining a corresponding one of said sequence of subsets and a closest matching illuminant of a previous image frame until all of said sequence of subsets are examined, wherein a final illuminant equals the closest matching illuminant after all of said sequence of subsets are examined; and

providing said final illuminant as the closest matching illuminant for some image frames following said sequence of frames.

15. A machine readable medium carrying one or more sequences of instructions for causing a system to determine a scene illuminant illuminating a scene of interest, wherein execution of said one or more sequence of instructions by one or more processors contained in said system causes said system to perform the actions of:

selecting a subset of illuminants from a plurality of potential illuminants;

receiving a present image frame representing said scene of interest and information representing how each of a set of colors would be manifested under each of said plurality of potential illuminants;

determining a current illuminant from said subset of illuminants by comparing at least a portion of said present image frame with said information corresponding to said subset of illuminants for a match; and

providing said current illuminant as said scene illuminant.

16. The machine readable medium of claim 15, wherein said selecting selects either an odd group or an even group as said subset of illuminants, wherein said odd group represents those of said plurality of potential illuminants in odd positions when ordered by correlated color temperature and said even group represents those of said plurality of potential illuminants in even positions when ordered by correlated color temperature.

17. The machine readable medium of claim 15, wherein execution of said one or more sequence of instructions by one or more processors contained in said system causes said system to perform further actions of:

receiving a second image frame;

identifying an current illuminant by comparing said information corresponding to said scene illuminant of said present image frame and a set of adjacent illuminants to said scene illuminant of said present image frame with at least a portion of said second image frame to ascertain a closest matching illuminant for said second image frame;

checking whether said scene or actual illuminant illuminating said scene has changed at time instances at which said present image frame and said second image frame were generated; and

providing said current illuminant as said scene illuminant for said second image frame if neither of said scene nor said actual illuminant has changed.

18. The machine readable medium of claim 15, wherein execution of said one or more sequence of instructions by one or more processors contained in said system causes said system to perform further actions of:

partitioning said plurality of potential illuminants into a sequence of subsets;

receiving a sequence of image frames including said present image frame;

for each image frame in said sequence of image frames, examining a corresponding one of said sequence of subsets and a closest matching illuminant of a previous image frame until all of said sequence of subsets are examined, wherein a final illuminant equals the closest matching illuminant after all of said sequence of subsets are examined; and

providing said final illuminant as the closest matching illuminant for some image frames following said sequence of frames.

19. A device comprising:

a memory unit designed to store information representing how each of a set of colors would be manifested under each of a plurality of potential illuminants;

a processor for determining a scene illuminant illuminating a scene of interest, said processor being operable to;

select a subset of illuminants from a plurality of potential illuminants;

receive a present image frame representing said scene of interest;

determine a current illuminant from said subset of illuminants by comparing at least a portion of said present image frame with said information corresponding to said subset of illuminants for a match; and

provide said current illuminant as said scene illuminant.

20. The invention of claim 19, wherein said device comprises an image capture device, said device further comprises:

a lens assembly and a image sensor array, wherein said lens assembly focuses light from said scene of interest on said image sensor array, wherein said image sensor array generates a plurality of pixel values representing said scene of interest, wherein said plurality of pixel values form said present image frame.