Illuminant estimation

• US 8,385,637 B2
• Filed: 11/08/2007
• Issued: 02/26/2013
• Est. Priority Date: 11/08/2006
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Finlayson, Graham D., Fredembach, Clement
• Primary Examiner(s): Liu, Li
• Application Number:
  US12/514,093
• Publication Number:
  US 20100172574A1
• Time in Patent Office: 1,937 Days
• Field of Search: None
• US Class Current: 382/162
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

In a method of chromagenic illuminant estimation pixels from mutually-corresponding images with different filtering (e.g. a filtered image and an unfiltered image) are compared, a fraction of the brightest pixels being selected for a subsequent chromagenic estimation. The pixels may be at corresponding locations or they may correspond in that their mean brightness is in the same rank order. In one method, in which, in a first preprocessing stage, for a database of m lights E_i (λ) and n surfaces S_j (λ) there is calculated T_i˜Q^F Q⁺ where Q₁^F and Q_F represent the matrices of unfiltered and filtered sensor responses to the n surfaces under the i th light and + denotes an inverse, and in a second operation stage, given P surfaces in an image and 3×P matrices Q and Q^F, from these matrices there are chosen the r % brightest pixels giving the matrices Q′ and Q^′F, and the scene illuminant P_est is estimated where formula (I) and (II).

Claims

1. A method of chromagenic illuminant estimation in which, from mutually corresponding images with different sets of spectral components, a fraction of the brightest pixels are selected for subsequent chromagenic estimation, wherein the act of selecting the fraction of the brightest pixels comprises selecting a particular fraction of pixels from each of the corresponding images in descending order of brightness from the brightest pixel in each of the corresponding images.

2. A method according to claim 1 wherein the images have a different filtering.

3. A method according to claim 2, wherein the images comprise a filtered image and an unfiltered image.

4. A method according to claim 1, wherein brightness values of pixels in a first one of the mutually corresponding images are compared with brightness values of pixels in a second one of the mutually corresponding images in rank order.

5. A method according to claim 1, wherein there are compared pixels in the images which are in the same pixel location.

6. A method according to claim 1, wherein 0.5 to 20% of the brightest pixels are selected.

7. A method according to claim 6, wherein 1 to 3% of the brightest pixels are selected.

8. A method according to claim 1 employing a chromagenic algorithm which works by comparing m responses in one image to a corresponding n responses in another image.

9. A method according to claim 1 wherein:

a. in a first preprocessing stage, for a database of m lights E_i (λ

) and n surfaces S_j (λ

) there is calculated T_i≈

Q_i^FQ_i⁺ where Q_i and Q_i^F represent the matrices of unfiltered and filtered sensor responses to the n surfaces under the i th light and + denotes an inverse, and

b. in a second operation stage, given P surfaces in an image and 3×

P matrices Q and Q^F, from these matrices there are chosen the fraction of the brightest pixels, giving the matrices Q′ and

Q′

^F and the scene illuminant ρ

_est is estimated where

10. A method of chromagenic illuminant estimation according to claim 1, further including the steps of:

a. removing the colour bias due to illumination from one of the images, and

b. rendering the image.

11. A method of chromagenic illuminant estimation according to claim 1, further comprising performing a gamut mapping process on the estimated illuminant.

12. A method of chromagenic illuminant estimation using mutually corresponding images with different sets of spectral components, wherein:

a. in a first preprocessing stage, for a database of m lights E_i (λ

) and n surfaces S_j (λ

) there is calculated T_i≈

Q_i^FQ_i⁺ where Q_i and Q_i^F represent the matrices of unfiltered and filtered sensor responses to the n surfaces under the i th light and + denotes an inverse, and

b. in a second operation stage, given P surfaces in an image and 3×

P matrices Q and Q^F, from these matrices there are chosen a particular fraction of the brightest pixels from each of the corresponding images in descending order of brightness from the brightest pixel in each of the corresponding images, giving the matrices Q′ and

Q′

^F, and the scene illuminant ρ

_est is estimated where

13. A method of chromagenic illuminant estimation according to claim 12 combined with a gamut mapping process.

14. A method of chromagenic illuminant estimation according to claim 12, further including:

the step of removing from the images the colour bias due to illumination.

15. An image treatment method comprising:

performing chromagenic illuminant estimation, in which, from mutually corresponding images with different sets of spectral components, a fraction of the brightest pixels are selected for subsequent chromagenic estimation,

wherein one of the mutually corresponding images is being treated to remove color bias,

wherein the selecting of the fraction of the brightest pixels comprises selecting a particular fraction of pixels from each of the corresponding images in descending order of brightness from the brightest pixel in each of the corresponding images; and

using the chromagenic illuminant estimation to remove colour bias from the image being treated.