DETECTING ILLUMINATION IN IMAGES

• US 20100098330A1
• Filed: 11/08/2007
• Published: 04/22/2010
• Est. Priority Date: 11/08/2006
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Drew, Mark Samuel, Finlayson, Graham D., Fredembach, Clement
• Granted Patent: US 8,385,648 B2
• Time in Patent Office: Days
• US Class Current: 382/164
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control

Abstract

An image having m light sources, with m preferably equaling 2 or 3, is sequenced into different regions, each of which is lit by only one of the m light sources, by obtaining paired imaged with different filtering, for example a filtered and an unfiltered image, applying to the image pairs sets of m pre-computed mappings at the pixel or region level, and selecting the most appropriate. The rendering of the information in the image maybe adjusted accordingly.

Claims

1-20. -20. (canceled)

21. A method of processing a captured image of a scene having m light sources, m being a natural number greater than 1, by segmenting the image into different regions wherein each region is lit by only one of the m light sources, the method including the steps of:

a. capturing a first image and a second image of the scene, wherein the first and second images are captured using different spectral components;

b. obtaining mappings between regions in the first image and corresponding regions in the second image, each mapping;

(1) being associated with only one of the m light sources, and

(2) providing a prediction of the spatially corresponding multispectral response in the second image on the basis of the multispectral response in the first image,

c. identifying one or more mappings having lower prediction error, and

d. obtaining the different regions of the image from one of the identified mappings having lower prediction error.

22. The method of claim 21 wherein the mappings between the first image and the second image are at the pixel level.

23. The method of claim 21 wherein the first image and the second image are counterparts of each other with different filtering.

24. The method of claim 21 further including the step of rendering an image of the scene, wherein the rendering is adjusted in dependence on the mapping of the images.

25. A method for processing an image having m light sources, m being a natural number greater than 1, by segmenting the image into different regions wherein each region is lit by only one of the m light sources, the method including the steps of:

a. obtaining paired images with different sets of spectral components, and

b. applying sets of m pre-computed mappings at the region level to the image pairs.

26. The method of claim 25 wherein the regions are pixels.

27. The method of claim 25 further including the step of selecting one of the pre-computed mappings, wherein the selected pre-computed mapping is selected in dependence on how well it predicts a particular response vector in the second image in the pair given a corresponding response vector in the first image in the pair.

28. The method of claim 25 further including the steps of:

a. obtaining the m pre-computed mappings by mapping the images using linear transforms, and

b. selecting the one of the pre-computed mappings which generates a new set of outputs which are closest to the outputs in the second image in the pair.

29. The method of claim 25 further including the step of selecting one of the pre-computed mappings, wherein statistical analysis is used to make the selection.

30. The method of claim 25 wherein:

a. each of the m pre-computed mappings is calculated across an image,

b. the difference between an actual secondary response and that predicted by each pre-computed mappings is summed across the image,

c. the goodness is assessed, and

d. the best pre-computed mapping is found by search.

31. The method of claim 25 wherein the paired images are counterparts of each other with different filtering.

32. The method of claim 25 wherein:

a. the first image in the pair is unfiltered, and

b. the second image in the pair is a filtered counterpart of the first image.

33. The method of claim 25 wherein:

a. the first image in the pair is produced by an RGB camera, and

b. the second image in the pair is a filtered RGB counterpart of the first image.

34. The method of claim 25 wherein:

a. the first image in the pair is produced by an RGB camera, and

b. the second image in the pair corresponds to a set of one or more sensor responses corresponding to the first image in the pair.

35. The method of claim 34 wherein the set of sensor responses includes CMY values.

36. The method of claim 25 wherein:

a. the first image is a multispectral image taken by a camera with at least two sensors, and

b. the second image is a multispectral image taken by at least one further sensor of the camera.

37. The method of claim 25 further including the step of determining which of the m pre-computed mappings best maps the first image in the pair to the second image in the pair.

38. The method of claim 25 further including the steps of:

a. finding a best mapping between the images;

b. assigning the majority of pixels transformed under the best mapping to a first label; and

c. assigning the remaining pixels to a second label.

39. The method of claim 25 further including the step of chromagenic processing of N illuminants (where N>

m) to produce N relations determining for each pixel which of the m relations of an m-element subset R best maps the two images.

40. The method of claim 25 further including the step of rendering one of the images, wherein the rendering is adjusted in dependence on the mapping of the images.

41. A method for processing an image having m light sources, m being a natural number greater than 1, by segmenting the image into different regions wherein each region is lit by only one of the m light sources, the method including the steps of:

a. obtaining paired images wherein the images in the pair have different sets of spectral components,

b. finding a best mapping between the images, and

c. assigning the majority of pixels best transformed under the best mapping to a first label, and

d. assigning the remaining pixels to a second label.

42. A method for processing an image having m of light sources by segmenting the image into different regions, each of which is lit by only one of the m light sources, the method comprising the steps of:

a. obtaining paired images wherein the images in the pair have different sets of spectral components,

b. chromagenic preprocessing of N illuminants (where N>

m) to produce N relations which determine for each region which of the m relations of an m-element subset R best maps the two images.