Illumination estimation for images

• US 8,391,634 B1
• Filed: 04/28/2009
• Issued: 03/05/2013
• Est. Priority Date: 04/28/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kwatra, Vivek, Han, Mei, Dai, Shengyang
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Thirugnanam, Gandhi
• Application Number:
  US12/431,629
• Time in Patent Office: 1,407 Days
• Field of Search: 382144-145, 382/151, 382/274
• US Class Current: 382/274
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model

Abstract

An image comprising varying illumination is selected. Instances of a repeating pattern in the image is determined. Illumination values for pixels at locations within instances of the repeating pattern are calculated responsive to pixel intensities of pixels at corresponding locations in other instances of the repeating pattern. The varying illumination is removed form the image responsive to the illumination values.

Claims

1. A computer-implemented method of removing illumination variation from an image, the method performed by a computer and comprising:

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

calculating inverse illumination values for pixels at locations within instances of the repeating pattern responsive to pixel intensities of pixels at corresponding locations in other instances of the repeating pattern; and

for pixels in the image, multiplying a pixel intensity of a pixel by an inverse illumination value calculated for the pixel to remove the illumination variation from the image responsive to the calculation of the inverse illumination values.

2. The computer-implemented method of claim 1, wherein determining instances of the repeating pattern comprises:

determining a period of the repeating pattern in a dimension by determining differences in pixel intensities among the image and versions of the images shifted in the dimension.

3. The computer-implemented method of claim 1, further comprising:

determining dimensions of the repeating pattern in the image;

defining regions of pixels of the image responsive to determining the dimensions of the repeating pattern; and

determining mean pixel intensities of pixels within the defined regions;

wherein the inverse illumination values for the pixels are determined responsive to the mean pixel intensities.

4. The computer-implemented method of claim 1, wherein calculating inverse illumination values for pixels comprises:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between the pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern.

5. The computer-implemented method of claim 4, wherein an inverse illumination value for a pixel within the instance of the repeating pattern is further calculated responsive to inverse illumination values of neighboring pixels.

6. The computer-implemented method of claim 1, further comprising:

determining dimensions of the repeating pattern in the image;

defining regions of pixels of the image responsive to the dimensions of the repeating pattern; and

determining mean pixel values of pixels within the defined regions.

7. The computer-implemented method of claim 1, wherein calculating inverse illumination values for pixels comprises:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern;

wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem.

8. A computer program product having a non-transitory computer-readable storage medium storing computer-executable code for removing illumination variation from an image, the code when executed performs steps comprising:

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

calculating inverse illumination values for pixels at locations within instances of the repeating pattern responsive to pixel intensities of pixels at corresponding locations in other instances of the repeating pattern; and

for pixels in the image, multiplying a pixel intensity of a pixel by an inverse illumination value calculated for the pixel to remove the illumination variation from the image responsive to the calculation of the inverse illumination values.

9. The computer program product of claim 8, wherein the code when executed is further configured to perform steps comprising:

determining dimensions of the repeating pattern in the image; and

defining regions of pixels of the image responsive to determining the dimensions of the repeating pattern;

determine mean pixel intensities of pixels within the defined region; and

wherein the illumination values for the pixels are determined responsive to the mean pixel intensities.

10. The computer program product of claim 8, wherein the code when executed is further configured to perform steps comprising:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between the pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern.

11. The computer program product of claim 10, wherein an inverse illumination value for a pixel within the instance of the repeating pattern is further calculated responsive to illumination values of neighboring pixels.

12. The computer program product of claim 8, wherein the code when executed is further configured to perform steps comprising:

determining dimensions of the repeating pattern in the image;

define regions of pixels of the image responsive to the dimensions of the repeating pattern; and

determine mean pixel values of pixels within the defined regions.

13. The computer program product of claim 8, wherein the code when executed is further configured to perform steps comprising:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern;

wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem.

14. A computer system for removing illumination variation from an image, the system comprising:

a computer processor; and

a computer-readable storage medium storing computer-executable code configured to execute on the computer processor, the code when executed performs steps comprising;

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

calculating inverse illumination values for pixels at locations within instances of the repeating pattern responsive to pixel intensities of pixels at corresponding locations in other instances of the repeating pattern; and

for pixels in the image, multiplying a pixel intensity of a pixel by an inverse illumination value calculated for the pixel to remove the illumination variation from the image responsive to the calculation of the illumination values.

15. The computer system of claim 14, wherein the code when executed is further configured to perform steps comprising:

determining dimensions of the repeating pattern in the image; and

defining regions of pixels of the image responsive to determining the dimensions of the repeating pattern;

determining mean pixel intensities of pixels within the defined region; and

wherein the inverse illumination values for the pixels are determined responsive to the mean pixel intensities.

16. The computer system of claim 14, wherein the code when executed is further configured to perform steps comprising:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between the pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern.

17. The computer system of claim 16, wherein an inverse illumination value for a pixel within the instance of the repeating pattern is further calculated responsive to inverse illumination values of neighboring pixels.

18. The computer system of claim 14, wherein the code when executed is further configured to perform steps comprising:

determining dimensions of the repeating pattern in the image;

defining regions of pixels of the image responsive to the dimensions of the repeating pattern; and

determining mean pixel values of pixels within the defined regions.

19. The computer system of claim 14, wherein the code when executed is further configured to perform steps comprising:

calculating an inverse illumination value for a pixel within the instance of the repeating pattern responsive to a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern;

wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem.

20. A computer-implemented method of removing illumination variation from an image, the method performed by a computer and comprising:

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

for each of a plurality of pixels in the image, calculating an illumination value for the pixel within an instance of the repeating pattern based at least in part on a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern, wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem; and

removing the illumination variation from the image based on the calculation of the illumination values.

21. The computer-implemented method of claim 20, wherein the illumination value for the pixel within the instance of the repeating pattern is further calculated based on illumination values of neighboring pixels.

22. A computer program product having a non-transitory computer-readable storage medium storing computer-executable code for removing illumination variation from an image, the code when executed performs steps comprising:

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

for each of a plurality of pixels in the image, calculating an illumination value for the pixel within an instance of the repeating pattern based at least in part on a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern, wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem; and

removing the illumination variation from the image based on the calculation of the illumination values.

23. The computer program product of claim 22, wherein the illumination value for the pixel within the instance of the repeating pattern is further calculated based on illumination values of neighboring pixels.

25. The computer system of claim 22, wherein the illumination value for the pixel within the instance of the repeating pattern is further calculated based on illumination values of neighboring pixels.

24. A computer system for removing illumination variation from an image, the system comprising:

a computer processor; and

a computer-readable storage medium storing computer-executable code configured to execute on the computer processor, the code when executed performs steps comprising;

selecting an image containing varying illumination;

determining instances of a repeating pattern in the image;

for each of a plurality of pixels in the image, calculating an illumination value for the pixel within an instance of the repeating pattern based at least in part on a difference between a pixel intensity of the pixel within the instance of the repeating pattern and a pixel intensity of a pixel at a corresponding location in an adjacent instance of the repeating pattern, wherein the calculation is performed as an optimization problem and wherein a mean pixel value for a defined region of the image is used as an initial value for the optimization problem; and

removing the illumination variation from the image based on the calculation of the illumination values.