DIGITAL IMAGE EXPOSURE CORRECTION

• US 20100329558A1
• Filed: 09/02/2010
• Published: 12/30/2010
• Est. Priority Date: 02/27/2003
• Status: Active Grant

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Inventors: Saquib, Suhail S., Thornton, Jay E.
• Granted Patent: US 8,265,420 B2
• Time in Patent Office: Days
• US Class Current: 382/171
• CPC Class Codes: H04N 1/4072 dependent on the contents of the original

Abstract

Techniques are disclosed for correcting the exposure of a digital image. An exposure predictor may be generated based on a set of images for which ground truth data are known. After identifying an optimal set of features, the exposure of the digital image may be corrected by extracting values of the selected optimal features from the image, using the predictor to predict a desired exposure correction for the image, and correcting the exposure of the image by the predicted desired amount. Exposure correction is based on a model that relates intensity of light in the world to the RGB digits of the digital image. The model comprises a gamma function that models the response of a typical monitor and a S-shaped curve that compresses the large dynamic range of the world to the small dynamic range of the RGB digit space.

Claims

1. A method for selecting a set of features for adjusting the exposure of images, the method comprising:

placing a set of features in a master feature set M;

initializing a current feature set C to a null value;

performing the following for each feature F in the master feature set M;

adding the current feature set C and the feature F to form a temporary feature set S;

calculating an error E using the set S as a feature set; and

performing at least one of the following;

determining that the error E is less than a minimum error E_MIN, assigning the value of E to E_MIN and recording the identity of feature F in a variable F_MIN; and

determining that the error E is greater than a minimum error E_MIN, leaving the values of E_MIN and F_MIN unchanged;

performing one of the following;

determining that E_MIN is less than a global error E_G, assigning the value of E_MIN to E_G, adding the feature F recorded F_MIN to the set C, and deleting the feature F recorded in F_MIN from the set M, and determining that the set M is empty; and

determining that E_MIN is less than a global error E_G, and leaving the value E_G, the set C, and the set M unchanged; and

selecting the set C as the set of features for adjusting the exposure of images.

2. The method of claim 1, wherein the calculating the error E comprises calculating the error E as a leave-n-out error value.

3. The method of claim 2, wherein the calculating the leave-n-out error E comprises generating a predicted exposure shift.

4. The method of claim 3, wherein the calculating the leave-n-out error E further comprises subtracting the predicted exposure shift from ground truth data for the images.

5. The method of claim 1, wherein the set of features comprises at least one feature from a group consisting of a region of interest histogram, an active histogram, an average histogram.

6. The method of claim 1, wherein the set of features comprises at least one feature from a group consisting of linear features and non-linear features.

7. The method of claim 1, wherein, after selecting the set C as the set of features for adjusting the exposure of images, the value E_MIN represents the minimum leave-n-out prediction error of the features in the master feature set M, and the value F_MIN represents the feature from the mater feature set M corresponding to the value E_MIN.

8. A tangible computer readable medium comprising instructions to select a set of features for use in a system for adjusting the exposure of images, the instructions comprising:

instructions to place a set of features in a master feature set M;

instructions to initialize a current feature set C to a null value;

instructions to perform the following for each feature F in the master set M;

adding the current feature set C and the feature F to form a temporary feature set S;

calculating an error E using the set S as a feature set; and

performing at least one of the following;

determining that the error E is less than a minimum error E_MIN, assigning the value of E to E_MIN and recording the identity of feature F in a variable F_MIN; and

determining that the error E is greater than a minimum error EMIN, leaving the values of E_MIN and F_MIN unchanged; and

instructions to perform one of the following;

determining that E_MIN is less than a global error E_G, assigning the value of E_MIN to E_G, adding the feature F recorded in F_MIN to the set C, deleting the feature F recorded in F_MIN from the set M, and determining that the set M is empty; and

determining that E_MIN is less than a global error E_G, and leaving the value E_G, the set C, and the set M unchanged; and

instructions to select the set C as the set of features for use in the system for adjusting the exposure of images.

9. The tangible computer readable medium of claim 8, wherein the calculating the error E comprises calculating the error E as a leave-n-out error value.

10. The tangible computer readable medium of claim 9, wherein the calculating the leave-n-out error E comprises generating a predicted exposure shift.

11. The tangible computer readable medium of claim 10, wherein the calculating the leave-n-out error E further comprises subtracting the predicted exposure shift from ground truth data for the images.

12. The tangible computer readable medium of claim 8, wherein the set of features comprises at least one feature from a group consisting of a region of interest histogram, an active histogram, an average histogram.

13. The tangible computer readable medium of claim 8, wherein the set of features comprises at least one feature from a group consisting of linear features and non-linear features.

14. The tangible computer readable medium of claim 8, the instructions further comprising:

instructions to identify, after selecting the set C as the set of features for adjusting the exposure of images, the value E_MIN as representing the minimum leave-n-out prediction error of the features in the master feature set M, and to identify the value F_MIN as representing the feature from the mater feature set M corresponding to the value E_MIN.

15. A system for selecting a set of features for adjusting the exposure of images, the system comprising:

means for placing a set of features in a master feature set M;

means for initializing a current feature set C to a null value;

means for performing the following for each feature F in the master feature set M;

adding the current feature set C and the feature F to form a temporary feature set S;

calculating an error E using the set S as a feature set; and

performing at least one of the following;

determining that the error E is less than a minimum error E_MIN, assigning the value of E to E_MIN and recording the identity of feature F in a variable F_MIN; and

determining that the error E is greater than a minimum error E_MIN, leaving the values of E_MIN and F_MIN unchanged;

means for performing one of the following;

determining that E_MIN is less than a global error E_G, assigning the value of E_MIN to E_G, adding the feature F recorded F_MIN to the set C, and deleting the feature F recorded in F_MIN from the set M, and determining that the set M is empty; and

determining that E_MIN is less than a global error E_G, and leaving the value E_G, the set C, and the set M unchanged; and

means for selecting the set C as the set of features for adjusting the exposure of images.

16. The system of claim 15, wherein the calculating the error E comprises calculating the error E as a leave-n-out error value.

17. The system of claim 16, wherein the calculating the leave-n-out error E comprises generating a predicted exposure shift.

18. The system of claim 17, wherein the calculating the leave-n-out error E further comprises subtracting the predicted exposure shift from ground truth data for the images.

19. The system of claim 15, wherein the set of features comprises at least one feature from a group consisting of a region of interest histogram, an active histogram, an average histogram.

20. The system of claim 15, wherein the set of features comprises at least one feature from a group consisting of linear features and non-linear features.

21. The system of claim 15, further comprising:

means for identifying, after selecting the set C as the set of features for adjusting the exposure of images, the value E_MIN as representing the minimum leave-n-out prediction error of the features in the master feature set M, and for identifying the value F_MIN as representing the feature from the mater feature set M corresponding to the value E_MIN.

22. A method for processing an image, the method comprising:

receiving the image from an image capture device;

extracting, from the image, values of at least one feature, wherein the at least one feature is a feature derived from a thumbnail of the image;

predicting a desired exposure correction of the image based on the extracted values; and

correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image.

23. The method of claim 22, wherein the thumbnail has a base size that is proportional to the dimensions of the image, and wherein the dimensions of the thumbnail are proportional to the base size multiplied by a measure of average activity in the image.

24. The method of claim 23, wherein the thumbnail comprises a plurality of pixels satisfying an activity threshold.

25. The method of claim 24, wherein an average activity within the thumbnail is above a predetermined minimum activity threshold.

26. The method of claim 22, wherein the at least one feature comprises an average histogram.

27. The method of claim 22, wherein the extracting comprises determining a region of interest within the thumbnail.

28. The method of claim 27, wherein the extracting further comprises calculating a likelihood that the region of interest influences the desired exposure shift correction.

29. The method of claim 28, wherein the likelihood that the region of interest influences the desired exposure shift correction is calculated using the formula:

30. A tangible computer readable medium comprising instructions to process an image, the instructions comprising:

instructions to receive the image from an image capture device;

instructions to extract, from the image, values of at least one feature, wherein the at least one feature is a feature derived from a thumbnail of the image;

instructions to predict a desired exposure correction of the image based on the extracted values; and

instructions to correct the exposure of the image by the predicted exposure correction to produce an exposure-corrected image.

31. The tangible computer readable medium of claim 30, wherein the thumbnail has a base size that is proportional to the dimensions of the image, and wherein the dimensions of the thumbnail are proportional to the base size multiplied by a measure of average activity in the image.

32. The tangible computer readable medium of claim 31, wherein the thumbnail comprises a plurality of pixels satisfying an activity threshold.

33. The tangible computer readable medium of claim 32, wherein an average activity within the thumbnail is above a predetermined minimum activity threshold.

34. The tangible computer readable medium of claim 30, wherein the at least one feature comprises an average histogram.

35. The tangible computer readable medium of claim 30, wherein the instructions to extract comprise instructions to determine a region of interest within the thumbnail.

36. The tangible computer readable medium of claim 35, wherein the instructions to extract further comprise instructions to calculate a likelihood that the region of interest influences the desired exposure shift correction.

37. The tangible computer readable medium of claim 36, wherein the likelihood that the region of interest influences the desired exposure shift correction is calculated using the formula:

38. A system for processing an image, the system comprising:

means for receiving the image from an image capture device;

means for extracting, from the image, values of at least one feature, wherein the at least one feature is a feature derived from a thumbnail of the image;

means for predicting a desired exposure correction of the image based on the extracted values; and

means for correcting the exposure of the image by the predicted exposure correction to produce an exposure-corrected image.

39. The system of claim 38, wherein the thumbnail has a base size that is proportional to the dimensions of the image, and wherein the dimensions of the thumbnail are proportional to the base size multiplied by a measure of average activity in the image.

40. The system of claim 39, wherein the thumbnail comprises a plurality of pixels satisfying an activity threshold.

41. The system of claim 40, wherein an average activity within the thumbnail is above a predetermined minimum activity threshold.

42. The system of claim 38, wherein the at least one feature comprises an average histogram.

43. The system of claim 38, wherein the means for extracting comprises means for determining a region of interest within the thumbnail.

44. The system of claim 43, wherein the means for extracting further comprises means for calculating a likelihood that the region of interest influences the desired exposure shift correction.

45. The system of claim 44, wherein the likelihood that the region of interest influences the desired exposure shift correction is calculated using the formula: