System and method for deciding when to correct image-specific defects based on camera, scene, display and demographic data

• US 20030095197A1
• Filed: 09/20/2001
• Published: 05/22/2003
• Est. Priority Date: 09/20/2001
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Wheeler, Richard B., Keelan, Brian W.
• Granted Patent: US 7,133,070 B2
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: H04N 1/40 Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A captured image is processed with one or more correction processes selected from a plurality of such processes, each associated with correction of a specific type of image defect, in order to improve the appearance of a viewed image generated from the captured image. Preliminary to the image processing, meta data related to image capture is obtained that is unique to each captured image, where the meta data is capable of indicating whether the specific types of image defects are likely to be present in the viewed image generated from the captured image. The processing then involves predicting the presence of the image defects based at least in part on the meta data, thereby generating process application criteria which indicate a level of image defect that if left untreated would reduce the perceived quality of the viewed image; selecting one or more correction processes to employ on the captured image based on the process application criteria; and applying the one or more selected correction processes to the captured image to generate the viewed image.

Claims

1. A method for processing a captured image with one or more correction processes selected from a plurality of such processes, each associated with correction of a specific type of image defect, in order to improve the appearance of a viewed image generated from the captured image, said method comprising the steps of:

collecting meta data related to image capture that is unique to each image that is captured, wherein the meta data is capable of indicating whether the specific types of image defects are likely to be present in the viewed image generated from the captured image;

predicting the presence of the image defects based at least in part on the meta data, thereby generating process application criteria which indicate a level of image defect that if left untreated would reduce the perceived quality of the viewed image;

selecting one or more correction processes to employ on the captured image based on the process application criteria; and

applying the one or more selected correction processes to the captured image to generate the viewed image.

2. The method as claimed in claim 1 wherein the meta data includes scene, camera or demographic data related to the image capture.

3. The method as claimed in claim 1 wherein the step of predicting the presence of the image defects also predicts the severity of the defects and the strength of the corresponding correction process can be altered in response to the degree of severity.

4. The method as claimed as in claim 1 wherein the meta data related to image capture is collected at the time of image capture.

5. The method as claimed as in claim 1 wherein the meta data related to image capture is collected at a time other than the time of image capture.

6. The method as claimed in claim 1 wherein the image defect is a noise defect and the meta data is selected from the group consisting of a lens exposure constant, standard (printing) reproduction magnification, non-standard (enlargement) magnification, diagonal dimension of a final display, ambient light level of the primary subject, exposure time, camera lens f-number, and main flash guide number.

7. The method as claimed in claim 1 wherein the image defect is a red-eye defect due to flash illumination of a subject and the meta data is selected from the group consisting of a use (on-off) of the flash illumination, illumination level of the primary subject, subject distance, flash-to-camera lens separation, focal length of camera lens, current reproduction (printing) magnification, diagonal dimension of final display, and preflash guide number.

8. The method as claimed in claim 1 wherein the image defect is a tone scale defect and the meta data is selected from the group consisting of respective illumination levels of the subject and background, subject distance, background distance, exposure time, camera lens f-number, use (on/off) of flash illumination, guide number of a main flash, lens exposure constant and ISO speed of a capture device.

9. The method as claimed in claim 1 wherein the image defect is a sharpness defect and the meta data is selected from the group consisting an exposure time, standard reproduction (printing) magnification, non-standard (enlargement) magnification, current reproduction (printing) magnification, camera lens focus range, camera lens focal length, camera shake factor and linear smear.

10. The method as claimed in claim 1 further comprising the step of collecting meta data related to display parameters of the viewed image generated from each image that is captured, wherein said meta data is capable of indicating whether the specific types of image defects are likely to be present in the viewed image.

11. A computer storage medium having instructions stored therein for causing a computer to perform the method of claim 1.

12. An image processing method incorporating a plurality of image defect location and correction processes, each associated with correction of a specific type of image defect, that are selectively and automatically applied to an image only when the image is predicted from specific capture variables associated with image capture to have a level of image defect, treatable by one or more of the processes, that if left untreated would reduce the perceived quality of the final viewed image.

13. The method as claimed in claim 12 wherein the image capture is a film capture on a film capture device and the capture variables includes at least one variable selected from the group consisting of an ISO rating of the film used, exposure time of an optical system used in the capture device, a f-number of the optical system, a distance of a subject from the capture device, a distance of a background from the capture device, an illumination level on the subject, an illumination level on the background, a distance between a flash illuminator and the optical system, an indication of whether the flash illuminator was used, a guide number for the flash illuminator, an indication of camera shake and an indication that a backlight indicator was turned on.

14. The method as claimed in claim 12 wherein the image capture is an electronic capture on a digital capture device and the capture variables includes at least one variable selected from the group consisting of an ISO equivalent rating of an image sensor used by the capture device, exposure time of an optical system used in the capture device, a f-number of the optical system, a distance of a subject from the capture device, a distance of a background from the capture device, an illumination level on the subject, an illumination level on the background, a distance between a flash illuminator and the optical system, an indication of whether the flash illuminator was used, a guide number for the flash illuminator, an indication of camera shake, an indication that a backlight indicator was turned on, a gain factor for the image sensor and a resolution setting of the capture device.

15. The method as claimed in claim 14 wherein the electronic capture is obtained with a digital camera.

16. The method as claimed in claim 14 wherein the electronic capture is obtained with a digital scanner.

17. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a image defect is likely to be present in the final viewed image;

b) storing and later transferring to a photofinishing control system at least one of said parameters;

c) employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the image will have a level of image defect that if left untreated would reduce the perceived quality of the final viewed image; and

d) applying to the image via automated or operator-assisted techniques, image defect location and correction means, when said prediction means indicates a level of image defect that if left untreated would reduce the perceived quality of the final viewed image.

18. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a image defect is likely to be present in the final viewed image;

b) employing in a camera control system a means for predicting from at least one of said image capture parameters, and from predefined or photographer-selected printing parameters, whether the current image will have a level of image defect that if left untreated would reduce the perceived quality of the final viewed image;

c) storing and later transferring to the photofinishing control system a value indicating whether or not image defect location and correction means should be applied in photofinishing; and

d) applying to the image via automated or operator-assisted techniques, image defect location and correction means, when said image defect prediction value indicates a level of image defect that if left untreated would reduce the perceived quality of the final viewed image.

19. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a noise defect is likely to be present in the final viewed image;

b) storing and later transferring to a photofinishing control system at least one of said parameters;

c) employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the image will have a level of noise defect that if left untreated would reduce the perceived quality of the final viewed image; and

d) applying to the image via automated or operator-assisted techniques, noise defect location and correction means, when said noise defect prediction means indicates a level of noise defect that if left untreated would reduce the perceived quality of the final viewed image.

20. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a noise defect is likely to be present in the final viewed image;

b) employing in a camera control system a means for predicting from at least one of said image capture parameters, and predefined or photographer-selected printing parameters, whether the image will have a level of noise defect that if left untreated would reduce the perceived quality of the final viewed image;

c) storing and later transferring to a photofinishing control system a value indicating whether or not noise defect location and correction means should be applied in photofinishing; and

d) applying to the image via automated or operator-assisted techniques, noise defect location and correction means, when said noise defect prediction value indicates a level of noise defect that if left untreated would reduce the perceived quality of the final viewed image.

21. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a redeye defect is likely to be present in the final viewed image;

b) storing and later transferring to a photofinishing control system at least one of said parameters;

c) employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the image will have a level of redeye defect that if left untreated would reduce the perceived quality of the final viewed image; and

d) applying to the image via automated or operator-assisted techniques, redeye defect location and correction means, when said redeye defect prediction means indicates a level of redeye defect that if left untreated would reduce the perceived quality of the final viewed image.

22. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a redeye defect is likely to be present in the final viewed image;

b) employing in a camera control system a means for predicting from at least one of said image capture parameters, and predefined or photographer-selected printing parameters, whether the image will have a level of redeye defect that if left untreated would reduce the perceived quality of the final viewed image;

c) storing and later transferring to a photofinishing control system a value indicating whether or not redeye defect location and correction means should be applied in photofinishing; and

d) applying to the image via automated or operator-assisted techniques, redeye defect location and correction means, when said redeye defect prediction value indicates a level of redeye defect that if left untreated would reduce the perceived quality of the final viewed image.

23. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a tone scale defect is likely to be present in the final viewed image;

b) storing and later transferring to a photofinishing control system at least one of said parameters;

c) employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the image will have a level of tone scale defect that if left untreated would reduce the perceived quality of the final viewed image; and

d) applying to the image via automated or operator-assisted techniques, tone scale defect location and correction means, when said tone scale defect prediction means indicates a level of tone scale defect that if left untreated would reduce the perceived quality of the final viewed image.

24. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a tone scale defect is likely to be present in the final viewed image;

b) employing in a camera control system a means for predicting from at least one of said image capture parameters, and predefined or photographer-selected printing parameters, whether the image will have a level of tone scale defect that if left untreated would reduce the perceived quality of the final viewed image;

c) storing and later transferring to a photofinishing control system a value indicating whether or not tone scale defect location and correction means should be applied in photofinishing; and

d) applying to the image via automated or operator-assisted techniques, tone scale defect location and correction means, when said tone scale defect prediction value indicates a level of tone scale defect that if left untreated would reduce the perceived quality of the final viewed image.

25. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one parameter known to indicate whether a sharpness defect is likely to be present in the final viewed image;

b) storing and later transferring to a photofinishing control system at least one of said parameters;

c) employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the current image will have a level of sharpness defect that if left untreated would reduce the perceived quality of the final viewed image; and

d) applying to the image via automated or operator-assisted techniques, sharpness defect location and correction means, when said sharpness defect prediction means indicates a level of sharpness defect that if left untreated would reduce the perceived quality of the final viewed image.

26. The image processing method of claim 12 further comprising the steps of:

a) recording at the time each image is captured, at least one image capture parameter known to indicate whether a sharpness defect is likely to be present in the final viewed image;

b) employing in a camera control system a means for predicting from at least one of said image capture parameters, and predefined or photographer-selected printing parameters, whether the image will have a level of sharpness defect that if left untreated would reduce the perceived quality of the final viewed image;

c) storing and later transferring to a photofinishing control system a value indicating whether or not sharpness defect location and correction means should be applied in photofinishing; and

d) applying to the image via automated or operator-assisted techniques, sharpness defect location and correction means, when said sharpness defect prediction value indicates a level of sharpness defect that if left untreated would reduce the perceived quality of the final viewed image.

27. An image processing method incorporating an image defect location and correction process that is applied to an image only when a demographic characteristic of the image is predicted to cause a level of image defect that if left untreated would reduce the perceived quality of the final viewed image.

28. The method as claimed in claim 27 wherein the image defect is a red-eye defect due to flash illumination of a human subject and the image defect location and correction process is a red-eye correction process.

29. The method as claimed in claim 28 wherein the demographic characteristic of the image includes the age or race of the human subject.

30. A system for processing a captured image with one or more correction processes selected from a plurality of such processes, each associated with correction of a specific type of image defect, in order to improve the appearance of a viewed image generated from the captured image, said system comprising:

a data recording stage for collecting and recording meta data related to image capture that is unique to each image that is captured, wherein the meta data is capable of indicating whether the specific types of image defects are likely to be present in the viewed image generated from the captured image;

a digital image processor for (a) predicting the presence of the image defects based at least in part on the meta data, thereby generating process application criteria which indicate a level of image defect that if left untreated would reduce the perceived quality of the viewed image and (b) selecting one or more correction processes to employ on the captured image based on the process application criteria; and

means for applying the one or more selected correction processes to the captured image to generate the viewed image.

31. The system as claimed in claim 30 wherein the meta data includes scene, camera or demographic data related to the image capture.

32. The system as claimed in claim 30 wherein the digital signal processor also predicts the severity of the defects based at least in part on the meta data and said means for applying the selected correction process alters the strength of the corresponding correction process in response to the degree of severity.

33. The system as claimed as in claim 30 wherein the data recording stage is part of an image capture device and the meta data related to image capture is collected at the time of image capture.

34. The system as claimed as in claim 30 wherein the data recording stage is separated from image capture and the meta data related to image capture is collected at a time other than the time of image capture.

35. The system as claimed in claim 30 further comprising means for collecting meta data related to display parameters of the viewed image, wherein said meta data related to display parameters is capable of indicating whether the specific types of image defects are likely to be present in the viewed image.

36. A system for processing a captured image with one or more correction processes selected from a plurality of such processes, each associated with correction of a specific type of image defect, in order to improve the appearance of a final viewed image generated from the captured image, said system comprising:

a) an image capture device utilizing a camera control system to capture the image;

b) a photofinishing system utilizing a photofinishing control system to produce the final viewed image;

c) means for recording at the time each image is captured, at least one image capture parameter known to indicate whether an image defect is likely to be present in the final viewed image;

d) means for storing and later transferring to the photofinishing control system at least one of said parameters;

e) means for employing in the photofinishing control system a means for predicting from at least one of said image capture parameters and current printing parameters, whether the captured image will have a level of image defect that if left untreated would reduce the perceived quality of the final viewed image; and

f) means for applying to the captured image via automated or operator-assisted techniques, image defect location and correction means, when said prediction means indicates a level of image defect that if left untreated would reduce the perceived quality of the final viewed image.

37. A system for processing a captured image with one or more correction processes selected from a plurality of such processes, each associated with correction of a specific type of image defect, in order to improve the appearance of a final viewed image generated from the captured image, said system comprising:

a) an image capture device utilizing a camera control system to capture the image;

b) a photofinishing system utilizing a photofinishing control system to produce the final viewed image;

c) means for recording at the time each image is captured, at least one image capture parameter known to indicate whether a image defect is likely to be present in the final viewed image;

d) means for employing in the camera control system a means for predicting from at least one of said image capture parameters, and from predefined or photographer-selected printing parameters, whether the captured image will have a level of image defect that if left untreated would reduce the perceived quality of the final viewed image;

e) means for storing and later transferring to the photofinishing control system a value indicating whether or not image defect location and correction means should be applied in photofinishing, and

f) means for applying to the captured image via automated or operator-assisted techniques, image defect location and correction means, when said image defect prediction value indicates a level of image defect that if left untreated would reduce the perceived quality of the final viewed image.

38. An image processing system incorporating a plurality of image defect location and correction processes, each associated with correction of a specific type of image defect, that are selectively and automatically applied to an image only when the image is predicted from specific capture variables associated with image capture to have a level of image defect, treatable by one or more of the processes, that if left untreated would reduce the perceived quality of the final viewed image.