Method, system and software for correcting image defects

• US 20030118249A1
• Filed: 04/19/2002
• Published: 06/26/2003
• Est. Priority Date: 04/19/2001
• Status: Active Grant

• Current Assignee: Kodak Alaris Inc (KPP (no. 2) Trustees Ltd.)
• Original Assignee: Eastman Kodak Company
• Inventors: Edgar, Albert D.
• Granted Patent: US 6,987,892 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 2207/10008 from scanner, fax or copier
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30144 Printing quality
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

A method, system and software are disclosed for correcting defects formed in a physical medium of an original image. Multiple scans of the original image are recorded, where the multiple scans have different properties. For example, the angle of the light incident to the physical medium or the properties of the light may be changed between scans of original image. The multiple scans can be used to generate a reference image from which defect corrections are made. Alternatively, a reference image can be generated directly from the original image. The multiple scans can also be used to determine the degree of defectiveness and/or an estimate of the signal strength of each portion of the original image. A decision is made on whether or not an image portion having one or more defects should be corrected, where the decision can be based on an evaluation of the potential benefit compared to the potential damage caused by correction of an image portion. In one embodiment, the potential benefit is proportional to the degree of defectiveness, while the potential damage is proportional to the image information that may be removed by correction. If the decision is made to correct an image portion, a variety of methods may be implemented to correct the image portion, such as cloning information from non-defective image portions surrounding the defective image portion. The present invention finds particular use in image capturing systems, such as flatbed scanners, photocopiers, facsimile machines, and the like.

Claims

1. A method comprising:

determining a degree of defectiveness of an image portion;

evaluating, based at least in part on the degree of defectiveness, a benefit of correcting the image portion and damage that would be caused by correcting the image portion; and

correcting the image portion if the benefit of correcting the image portion is greater than the damage that would be caused by correcting the image portion.

2. The method as in claim 1, wherein evaluation includes determining a ratio of the benefit to the damage.

3. The method as in claim 2, wherein the ratio is compared with a threshold.

4. The method as in claim 3, wherein the threshold is determined according to the formula

5. The method as in claim 4, wherein the scaling_factor is 1.4.

6. The method as in claim 1, wherein:

the benefit is proportional to the degree of defectiveness of the image portion; and

damage is proportional to a degree of signal that would be removed by correcting the image portion.

7. The method as in claim 1, wherein evaluating includes comparing an estimate of a signal strength of the image portion to a threshold value, the threshold value dependent on an expected signal strength of the image portion.

8. The method as in claim 1, wherein the degree of defectiveness is determined in relation to image detail in areas of the image surrounding the image portion.

9. The method as in claim 1, wherein determining the degree of defectiveness of an image portion includes:

recording a first representation of the image portion using a light source having a first position relative to the image;

recording a second representation of the image portion using a light source having a second position relative to the image, the second position different from the first position;

low-pass filtering the first representation to generate a first filtered representation;

low-pass filtering the second representation to generate a second filtered representation;

dividing the first representation by the first filtered representation to generate a first output; and

dividing the second representation by the second filtered representation to generate a second output.

10. The method as in claim 9, further comprising:

comparing the first output and the second output; and

outputting a signal strength estimate based on the comparison, wherein;

the lesser of the first output and the second output is output as the signal strength estimate when both the first output and the second output are greater than one;

the greater of the first output and the second output is output as the signal strength estimate when both the first output and the second output are less than one; and

a value of one is output as the signal strength estimate when one of the first output and the second output is greater than one, and the other of the first output and the second output is less than one.

11. The method as in claim 9, further including estimating a defect strength, wherein the defect strength is an absolute value of a difference between the first output and the second output.

12. The method as in claim 1, further including performing a grow operation on the image portion followed by a shrink operation on the image portion.

13. The method as in claim 12, wherein:

the grow operation is performed over a radius of four; and

the shrink operation is performed over a radius of three.

14. The method as in claim 12, wherein radii of the grow operation and the shrink operation may be independently changed.

15. The method as in claim 1, wherein evaluation includes generating a goodness map of an image based on the degree of defectiveness of a plurality of image portions.

16. The method as in claim 1, wherein defective portions of an image are corrected by cloning appropriate non-defective portions of the image into the defective portions.

17. The method as in claim 16, wherein the non-defective portions of the image used for cloning are chosen dependent on the direction of the defective portions.

18. The method as in claim 16, wherein the non-defective portions of the image used for cloning are different for different frequency components of the image.

19. The method as in claim 18, wherein the non-defective portions of the image used for cloning are chosen to be one fundamental wavelength away from the defective portions.

20. The method as in claim 16, wherein cloning includes using non-defective portions of the image on both sides of the defective portions.

21. The method as in claim 1, wherein correcting the image portion includes using pyramidal decomposition.

21-1. A system comprising:

at least one processor;

memory operably associated with said processor; and

a program of instructions to be stored in the memory and executed by the processor, wherein the program of instructions include instructions to;

determine a degree of defectiveness of an image portion;

evaluate, based at least in part on the degree of defectiveness, a benefit of correcting the image portion and damage that would be caused by correcting the image portion; and

correct the image portion based on the evaluation.

22. The system as in claim 22, wherein the instructions to evaluate include instructions to determine a ratio of the benefit to the damage.

26. The system as in claim 22, wherein:

the benefit is proportional to the degree of defectiveness of the image portion; and

the damage is proportional to a degree of signal that would be removed by correcting the image portion.

27. The system as in claim 22, wherein the instructions to evaluate include instructions to compare an estimate of a signal strength of the image portion to a threshold value, the threshold value dependent on an expected signal strength of the image portion.

28. The system as in claim 22, wherein the degree of defectiveness is determined in relation to image detail in areas of the image surrounding the image portion.

29. The system as in claim 22, wherein the instructions to determine the degree of defectiveness of an image portion include instructions to:

record a first representation of the image portion using a light source having a first position relative to the image;

record a second representation of the image portion using a light source having a second position relative to the image, the second position different from the first position;

low-pass filter the first representation to generate a first filtered representation;

low-pass filter the second representation to generate a second filtered representation;

divide the first representation by the first filtered representation to generate a first output; and

divide the second representation by the second filtered representation to generate a second output.

32. The system as in claim 22, wherein said program of instructions further includes instructions to perform a grow operation on the image portion followed by a shrink operation on the image portion.

35. The system as in claim 22, wherein the instructions to evaluate include instructions to generate a goodness map of an image based on the degree of defectiveness of a plurality of image portions.

36. The system as in claim 22, wherein defective portions of an image are corrected by cloning appropriate non-defective portions of the image into the defective portions.

41. The system as in claim 22, wherein correcting the image portion includes using pyramidal decomposition.

23. The system as in claim 23, wherein the instructions to evaluate include instructions to compare the ratio with a threshold.

24. The system as in claim 24, wherein the threshold is determined according to the formula:

25. The system as in claim 25, wherein the scaling_factor is 1.4.

30. The system as in claim 30, wherein said program of instructions further includes instructions to:

compare the first output and the second output; and

output a signal strength estimate based on the comparison wherein;

the lesser of the first output and the second output is output as the signal strength estimate when both the first output and the second output are greater than one;

the greater of the first output and the second output is output as the signal strength estimate when both the first output and the second output are less than one; and

a value of one is output as the signal strength estimate when one of the first output and the second output is greater than one, and the other of the first output and the second output is less than one.

31. The system as in claim 30, wherein said program of instructions further includes instructions to estimate a defect strength, wherein the defect strength is an absolute value of a difference between the first output and the second output.

33. The system as in claim 33, wherein:

the grow operation is performed over a radius of four; and

the shrink operation is performed over a radius of three.

34. The system as in claim 33, wherein radii of the grow operation and the shrink operation may be independently changed.

37. The system as in claim 37, wherein the non-defective portions of the image used for cloning are chosen dependent on the direction of the defective portions.

38. The system as in claim 37, wherein the non-defective portions of the image used for cloning are different for different frequency components of the image.

39. The system as in claim 37, wherein the non-defective portions of the image used for cloning are chosen to be one fundamental wavelength away from the defective portions.

40. The system as in claim 37, wherein cloning includes using non-defective portions of the image on both sides of the defective portions.

42. A computer readable medium tangibly embodying a program of instructions, said program of instructions including instructions to:

determine a degree of defectiveness of an image portion;

evaluate, based at least in part on the degree of defectiveness, a benefit of correcting the image portion and damage that would be caused by correcting the image portion; and

determine whether to correct the image portion based on the evaluation.

43. The computer readable medium as in claim 43, wherein the instructions to evaluate include instructions to determine a ratio of the benefit to the damage.

47. The computer readable medium as in claim 43, wherein:

the benefit is proportional to the degree of defectiveness of the image portion and the damage is proportional to a degree of signal that would be removed by correcting the image portion.

48. The computer readable medium as in claim 43, wherein the instructions to evaluate include instructions to compare an estimate of a signal strength of the image portion to a threshold value, the threshold value dependent on an expected signal strength of the image portion.

49. The computer readable medium as in claim 43, wherein the degree of defectiveness is determined in relation to image detail in areas of the image surrounding the image portion.

50. The computer readable medium as in claim 43, wherein the instructions to determine the degree of defectiveness of an image portion include instructions to:

record a first representation of the image portion using a light source having a first position relative to the image;

record a second representation of the image portion using a light source having a second position relative to the image, the second position different from the first position;

low-pass filter the first representation to generate a first filtered representation;

low-pass filter the second representation to generate a second filtered representation;

divide the first representation by the first filtered representation to generate a first output; and

divide the second representation by the second filtered representation to generate a second output.

53. The computer readable medium as in claim 43, wherein said program of instructions further includes instructions to perform a grow operation on the image portion followed by a shrink operation on the image portion.

56. The computer readable medium as in claim 43, wherein the instructions to evaluate include instructions to generate a goodness map of an image based on the degree of defectiveness of a plurality of image portions.

57. The computer readable medium as in claim 43, wherein defective portions of an image are corrected by cloning appropriate non-defective portions of the image into the defective portions.

62. The computer readable medium as in claim 43, wherein correcting the image portion includes using pyramidal decomposition.

44. The computer readable medium as in claim 44, wherein the instructions to evaluate include instructions to compare the ratio with a threshold.

45. The computer readable medium as in claim 45, wherein the threshold is determined according to the formula

46. The computer readable medium as in claim 45, wherein the scaling_factor is 1.4.

51. The computer readable medium as in claim 51, wherein said program of instructions further includes instructions to:

compare the first output and the second output; and

output a signal strength estimate based on the comparison wherein;

the lesser of the first output and the second output is output as the signal strength estimate when both the first output and the second output are greater than one;

the greater of the first output and the second output is output as the signal strength estimate when both the first output and the second output are less than one; and

a value of one is output as the signal strength estimate when one of the first output and the second output is greater than one, and the other of the first output and the second output is less than one.

52. The computer readable medium as in claim 51, wherein said program of instructions further includes instructions to estimate a defect strength, wherein the defect strength is an absolute value of a difference between the first output and the second output.

54. The computer readable medium as in claim 51, wherein:

the grow operation is performed over a radius of four; and

the shrink operation is performed over a radius of three.

55. The computer readable medium as in claim 54, wherein radii of the grow operation and the shrink operation may be independently changed.

58. The computer readable medium as in claim 58, wherein the non-defective portions of the image used for cloning are chosen dependent on the direction of the defective portions.

59. The computer readable medium as in claim 58, wherein the non-defective portions of the image used for cloning are different for different frequency components of the image.

61. The computer readable medium as in claim 58, wherein cloning includes using non-defective portions of the image on both sides of the defective portions.

60. The computer readable medium as in claim 60, wherein the non-defective portions of the image used for cloning are chosen to be one fundamental wavelength away from the defective portions.