Detection and removal of blemishes in digital images utilizing original images of defocused scenes

• US 8,009,208 B2
• Filed: 02/19/2010
• Issued: 08/30/2011
• Est. Priority Date: 02/14/2006
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Zamfir, Adrian, Bigioi, Petronel
• Primary Examiner(s): Daniels; Anthony J
• Application Number:
  US12/708,654
• Publication Number:
  US 20100141798A1
• Time in Patent Office: 557 Days
• Field of Search: 348/241, 348/246
• US Class Current: 348/241
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: H04N 1/4097 Removing errors due external factors, e.g. dust, scratches
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

A method automatically corrects dust artifact within image acquired by a system including a digital acquisition device including a lens assembly and a translucent lens cap. Multiple original digital images are acquired with the digital acquisition device. Probabilities that certain pixels correspond to dust artifact regions within the images are determined based at least in part on a comparison of suspected dust artifact regions within two or more of the images. Probable dust artifact regions are associated with extracted parameter values relating to the lens assembly when the images were acquired, A statistical dust map is formed including mapped dust regions based on the determining and associating. Pixels corresponding to correlated dust artifact regions are corrected within further digitally-acquired images based on the associated statistical dust map.

Claims

1. A digital image acquisition device, comprising:

an image acquisition sensor,

a shutter,

optical elements for focusing an image on the sensor,

an aperture disposed between said optical elements and said shutter,

a translucent lens cap which defocuses images when in place,

digitally-embedded programming such that the device is arranged to automatically capture a calibration image of a de-focused scene when said shutter and said aperture are open, and to derive a color or luminance map, or a combination thereof, from the calibration image acquired by the sensor when illuminated by ambient light, the digitally-embedded programming capable of classifying color and/or luminance defects, respectively, and correcting said color and/or luminance defects when determining sufficient color and/or luminance defects to render said color and/or luminance acceptable within an acquired image; and

a trigger for automatically acquiring a de-focused calibration image which is arranged to automatically detect whether the translucent lens cap is in place as a pre-requisite for triggering capture of the defocused calibration image.

2. The device of claim 1, wherein the trigger is arranged to trigger said capture based at least in part on time interval, change of lens, or amount of images taken, or a combination thereof

3. The device of claim 1, wherein the trigger includes programming for analyzing whether the image is sufficiently de-focused as a pre-requisite for capture of the de-focused calibration image.

4. The device of claim 1, wherein the trigger is arranged to initiate capture of a plurality of de-focused calibration images of the scene within a certain time frame.

5. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting color or luminance defect conditions, or a combination thereof, within acquired images, the method comprising:

acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

automatically acquiring a defocused calibration image when a trigger automatically detects that the translucent lens cap is in place;

determining probabilities that certain pixels correspond to color or luminance defect regions, or a combination thereof, within said one or more original digital images;

associating the color and/or luminance defect regions with one or more acquisition parameters when the one or more original digital images were acquired;

forming a statistical color and/or luminance defect map including mapped color and/or luminance defect regions based on the calibration image and the color and/or luminance defect region probability determining and associating; and

correcting pixels corresponding to color and/or luminance defect regions within each of said one or more original images based on the associated statistical color and/or luminance defect map; and

wherein said determining probabilities further comprises statistically combining a plurality of individual probabilities based on each said region within two or more images.

6. The device of claim 5, wherein said one or more acquisition parameters comprise aperture size or focal length or both.

7. The device of claim 5, wherein said one or more original images comprise original images of de-focused scenes.

8. The device of claim 5, wherein the method further comprises determining probabilities that certain pixels correspond to regions free of color and/or luminance defects within said one or more images based at least in part on a comparison of suspected defect regions within said one or more images.

9. The device of claim 5, wherein the method further comprises eliminating certain suspected color and/or luminance defect regions as having a probability below a first threshold value.

10. The device of claim 5, wherein the method further comprises digitally-acquiring additional images with said digital acquisition device, repeating said determining and associating, and updating said statistical defect map including updating said mapped defect regions based on the additional color and/or luminance defect determining and associating.

11. The device of claim 5, the method further comprising validating that said additional digitally-acquired images have non-contradicting probability data that certain pixels correspond to defect regions within said additional digitally-acquired images prior to correcting pixels corresponding to correlated defect regions within additional digitally-acquired images based on the associated statistical defect map.

12. The device of claim 5, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation within the statistical defect map having acquisition condition variable-dependencies; and

comparing defect region detection data with the acquisition condition dependent statistical defect map data.

13. The device of claim 5, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation for relating the statistical defect map with the present image according to a selected value of an acquisition condition which is otherwise uncorrelated between the present image and the defect map; and

comparing defect artifact detection data with the now correlated statistical defect map data.

14. The device of claim 5, wherein the method further comprises sorting defect artifact distribution data within said defect map according to meta-data.

15. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting color or luminance defect conditions, or a combination thereof, within acquired images, the method comprising:

acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

automatically acquiring a defocused calibration image when a trigger automatically detects that the translucent lens cap is in place;

determining probabilities that certain pixels correspond to color or luminance defect regions, or a combination thereof, within said one or more original digital images;

associating the color and/or luminance defect regions with one or more acquisition parameters when the one or more original digital images were acquired;

forming a statistical color and/or luminance defect map including mapped color and/or luminance defect regions based on the calibration image and the color and/or luminance defect region probability determining and associating; and

correcting pixels corresponding to color and/or luminance defect regions within each of said one or more original images based on the associated statistical color and/or luminance defect map; and

judging certain probable color and/or luminance defect regions as having a probability above a second threshold value such as to be subject to said correcting; and

wherein said determining probabilities further comprises statistically combining a plurality of individual probabilities based on each said region within two or more said images.

16. The device of claim 15, wherein said one or more acquisition parameters comprise aperture size or focal length or both.

17. The device of claim 15, wherein said one or more original images comprise original images of de-focused scenes.

18. The device of claim 15, wherein the method further comprises determining probabilities that certain pixels correspond to regions free of color and/or luminance defects within said one or more images based at least in part on a comparison of suspected defect regions within said one or more images.

19. The device of claim 15, wherein the method further comprises eliminating certain suspected color and/or luminance defect regions as having a probability below a first threshold value.

20. The device of claim 15, wherein the method further comprises digitally-acquiring additional images with said digital acquisition device, repeating said determining and associating, and updating said statistical defect map including updating said mapped defect regions based on the additional color and/or luminance defect determining and associating.

21. The device of claim 15, the method further comprising validating that said additional digitally-acquired images have non-contradicting probability data that certain pixels correspond to defect regions within said additional digitally-acquired images prior to correcting pixels corresponding to correlated defect regions within additional digitally-acquired images based on the associated statistical defect map.

22. The device of claim 15, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation within the statistical defect map having acquisition condition variable-dependencies; and

comparing defect region detection data with the acquisition condition dependent statistical defect map data.

23. The device of claim 15, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation for relating the statistical defect map with the present image according to a selected value of an acquisition condition which is otherwise uncorrelated between the present image and the defect map; and

comparing defect artifact detection data with the now correlated statistical defect map data.

24. The device of claim 15, wherein the method further comprises sorting defect artifact distribution data within said defect map according to meta-data.

25. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting color or luminance defect conditions, or a combination thereof, within acquired images, the method comprising:

acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

automatically acquiring a defocused calibration image when a trigger automatically detects that the translucent lens cap is in place;

determining probabilities that certain pixels correspond to color or luminance defect regions, or a combination thereof, within said one or more original digital images;

associating the color and/or luminance defect regions with one or more acquisition parameters when the one or more original digital images were acquired;

forming a statistical color and/or luminance defect map including mapped color and/or luminance defect regions based on the calibration image and the color and/or luminance defect region probability determining and associating; and

correcting pixels corresponding to color and/or luminance defect regions within each of said one or more original images based on the associated statistical color and/or luminance defect map; and

digitally-acquiring additional images with said digital acquisition device, repeating said determining and associating, and updating said statistical defect map including updating said mapped defect regions based on the additional defect region determining and associating, and wherein said determining includes dynamically updating said probabilities based on comparisons with suspected equivalent defect regions within the additional digitally-acquired images.

26. The device of claim 25, wherein said one or more acquisition parameters comprise aperture size or focal length or both.

27. The device of claim 25, wherein said one or more original images comprise original images of de-focused scenes.

28. The device of claim 25, wherein the method further comprises determining probabilities that certain pixels correspond to regions free of color and/or luminance defects within said one or more images based at least in part on a comparison of suspected defect regions within said one or more images.

29. The device of claim 25, wherein the method further comprises eliminating certain suspected color and/or luminance defect regions as having a probability below a first threshold value.

30. The device of claim 25, wherein the method further comprises digitally-acquiring additional images with said digital acquisition device, repeating said determining and associating, and updating said statistical defect map including updating said mapped defect regions based on the additional color and/or luminance defect determining and associating.

31. The device of claim 25, the method further comprising validating that said additional digitally-acquired images have non-contradicting probability data that certain pixels correspond to defect regions within said additional digitally-acquired images prior to correcting pixels corresponding to correlated defect regions within additional digitally-acquired images based on the associated statistical defect map.

32. The device of claim 25, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation within the statistical defect map having acquisition condition variable-dependencies; and

comparing defect region detection data with the acquisition condition dependent statistical defect map data.

33. The device of claim 25, the defect region determining including:

loading the statistical defect map;

loading acquisition condition information of a present image;

performing a calculation for relating the statistical defect map with the present image according to a selected value of an acquisition condition which is otherwise uncorrelated between the present image and the defect map; and

comparing defect artifact detection data with the now correlated statistical defect map data.

34. The device of claim 25, wherein the method further comprises sorting defect artifact distribution data within said defect map according to meta-data.