Detection and removal of blemishes in digital images utilizing original images of defocused scenes

• US 7,295,233 B2
• Filed: 02/10/2007
• Issued: 11/13/2007
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Zamfir, Adrian, Bigioi, Petronel
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): DANIELS, ANTHONY J
• Application Number:
  US11/673,577
• Publication Number:
  US 20070122056A1
• Time in Patent Office: 276 Days
• Field of Search: None
• US Class Current: 348/241
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: H04N 1/4097 Removing errors due external factors, e.g. dust, scratches
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

A method automatically corrects dust artifact within images acquired by a system including a digital acquisition device including a lens assembly and a translucent lens cap. Multiple original digital images are acquired with the digital acquisition device. Probabilities that certain pixels correspond to dust artifact regions within the images are determined based at least in part on a comparison of suspected dust artifact regions within two or more of the images. Probable dust artifact regions are associated with extracted parameter values relating to the lens assembly when the images were acquired. A statistical dust map is formed including mapped dust regions based on the determining and associating. Pixels corresponding to correlated dust artifact regions are corrected within further digitally-acquired images based on the associated statistical dust map.

Claims

1. A digital image acquisition device, comprising

(a) an image acquisition sensor,

(b) a shutter,

(c) optical elements for focusing an image on the sensor,

(d) an aperture disposed between said optical elements and said shutter,

(e) a translucent lens cap which defocuses images when in place,

(f) digitally-embedded programming such that the device is arranged to automatically capture a calibration image of a de-focused scene when said shutter and said aperture are open, and to derive a map of defects from the calibration image acquired by the sensor when illuminated by ambient light, and

(g) a trigger for automatically acquiring a de-focused calibration image which is arranged to automatically acquire such calibration image for the map of defects based on change of lens, amount of images taken, replacement of lens cap, or combinations of these parameters.

2. A digital image acquisition device, comprising

(a) an image acquisition sensor,

(b) a shutter,

(c) optical elements for focusing an image on the sensor,

(d) an aperture disposed between said optical elements and said shutter,

(e) a lens which defocuses images when in a dust-detection mode,

(f) digitally-embedded programming such that the device is arranged to automatically capture a calibration image of a de-focused scene when said shutter and said aperture are open, and to derive a map of defects from the calibration image acquired by the sensor when illuminated by ambient light, and

(g) a trigger for automatically acquiring a de-focused calibration image which is arranged to automatically acquire such calibration image for the map of defects based on change of lens, amount of images taken, replacement of lens cap, or combinations of these parameters.

3. A digital image acquisition device, comprising

(a) an image acquisition sensor,

(b) a shutter,

(c) optical elements for focusing an image on the sensor,

(d) an aperture disposed between said optical elements and said shutter,

(e) a translucent lens cap which defocuses images when in place,

(f) digitally-embedded programming such that the device is arranged to automatically capture a calibration image of a de-focused scene when said shutter and said aperture are open, and to derive a map of defects from the calibration image acquired by the sensor when illuminated by ambient light;

and

(g) a trigger for automatically acquiring a de-focused calibration image which is arranged to automatically detect whether the translucent lens cap is in place as a pre-requisite for triggering capture of the defocused calibration image.

4. The device of claim 3, wherein the trigger is arranged to trigger said capture based at least in part on time interval, change of lens, or amount of images taken, or a combination thereof.

11. The device of claim 4, wherein the method further comprises eliminating certain suspected dust or other blemish artifact regions as having a probability below a first threshold value.

5. A digital image acquisition device, comprising

(a) an image acquisition sensor,

(b) a shutter,

(c) optical elements for focusing an image on the sensor,

(d) an aperture disposed between said optical elements and said shutter,

(e) a lens which defocuses images when in a dust-detection mode, and

(f) digitally-embedded programming such that the device is arranged to automatically capture a calibration image of a de-focused scene when said shutter and said aperture are open, and to derive a map of defects from the calibration image acquired by the sensor when illuminated by ambient light,

wherein the digitally-embedded programming includes a trigger for analyzing whether the image is sufficiently de-focused as a pre-requisite for capture of the de-focused calibration image.

6. The device of claim 5, wherein the trigger is arranged to initiate capture of a plurality of de-focused calibration images of the scene within a certain time frame.

7. The device of claim 5, further comprising validating that additional digitally-acquired images have non-contradicting probability data that certain pixels correspond to dust or other blemish artifact regions within said additional digitally-acquired images prior to correcting pixels corresponding to correlated dust artifact regions within the additional digitally-acquired images based on the associated statistical dust map.

8. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting dust artifact regions within acquired images, the method comprising:

(a) acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

(b) determining probabilities that certain pixels correspond to dust or other blemish artifact regions within said one or more original digital images;

(c) associating the dust or other blemish artifact regions with one or more extracted parameters relating to the optical system when the one or more original digital images were acquired;

(d) forming a statistical dust map including mapped dust or other blemish regions based on the dust or other blemish artifact probability determining and associating;

(e) correcting pixels corresponding to dust artifact regions within each of said one or more original images based on the associated statistical dust map; and

(f) triggering an automatic acquisition of a de-focused calibration image including analyzing whether the image is sufficiently de-focused as a pre-requisite for triggering capture of the defocused calibration image.

9. The device of claim 8, wherein said one or more original images comprise original images of de-focused scenes.

12. The device of claim 8, wherein said one or more extracted parameters comprise aperture size or focal length or both.

13. The device of claim 8, wherein the method further comprises digitally-acquiring additional images with said digital acquisition device, repeating said determining and associating, and updating said statistical dust map including updating said mapped dust or other blemish regions based on the additional dust or other blemish artifact determining and associating.

14. The device of claim 8, the dust or other blemish artifact determining including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and

(IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data.

15. The device of claim 8, the dust or other blemish artifact determining including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing a calculation for relating the statistical dust map with the present image according to a selected value of an extracted parameter which is otherwise uncorrelated between the present image and the dust map; and

(IV) comparing dust artifact detection data with the now correlated statistical dust map data.

16. The device of claim 8, wherein the method further comprises sorting dust or other blemish artifact distribution data within said dust map according to meta-data.

17. The device of claim 8, wherein said digital acquisition device captures said images from film images.

18. The device of claim 8, wherein the method further comprises determining probabilities that certain pixels correspond to regions free of dust or other blemishes within said images based at least in part on a comparison of suspected dust artifact regions within one or more of said images.

10. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting dust artifact regions within acquired images, the method comprising:

(a) acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

(b) determining probabilities that certain pixels correspond to dust or other blemish artifact regions within said one or more original digital images;

(c) associating the dust or other blemish artifact regions with one or more extracted parameters relating to the optical system when the one or more original digital images were acquired;

(d) forming a statistical dust map including mapped dust or other blemish regions based on the dust or other blemish artifact probability determining and associating; and

(e) correcting pixels corresponding to dust artifact regions within each of said one or more original images based on the associated statistical dust map,

wherein said determining probabilities further comprises statistically combining a plurality of individual probabilities based on each said region within two or more said images.

19. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting dust artifact regions within acquired images, the method comprising:

(a) acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

(b) determining probabilities that certain pixels correspond to dust or other blemish artifact regions within said one or more original digital images;

(c) associating the dust or other blemish artifact regions with one or more extracted parameters relating to the optical system when the one or more original digital images were acquired;

(d) forming a statistical dust map including mapped dust or other blemish regions based on the dust or other blemish artifact probability determining and associating; and

(e) correcting pixels corresponding to dust artifact regions within each of said one or more original images based on the associated statistical dust map,

wherein the method further comprises judging certain probable dust or other blemish artifact regions as having a probability above a second threshold value such as to be subject to said correcting.

20. A digital image acquisition device having no photographic film, comprising an optical system, a processor and one or more storage devices for programming the processor to control the camera to perform a method of automatically correcting dust artifact regions within acquired images, the method comprising:

(a) acquiring one or more original digital images through a translucent lens cap disposed along an optical path between an object and one or more components of the optical system including a digital sensor;

(b) determining probabilities that certain pixels correspond to dust or other blemish artifact regions within said one or more original digital images;

(c) associating the dust or other blemish artifact regions with one or more extracted parameters relating to the optical system when the one or more original digital images were acquired;

(d) forming a statistical dust map including mapped dust or other blemish regions based on the dust or other blemish artifact probability determining and associating;

(e) correcting pixels corresponding to dust artifact regions within each of said one or more original images based on the associated statistical dust map,

(f) digitally-acquiring additional images with said digital acquisition device,

(g) repeating said determining and associating, and

(h) updating said statistical dust map including updating said mapped dust or other blemish regions based on the additional dust or other blemish artifact determining and associating,

wherein said determining includes dynamically updating said probabilities based on comparisons with suspected equivalent dust artifact regions within the additional digitally-acquired images.