Automated statistical self-calibrating detection and removal of blemishes in digital images based on multiple occurrences of dust in images

• US 7,536,060 B2
• Filed: 02/15/2008
• Issued: 05/19/2009
• Est. Priority Date: 09/30/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Zamfir, Adrian, Bigioi, Petronel, Corcoran, Peter, Buzuloiu, Vasile, Zamfir, Marta, Ursu, Danutz
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US12/032,595
• Publication Number:
  US 20080144965A1
• Time in Patent Office: 459 Days
• Field of Search: 382/260, 382/274, 382/275, 358/3.26, 358/3.27, 358/463
• US Class Current: 382/274
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response

Abstract

A method automatically corrects dust artifact within images acquired by a system including a digital acquisition device including a lens assembly. Multiple original digital images are acquired with the digital acquisition device. Probabilities that certain pixels correspond to dust artifact regions within the images are determined based at least in part on a comparison of suspected dust artifact regions within two or more of the images. Probable dust artifact regions are associated with extracted parameter values relating to the lens assembly when the images were acquired. A statistical dust map is formed including mapped dust regions based on the determining and associating. Pixels corresponding to correlated dust artifact regions are corrected within further digitally-acquired images based on the associated statistical dust map.

Claims

1. A portable digital image acquisition device including a lens assembly, an image sensor, a processor, and a processor-readable medium having code embedded therein for programming the processor to perform a method of automatically correcting dust artifact within images acquired by the device, wherein the method comprises:

(a) acquiring one or more original digital images with said digital acquisition device;

(b) determining probabilities that certain pixels correspond to dust artifact regions within said one or more images;

(c) associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the one or more images were acquired;

(d) eliminating certain suspected dust artifact regions as having a probability below a first threshold value;

(e) forming a statistical dust map including mapped dust regions based on the determining, associating and eliminating; and

(f) correcting pixels corresponding to correlated dust artifact regions within same or different images, or both, based on the associated statistical dust map.

2. The portable device of claim 1, wherein said determining probabilities further comprises statistically combining a plurality of individual probabilities based on corresponding regions within at least two of said one or more images.

3. The portable device of claim 1, wherein the method further comprises determining probabilities that certain pixels correspond to regions free of dust within said images based at least in part on a comparison of suspected dust artifact regions within one or more of said images.

4. The portable device of claim 1, wherein the method further comprises judging certain further dust artifact regions as having a probability above said threshold value, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said eliminating operation.

5. The portable device of claim 1, wherein the method further comprises judging certain probable dust artifact regions as having a probability above a second threshold value such as to be subject to said correcting operation.

6. The portable device of claim 5, wherein said first and second threshold values are different.

7. The portable device of claim 6, wherein the method further comprises judging certain further dust artifact regions as having a probability between said first and said second threshold values, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said correcting operation.

8. The portable device of claim 1, wherein said determining includes dynamically updating said probabilities based on comparisons with suspected equivalent dust artifact regions within said further digitally-acquired images.

9. The portable device of claim 1, wherein said determining of said probabilities is further based on a pixel analysis of the suspected dust artifact regions in view of predetermined characteristics indicative of the presence of a dust artifact region.

10. A portable digital image acquisition device including a lens assembly, an image sensor, a processor, and a processor-readable medium having code embedded therein for programming the processor to perform a method of automatically correcting dust artifact within images acquired by the device, wherein the method comprises:

(a) acquiring one or more original digital images with said digital acquisition device;

(b) determining probabilities that certain pixels correspond to dust artifact regions within said one or more images;

(c) associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the one or more images were acquired;

(d) forming a statistical dust map including mapped dust regions based on the determining and associating;

(e) validating that one or more further digitally-acquired images have non-contradicting probability data that certain pixels correspond to dust artifact regions within prior to correcting pixels corresponding to correlated dust artifact regions based on the associated statistical dust map; and

(f) correcting pixels corresponding to correlated dust artifact regions within same or different images, or both, based on the associated statistical dust map.

11. The portable device of claim 10, wherein the method further comprises instructing a user to create a new said dust map if said validating determines a noticeable change in the dust map.

12. The portable device of claim 10, wherein the method further comprises determining and associating comprises calculating effects of differences in values of the one or more extracted parameters in different images on dust artifact illumination, shape, position, reflection or transmission properties, distance of dust to the sensor, aperture, exit pupil, or focal length, or combinations thereof.

13. The portable device of claim 10, wherein the method further comprises judging certain probable dust artifact regions as having a probability above a threshold value such as to be subject to said correcting.

14. The portable device of claim 13, wherein the method further comprises judging certain further dust artifact regions as having a probability below said threshold value, such as to be subject to further probability determining including comparison with further acquired images prior to judging whether each said further dust artifact region will be subject to said correcting operation.

15. A portable digital image acquisition device (hereinafter “

portable device”

) including a lens assembly, an image sensor, a processor, and a processor-readable medium having code embedded therein for programming the processor to perform a method of automatically correcting dust artifact within images acquired by the device, wherein the method comprises;

(a) acquiring one or more original digital images with said digital acquisition device;

(b) determining probabilities that certain pixels correspond to dust artifact regions within said one or more images;

(c) associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the one or more images were acquired;

(d) forming a statistical dust map including mapped dust regions based on the determining and associating;

(e) sorting dust artifact distribution data within said dust map according to meta-data; and

(f) correcting pixels corresponding to correlated dust artifact regions within same or different images, or both, based on the associated statistical dust map.

16. The portable device of claim 15, wherein said one or more extracted parameters comprise aperture size, F-number, magnification, lens type, focal length of an optical system of the digital camera, or combinations thereof.

17. The portable device of claim 16, wherein said one or more extracted parameters are calculated empirically from a comparison of one or more said dust artifact regions within said one or more original digital images with said digital acquisition device.

18. The portable device of claim 15, wherein said one or more extracted parameters comprise aperture size or focal length or both.

19. The portable device of claim 15, wherein at least two different images of the one or more images are acquired with different values of said one or more extracted parameters.

20. The portable device of claim 15, wherein the dust artifact determining comprises:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and

(IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data.

21. The portable device of claim 15, wherein the extracted parameter information comprises values of aperture size and focal length.

22. The portable device of claim 15, wherein the extracted parameter information comprises lens type information.

23. The portable device of claim 15, wherein said suspected dust artifact regions of said at least two images comprise inner regions and aura regions, and wherein said comparison comprises a first comparison of said inner regions and a second comparison of said aura regions.