Automated statistical self-calibrating detection and removal of blemishes in digital images dependent upon changes in extracted parameter values

• US 7,545,995 B2
• Filed: 02/25/2008
• Issued: 06/09/2009
• Est. Priority Date: 09/30/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Zamfir, Adrian, Bigioi, Petronel, Corcoran, Peter, Buzuloiu, Vasile, Zamfir, Marta, Ursu, Danutz
• Primary Examiner(s): Kassa; Yosef
• Application Number:
  US12/036,421
• Publication Number:
  US 20080152255A1
• Time in Patent Office: 470 Days
• Field of Search: 382/260, 382/274, 382/275
• US Class Current: 382/275
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: H04N 1/4097 Removing errors due external factors, e.g. dust, scratches
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Abstract

A method of automatically correcting dust artifact within images acquired by a system including a digital camera includes determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image. The dust artifact regions are associated with one or more extracted parameters relating to a variable optical system of the digital camera when the first image was acquired. An associated statistical dust map is formed including mapped dust regions based on known correspondence between effects on dust artifact data of values of extracted parameters. Pixels corresponding to correlated dust artifact regions within a second image acquired with said digital camera are corrected according to one or more different values of the extracted parameters relating to the variable optical system based on the associated statistical dust map.

Claims

1. A digital image acquisition device, comprising a lens assembly, a sensor array, a processor, and a memory having stored therein code for programming the processor to perform a method of automatically correcting dust artifact within images acquired by the device, wherein the method comprises:

(a) determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image;

(b) associating the dust artifact regions with one or more extracted parameters relating to a variable optical system of said digital camera when the first image was acquired;

(c) forming an associated statistical dust map including mapped dust regions based on the dust artifact determining and associating operations and known correspondence between effects on dust artifact data of one or more different values of said one or more extracted parameters; and

(d) correcting pixels corresponding to correlated dust artifact regions within the first digitally-acquired image or one or more different images acquired with said digital camera according to one or more different values of said one or more extracted parameters relating to said variable optical system, or combinations thereof, based on said associated statistical dust map.

2. The device of claim 1, said one or more extracted parameters comprising aperture size, F-number, magnification, lens type or focal length of an optical system of the digital camera, or combinations thereof.

3. The device of claim 1, said one or more extracted parameters comprising aperture size or focal length or both.

4. The device of claim 1, said determining comprising determining probabilities that certain pixels correspond to dust artifact regions within said first acquired image.

5. The device of claim 1, wherein said determining is based at least in part on a comparison of suspected dust artifact regions within multiple images including said first image and at least one other digitally-acquired image.

6. The device of claim 1, wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region;

and wherein said determining is based on an extracted parameter-dependent shadow region analysis, wherein the shadow region analysis presumes that certain regions on the sensor of the device are fully obscured by said dust.

7. The device of claim 1, wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region;

and wherein said determining is based on an extracted parameter-dependent aura region analysis, wherein the aura region analysis presumes that certain regions on the sensor of the device are partially obscured by said dust.

8. A digital image acquisition device, comprising a lens assembly, a sensor array, a processor, and a memory having stored therein code for programming the processor to perform a method of automatically correcting dust artifact regions within images acquired by the device, wherein the method comprises:

(a) determining that certain pixels within one or more digitally-acquired images correspond to one or more dust artifact regions;

(b) associating the dust artifact regions with values of one or more extracted parameters relating to the lens assembly of the device when the images were acquired;

(c) forming a dust map including mapped dust regions based on the dust artifact determining and associating operations;

(d) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the dust map;

(e) repeating the dust artifact determining and associating operations for further acquired images after forming said statistical dust map;

(f) modifying said statistical dust map based on said further images including mathematically correlating determined and associated dust artifact regions of said further acquired images with dust artifact regions in said dust map including combining data associated with dust artifact regions within images acquired with different values of one or more extracted parameters; and

(g) correcting digital data corresponding to the correlated dust artifact regions within acquired images based on the modified statistical dust map.

9. The device of claim 8, wherein the method further comprises filtering from the statistical dust map dust artifact regions having computed dust artifact probabilities below a threshold level.

10. The device of claim 8, wherein the method further comprises allocating greater dust artifact determination weights to more recently acquired images.

11. One or more processor-readable devices having stored therein code for programming a processor to perform a method of automatically correcting dust artifact within images acquired by the device, wherein the method comprises:

(a) determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image;

(b) associating the dust artifact regions with one or more extracted parameters relating to a variable optical system of said digital camera when the first image was acquired;

(c) forming an associated statistical dust map including mapped dust regions based on the dust artifact determining and associating operations and known correspondence between effects on dust artifact data of one or more different values of said one or more extracted parameters; and

(d) correcting pixels corresponding to correlated dust artifact regions within the first digitally-acquired image or one or more different images acquired with said digital camera according to one or more different values of said one or more extracted parameters relating to said variable optical system, or combinations thereof, based on said associated statistical dust map.

12. The one or more processor-readable devices of claim 11, said one or more extracted parameters comprising aperture size, F-number, magnification, lens type or focal length of an optical system of the digital camera, or combinations thereof.

13. The one or more processor-readable devices of claim 11, said one or more extracted parameters comprising aperture size or focal length or both.

14. The one or more processor-readable devices of claim 11, said determining comprising determining probabilities that certain pixels correspond to dust artifact regions within said first acquired image.

15. The one or more processor-readable devices of claim 11, wherein said determining is based at least in part on a comparison of suspected dust artifact regions within multiple images including said first image and at least one other digitally-acquired image.

16. The one or more processor-readable devices of claim 11, wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region;

and wherein said determining is based on an extracted parameter-dependent shadow region analysis, wherein the shadow region analysis presumes that certain regions on the sensor of the device are fully obscured by said dust.

17. The one or more processor-readable devices of claim 11, wherein said dust artifact regions include an aura region partially obscured by dust and a shadow region substantially obscured by dust inside said aura region;

and wherein said determining is based on an extracted parameter-dependent aura region analysis, wherein the aura region analysis presumes that certain regions on the sensor of the device are partially obscured by said dust.

19. The one or more processor-readable devices of claim 14, wherein the method further comprises filtering from the statistical dust map dust artifact regions having computed dust artifact probabilities below a threshold level.

18. One or more processor-readable device having stored therein code for programming a processor to perform a method of automatically correcting dust artifact regions within images acquired by the device, wherein the method comprises:

(a) determining that certain pixels within one or more digitally-acquired images correspond to one or more dust artifact regions;

(b) associating the dust artifact regions with values of one or more extracted parameters relating to the lens assembly of the device when the images were acquired;

(c) forming a dust map including mapped dust regions based on the dust artifact determining and associating operations;

(d) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the dust map;

(e) repeating the dust artifact determining and associating operations for further acquired images after forming said statistical dust map;

(f) modifying said statistical dust map based on said further images including mathematically correlating determined and associated dust artifact regions of said further acquired images with dust artifact regions in said dust map including combining data associated with dust artifact regions within images acquired with different values of one or more extracted parameters; and

(g) correcting digital data corresponding to the correlated dust artifact regions within acquired images based on the modified statistical dust map.

20. The one or more processor-readable devices of claim 18, wherein the method further comprises allocating greater dust artifact determination weights to more recently acquired images.