Digital image acquisition and processing system

• US 20050068447A1
• Filed: 09/30/2003
• Published: 03/31/2005
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia, Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia, Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Zamfir, Adrian, Bigioi, Petronel, Corcoran, Peter, Buzuloiu, Vasile, Zamfir, Marta, Ursu, Danutz
• Granted Patent: US 7,206,461 B2
• Time in Patent Office: Days
• US Class Current: 348/335
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/155 Removing patterns interfering with the pattern to be recognised, such as ruled lines or underlines
• CPC Class Codes: H04N 1/4097 Removing errors due external factors, e.g. dust, scratches
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/67 applied to fixed-pattern noise, e.g. non-uniformity of response

Abstract

A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions is provided. The system includes a digital camera and an external processing device. The camera includes an optical system for acquiring an image including a lens assembly and an aperture stop, and an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image. The external processing device couples with the digital camera to receive the image data or converted image data internally processed within the digital camera. The external device includes digital processing electronics including a processor for processing the image data according to programming instructions. The system includes one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions.

Claims

1. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) acquiring multiple original digital images with said digital acquisition device;

(ii) determining probabilities that certain pixels correspond to dust artifact regions within said images based at least in part on a comparison of suspected dust artifact regions within two or more of said images;

(iii) associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the images were acquired;

(iv) forming a statistical dust map including mapped dust regions based on the dust artifact determining and associating;

(v) correcting pixels corresponding to correlated dust artifact regions within further digitally-acquired images based on the associated statistical dust map.

2. The system of claim 1, wherein said determining probabilities further comprises statistically combining an plurality of individual probabilities based on each said regions within two or more said images.

3. The system of claim 1, the method further comprising eliminating certain suspected dust artifact regions as having a probability below a first threshold value.

4. The system of claim 1, the method further comprising judging certain probable dust artifact regions as having a probability above a threshold value such as to be subject to said correcting operation.

5. The system of claim 1, wherein said probability determining includes weighting suspected dust artifact regions according to one or more predetermined probability weighting assessment conditions.

6. The system of claim 1, wherein said further digitally-acquired images comprising different images than said originally acquired images.

7. The system of claim 1, the method further comprising correcting pixels corresponding to correlated dust artifact regions within each of said original images based on the associated statistical dust map.

8. The system of claim 1, the method further comprising correcting pixels corresponding to correlated dust artifact regions within said original images based on the associated statistical dust map.

9. The system of claim 1, the method further comprising digitally-acquiring additional images with said digital camera, repeating said determining and associating, and updating said statistical dust map including updating said mapped dust regions based on the additional dust artifact determining and associating.

10. The system of claim 1, wherein said determining includes dynamically updating said probabilities based on comparisons with suspected equivalent dust artifact regions within said further digitally-acquired images.

11. The system of claim 1, wherein said determining of said probabilities is further based on a pixel analysis of the suspected dust artifact regions in view of predetermined characteristics indicative of the presence of a dust artifact region.

12. The system of claim 1, the method further comprising validating whether said further digitally-acquired image has non contradicting data that said probability that certain pixels correspond to dust artifact regions within said further digitally-acquired image prior to correcting pixels corresponding to correlated dust artifact regions within further digitally-acquired images based on the associated statistical dust map.

13. The system of claim 1, said image correction method being performed on a processed image after being converted from raw format to a known red, green, blue representation.

14. The system of claim 1, wherein said correcting includes replacing said pixels within said one or more digitally-acquired images with new pixels.

15. The system of claim 1, wherein said correcting includes enhancing said values of pixels within said one or more digitally-acquired images.

16. The system of claim 1, the dust artifact determining including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and

(IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data.

17. The system of claim 1, the dust artifact determining including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing a calculation for relating the statistical dust map with the present image according to a selected value of an extracted parameter which is otherwise uncorrelated between the present image and the dust map; and

(IV) comparing dust artifact detection data with the now correlated statistical dust map data.

18. The system of claim 1, wherein said suspected dust artifact regions of at least two images comprise shadow regions and partially-occluded aura regions, and the method further comprises a first comparison of said shadow regions and a second comparison of said aura regions.

19. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) digitally-acquiring one or more original images with said digital camera;

(ii) determining probabilities that certain pixels correspond to dust artifact regions within said one or more digitally-acquired images;

(iii) associating the dust artifact regions with one or more extracted parameters relating to the optical system when the one or more images were acquired;

(iv) forming a statistical dust map including mapped dust regions based on the dust artifact probability determining and associating;

(v) correcting pixels corresponding to dust artifact regions within each of said one or more original images based on the associated statistical dust map.

20. The system of claim 19, the method further comprising digitally-acquiring further images with said digital camera, repeating said determining and associating, and updating said statistical dust map including updating said mapped dust regions based on the further dust artifact determining and associating.

21. The system of claim 19, said determining comprising determining probabilities that certain pixels correspond to dust artifact regions within said acquired images based at least in part on a comparison of suspected dust artifact regions within two or more digitally-acquired images, or on a pixel analysis of the suspected dust artifact regions in view of predetermined characteristics indicative of the presence of a dust artifact region, or both.

22. The system of claim 19, wherein said determining is based at least in part on a comparison of suspected dust artifact regions within two or more digitally-acquired images.

23. The system of claim 19, wherein said determining of said probabilities is further based on a pixel analysis of the suspected dust artifact regions in view of predetermined characteristics indicative of the presence of a dust artifact region.

24. The system of claim 19, wherein said suspected dust artifact regions of said at least two images comprise shadow regions and aura regions, and wherein said correcting comprises a first correcting of said aura regions and a second correcting of said shadow regions.

25. The system of claim 19, the dust map including dust artifact regions with probabilities above a threshold probability and not including regions with lower probabilities.

26. The system of claim 19, the dust map including dust artifact regions with probabilities above a first threshold value, not including regions with probabilities below a second threshold value, and where further regions having a probability between said first and second threshold values exist within said dust map, then further image information is combined into said probability analysis before determining whether said region are included as dust artifact regions.

27. The system of claim 19, wherein said correcting includes replacing said pixels within said one or more digitally-acquired images with new pixels.

28. The system of claim 19, wherein said correcting includes enhancing said values of pixels within said one or more digitally-acquired images.

29. The system of claim 19, the dust artifact determining operation including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and

(IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data.

30. The system of claim 19, the dust artifact determining operation including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing a calculation for relating the statistical dust map with the present image according to a selected value of an extracted parameter which is otherwise uncorrelated between the present image and the dust map; and

(IV) comparing dust artifact detection data with the now correlated statistical dust map data.

31. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) determining that certain pixels correspond to dust artifact regions within a first digitally-acquired image;

(ii) associating the dust artifact regions with one or more extracted parameters relating to a variable optical system of said digital camera when the first image was acquired;

(iii) forming an associated statistical dust map including mapped dust regions based on the dust artifact determining and associating operations and known correspondence between effects on dust artifact data of one or more different values of said one or more extracted parameters; and

(iv) correcting pixels corresponding to correlated dust artifact regions within a second image acquired with said digital camera according to one or more different values of said one or more extracted parameters relating to said variable optical system based on said associated statistical dust map.

32. The system of claim 31, wherein said first and second images were acquired of different objects.

33. The system of claim 31, further comprising correcting pixels corresponding to correlated dust artifact regions within said first image based on the associated statistical dust map.

34. The system of claim 31, the method further comprising digitally-acquiring additional images with said digital camera, repeating said determining and associating, and updating said statistical dust map including updating said mapped dust regions based on the additional dust artifact determining and associating.

35. The system of claim 31, said determining comprising determining probabilities that certain pixels correspond to dust artifact regions within said first acquired image.

36. The system of claim 31, wherein said determining is based at least in part on a comparison of suspected dust artifact regions within multiple images including said first image and at least one other digitally-acquired image.

40. The system of claim 36, said forming said statistical dust map being based upon multiple acquired images and including mathematically correlating dust artifact regions of the multiple acquired images according to their extracted parameter values including said one or more different extracted parameter values of said at least two acquired images.

37. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) determining probabilities that certain pixels within multiple digitally-acquired images correspond to one or more dust artifact regions;

(ii) associating probable dust artifact regions with values of one or more extracted parameters relating to a lens system of the digital image acquisition device when the images were acquired;

(iii) forming a statistical dust map including mapped dust regions based on the dust artifact determining and associating operations; and

(iv) correcting digital data corresponding to pixels within one or more digitally-acquired images correlated with mapped dust regions of the statistical dust map.

38. The system of claim 37, the method further comprising repeating the dust artifact determining and associating operations for further acquired images, values of the one or more extracted parameters relating to the optical system differing between at least two acquired images.

39. The system of claim 38, said at least two acquired images having been acquired of different objects.

41. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) determining probabilities of dust artifact regions corresponding to pixels within a digitally-acquired image based at least in part on a pixel analysis of the region in view of predetermined characteristics indicative of the presence of a dust artifact region;

(ii) associating the dust artifact regions with one or more extracted parameters relating to the optical system when the image was acquired;

(iii) forming a statistical dust map including mapped-dust-regions based on the dust artifact determining and associating operations; and

(iv) correcting the digital data corresponding to the correlated dust artifact regions within the acquired image based on the associated statistical dust map.

42. The system of claim 41, the method further comprising eliminating certain suspected dust artifact regions as having a probability below a first threshold value.

43. The system of claim 41, the method further comprising judging certain probable dust artifact regions as having a probability above a threshold value such as to be subject to said correcting operation.

44. The system of claim 41, wherein said probability determining includes weighting suspected dust artifact regions according to one or more predetermined probability weighting assessment conditions.

45. The system of claim 41, wherein said further digitally-acquired images comprising different images than said originally acquired images.

46. The system of claim 41, the method further comprising correcting pixels corresponding to correlated dust artifact regions within each of said original images based on the associated statistical dust map.

47. The system of claim 41, the method further comprising correcting pixels corresponding to correlated dust artifact regions within said original images based on the associated statistical dust map.

48. The system of claim 41, wherein said determining includes dynamically updating said probabilities based on comparisons with suspected equivalent dust artifact regions within said further digitally-acquired images.

49. The system of claim 41, wherein said forming includes dynamically updating said statistical dust map based on further pixel analyses of suspected dust artifact regions in further images in view of predetermined characteristics indicative of the presence of a dust artifact region.

50. The system of claim 41, wherein said suspected dust artifact regions of said at least two images comprise shadow regions and aura regions, and wherein said comparison comprises a first comparison of said shadow regions and a second comparison of said aura regions.

51. The system of claim 41, wherein said determining probabilities further comprises statistically combining an plurality of individual probabilities based on each said regions within two or more said images.

52. The system of claim 41, the method further comprising validating whether said further digitally-acquired image has non contradicting data that said probability that certain pixels correspond to dust artifact regions within said further digitally-acquired image prior to correcting pixels corresponding to correlated dust artifact regions within further digitally-acquired images based on the associated statistical dust map.

53. The system of claim 41, wherein said correcting includes replacing said pixels within said one or more digitally-acquired images with new pixels.

54. The system of claim 41, wherein said correcting includes enhancing said values of pixels within said one or more digitally-acquired images.

55. The system of claim 41, the dust artifact determining operation including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing calculations within the statistical dust map having extracted parameter variable-dependencies; and

(IV) comparing dust artifact detection data with the extracted parameter dependent statistical dust map data.

56. The system of claim 41, the dust artifact determining operation including:

(I) loading the statistical dust map;

(II) loading extracted parameter information of a present image;

(III) performing a calculation for relating the statistical dust map with the present image according to a selected value of an extracted parameter which is otherwise uncorrelated between the present image and the dust map; and

(IV) comparing dust artifact detection data with the now correlated statistical dust map data.

57. The system of claim 41, said dust artifact regions including an aura region partially obscured by dust and a shadow region at least substantially obscured by dust inside said aura region.

58. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) analyzing pixels within one ore more acquired digital images according to probability determinations that such pixels correspond to blemish artifacts;

(ii) determining whether a threshold distribution of blemish artifacts is present within one or more of said digital images; and

(iii) indicating a need for service when at least said threshold distribution is determined to be present.

59. The system of claim 58, the method further comprising:

(a) determining probabilities of dust artifact regions corresponding to said pixels within said digitally-acquired image;

(b) associating the dust artifact regions with one or more extracted parameters relating to the lens assembly when the image was acquired;

(c) forming a statistical record including dust artifact regions based on the dust artifact determining and associating; and

(d) determining said threshold distribution based on predetermined characteristics of said statistical record.

60. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) acquiring multiple original digital images with said digital acquisition device;

(ii) determining probabilities that certain pixels correspond to dust artifact regions within said images based at least in part on a comparison of suspected dust artifact regions within two or more of said images;

(iii) forming a statistical dust record including probabilities of dust artifact regions based on the dust artifact determining and associating; and

(iv) determining a need for service when the statistical dust record indicates that a predetermined threshold dust artifact distribution is present within digital images acquired with said digital acquisition device.

61. The system of claim 60, wherein said probability determining includes weighting suspected dust artifact regions according to one or more predetermined probability weighting assessment conditions.

62. The system of claim 60, the method further comprising associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the images were acquired, said statistical dust record being formed from multiple images including at least two images having different values of one or more extracted parameters that are mathematically correlated based on known effects of said different values on dust artifact regions appearing within said digital images.

63. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) determining probabilities that certain pixels correspond to dust artifact regions within a digitally-acquired image based at least in part on a pixel analysis of the region in view of predetermined characteristics indicative of the presence of a dust artifact region;

(ii) forming a statistical dust record including probabilities of dust artifact regions based on the dust artifact determining and associating operations; and

(iii) determining a need for service when the statistical dust record indicates that a predetermined threshold dust artifact distribution is present within digital images acquired with said digital acquisition device.

64. The system of claim 63, wherein said probability determining includes weighting suspected dust artifact regions according to one or more predetermined probability weighting assessment conditions.

65. The system of claim 63, the method further comprising associating probable dust artifact regions with one or more values of one or more extracted parameters relating to the lens assembly of the digital acquisition device when the images were acquired, said statistical dust record being formed from multiple images including at least two images having different values of one or more extracted parameters that are mathematically correlated based on known effects of said different values on dust artifact regions appearing within said digital images.

66. A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions, comprising:

(A) a digital camera, comprising;

(i) an optical system for acquiring an image including a lens assembly and an aperture stop; and

(ii) an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image; and

(B) an external processing device for coupling with the digital camera to receive the image data or converted image data internally processed within the digital camera, the external device comprising digital processing electronics including a processor for processing the image data according to programming instructions; and

(C) wherein the system further comprises one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions, the image correction method comprising;

(i) determining probabilities that certain pixels within multiple digitally-acquired images correspond to one or more dust artifact regions;

(ii) forming a statistical dust map including mapped dust artifact regions based on the dust artifact determining;

(iii) separating said mapped dust artifact regions into aura regions partially obscured by dust and shadow regions substantially obscured by dust inside said aura regions; and

(iv) correcting digital data corresponding to pixels within one or more digitally-acquired images including dust artifact regions separated into aura regions and shadow regions correlated with mapped aura and shadow regions of the statistical dust map.

67. The system of claim 66, said dust artifact region probability determining including separating suspected dust artifact regions into aura regions and shadow regions, and determining separate probabilities whether said aura regions and said shadow regions of said suspected dust artifact regions in fact correspond to dust artifact regions within said images.

68. The system of claim 66, the correcting operation including calculating said aura region and said shadow region.

69. The system of claim 66, wherein said dust artifact region probability determining is based at least in part on a comparison of suspected dust artifact regions within two or more images.

70. The system of claim 66, wherein said dust artifact region probability determining is based at least in part on a comparison of suspected dust artifact regions within images with predetermined characteristics indicative of the presence of a dust artifact region.