Digital image acquisition device with built in dust and sensor mapping capability

• US 7,702,236 B2
• Filed: 02/13/2007
• Issued: 04/20/2010
• Est. Priority Date: 02/14/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Petrescu, Stefan, Zamfir, Adrian
• Primary Examiner(s): Perkey; W. B.
• Assistant Examiner(s): Phan; Minh Q
• Application Number:
  US11/674,650
• Publication Number:
  US 20070189757A1
• Time in Patent Office: 1,162 Days
• Field of Search: 396/121, 396125-128, 348/180, 348/187, 348/189, 348/335
• US Class Current: 396/213
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/0008 checking presence/absence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

A digital image acquisition device has an image acquisition sensor, a shutter, an aperture and optical elements for focusing an image on the sensor. The device includes a light source located in the body of the device. The light source is periodically activated with one of the aperture or shutter closed, and the device derives a map of defects on the surface of the sensor from a calibration image acquired by the sensor when illuminated by the light source.

Claims

1. A digital image acquisition device having an image acquisition sensor, a shutter, one or more optical elements for focusing an image on the sensor, and an aperture disposed between at least one of said one or more optical elements and said shutter, the device including a light source located in a body of the device, and being arranged to periodically activate the light source only when said aperture is closed such that said sensor is substantially unexposed to ambient light, and to derive a map of defects in the sensor or on the surface of the sensor, or both, from a calibration image acquired by the sensor when illuminated by the light source and when said sensor is substantially unexposed to ambient light due to said aperture being closed, and the device is further arranged to correct an acquired image based on the map of defects, and to construct a second defect map based on a second calibration image, and to correct the corrected image in accordance with said second defect map.

2. A digital image acquisition device according to claim 1, wherein the device is arranged to activate the light source in one or a combination of the following events:

just before taking an image of a scene;

or

during inactivity of the device in accordance with a predefined schedule;

or

just after each turn-ON of the device;

or

when a lens is replaced;

or

after each Nth image of a scene is captured.

3. A digital acquisition device according to claim 2, where N is between 3 and 10.

4. A digital image acquisition device according to claim 1, wherein said sensor comprises one of a CCD or a CMOS array.

5. A digital image acquisition device according to claim 1, wherein said light source comprises one or more LEDs.

7. A digital image acquisition device according to claim 1, wherein said light source is disposed between the shutter and the aperture, and wherein the device is arranged to activate the light source when the aperture is closed and when the shutter is open.

8. A digital image acquisition device according to claim 1, wherein said device is arranged to activate said light source when the optical elements are covered by the lens cap.

9. A digital image acquisition device according to claim 1, wherein the device is arranged to compare two images captured by the sensor when illuminated by the light source, and responsive to said images differing by greater than a threshold amount, to derive said map of defects.

10. A digital image acquisition device according to claim 1, wherein the device is arranged to analyze an image captured by the sensor when illuminated by the light source, and responsive to said image including greater than a threshold amount of defects, to derive said map of defects.

11. A digital image acquisition device according to claim 1, wherein said device is arranged to map said calibration image to said map of defects in association with a device specific look-up table.

12. A digital image acquisition device according to claim 1, wherein said calibration image comprises a low resolution image or a full resolution image.

13. A digital image acquisition device according to claim 1, wherein said calibration image comprises a full color image or a grayscale image.

14. A digital image acquisition device according to claim 1, further comprising a body that contains said light source and a removable lens that comprises at least one of said one or more optical elements and said aperture.

15. A digital image acquisition device according to claim 1, wherein the device is arranged to correct an image of a scene in accordance with said map of defects.

16. A digital image acquisition device according to claim 15, wherein the device is arranged to inform the user of a need to physically de-dust said device due to excess dust beyond reasonable digital correction.

18. A digital image acquisition device according to claim 1, wherein the device is arranged to store said map of defects with an image of a scene.

19. A digital image acquisition device according to claim 1, wherein said map of defects comprises a binary, a gray level or a probabilistic image, or combinations thereof.

6. A digital image acquisition device having an image acquisition sensor, a shutter, one or more optical elements for focusing an image on the sensor, and an aperture disposed between at least one of said one or more optical elements and said shutter, the device including a light source located in a body of the device, and being arranged to periodically activate the light source when one of a lens cap, shutter or said aperture is closed, or a combination thereof, and to derive a map of defects in the sensor or on the surface of the sensor, or both, from a calibration image acquired by the sensor when illuminated by the light source, wherein said light source is disposed between the sensor and both the aperture and the shutter, and wherein the device is arranged to activate the light source when the aperture or shutter, or a combination thereof, is closed, and the device is further arranged to correct an acquired image based on the map of defects, and to construct a second defect map based on a second calibration image, and to correct the corrected image in accordance with said second defect map.

17. A digital image acquisition device having an image acquisition sensor, a shutter, one or more optical elements for focusing an image on the sensor, and an aperture disposed between at least one of said one or more optical elements and said shutter, the device including a light source located in a body of the device, and being arranged to periodically, at intervals of at least six minutes, activate the light source when one of said shutter or said aperture is closed, or a combination thereof, and to derive a map of defects in the sensor or on the surface of the sensor, or both, from a calibration image acquired by the sensor when illuminated by the light source, wherein the device is arranged to correct an image of a scene in accordance with said map of defects, and the device is further arranged to construct a second map of defects of said optical elements and being arranged to correct said corrected image in accordance with said second map.

50. The device of claim 17, wherein said intervals comprise at least one day.

51. The device of claim 17, wherein said intervals comprise at least four days.

52. The device of claim 17, wherein said intervals comprise at least one month.

53. The device of claim 17, wherein said intervals comprise at least one hour.

20. A digital image acquisition device, comprising:

a camera body configured for coupling with a lens assembly including one or more lenses and an aperture;

an electronic image sensor within the camera body;

a shutter for controlling exposure of the sensor by ambient light;

a light source, disposed between the sensor and both said shutter and said aperture, for illuminating the sensor;

a processor; and

one or more digital storage devices having instructions embedded therein for programming the processor to;

control the light source to illuminate the sensor on multiple occasions only when said shutter or said aperture, or a combination thereof, is closed such that said sensor is substantially unexposed to ambient light;

create a sensor defect map based on a calibration image acquired by the sensor when illuminated by the light source and when said sensor is substantially unexposed to ambient light due to said shutter or aperture, or combination thereof, being closed;

correct an acquired image based on the sensor defect map;

construct a second defect map based on a second calibration image; and

correct the corrected image in accordance with said second defect map.

21. The device of claim 20, further comprising said lens assembly coupled with said camera body.

22. The device of claim 20, wherein the processor is programmed to initiate illumination of the sensor and creation of the sensor defect map when the aperture is closed.

23. The device of claim 22, wherein the shutter is disposed between the light source and the sensor, and the shutter is open when the sensor is illuminated for creating the sensor defect map.

24. The device of claim 21, further comprising a lens cap for coupling with the lens assembly thereby preventing exposure of the sensor by ambient light traversing the lens assembly.

25. The device of claim 24, wherein the processor is programmed to initiate illumination of the sensor for creating the sensor defect map when the lens cap is coupled to the lens assembly.

32. The device of claim 20, wherein the processor is programmed to automatically update an existing sensor defect map upon creation of a new sensor defect map.

33. The device of claim 32, wherein the both the existing and new sensor defect maps are stored for application to images acquired at different times.

34. The device of claim 20, wherein the light source is built into the camera body.

35. The device of claim 20, further comprising a look up table stored therein for to provide a mapping from an acquired calibration image to a real sensor defect position map.

26. A digital image acquisition device, comprising:

a camera body configured for coupling with a lens assembly including one or more lenses and an aperture;

an electronic image sensor within the camera body;

a shutter for controlling exposure of the sensor by ambient light;

a light source, disposed between the sensor and the one or more lenses of the lens assembly, as well as between the sensor and both the shutter and the aperture, the light source for illuminating the sensor on multiple occasions when a lens cap is on, or the shutter or the aperture is closed, or combinations thereof;

a processor; and

one or more digital storage devices having instructions embedded therein for programming the processor to;

control the light source to illuminate the sensor;

create a sensor defect map based on a calibration image acquired by the sensor when illuminated by the light source;

correct an acquired image based on the sensor defect map;

construct a second defect map based on a second calibration image; and

correct the corrected image in accordance with said second defect map.

27. The device of claim 26, wherein the processor is programmed to initiate illumination of the sensor for creating the sensor defect map when the shutter is closed.

28. The device of claim 27, wherein the processor is programmed to interpret coupling of the lens assembly to the camera body acts as a trigger for said initiating illumination of the sensor for creating the sensor defect map.

29. The device of claim 27, wherein the processor is programmed to initiate said illumination of the sensor for creating the sensor defect map just before an image of a scene is acquired by the device.

30. The device of claim 27, wherein the processor is programmed to initiate said illumination of the sensor for creating the sensor defect map on a predetermined schedule during inactivity of the device.

31. The device of claim 27, wherein the processor is programmed to interpret turning-on the device as a trigger for initiating illumination of the sensor for creating the sensor defect map.

36. A method of correcting acquired digital images for detected sensor defects, comprising:

providing a camera body that is configured for coupling with a lens assembly and that contains an electronic image sensor, and a light source;

optically enclosing the sensor and light source thereby preventing exposure of the sensor by ambient light, including closing an aperture of the lens assembly;

only while said sensor and light source are optically enclosed due to said closing of said aperture, illuminating the sensor with the light source on multiple occasions;

detecting a calibration image acquired by the sensor when illuminated by the light source on a first occasion; and

creating a sensor defect map based on the calibration image,

generating a corrected image based on an acquired image and the sensor defect map;

constructing a second defect map based on a further calibration image acquired by the sensor on a second occasion; and

correcting the corrected image in accordance with said second defect map.

37. The method of claim 36, wherein preventing exposure of the sensor by ambient light by enclosing the light source is controlled by a camera body shutter, a lens aperture, a lens cap, an on/off camera switch, a predetermined schedule, or a lens assembly coupling detection switch, or combinations thereof.

38. The method of claim 36, further comprising coupling the lens assembly to the camera body.

39. The method of claim 36, wherein the light source is disposed between a lens aperture and a camera body shutter, and the shutter is disposed between the light source and the sensor, and wherein the optical enclosing comprises closing the aperture, and the illuminating further comprises opening the shutter.

41. The method of claim 36, further comprising mapping an acquired calibration image to a real sensor defect position map.

42. The method of claim 36, further comprising automatically updating an existing sensor defect map upon creating a new sensor defect map.

40. A method of correcting acquired digital images for detected sensor defects, comprising:

providing a camera body that is configured for coupling with a lens assembly and that contains an electronic image sensor, and a light source;

optically enclosing the sensor and light source thereby preventing exposure of the sensor by ambient light, wherein the light source is disposed between both a camera body shutter and aperture of the lens assembly, and the sensor, and wherein the optical enclosing comprises closing the shutter;

illuminating the sensor with the light source on multiple occasions;

detecting a calibration image acquired by the sensor when illuminated by the light source;

creating a sensor defect map based on the calibration image; and

generating a corrected image based on an acquired image and the sensor defect map;

constructing a second defect map based on a further calibration image acquired by the sensor on a second occasion; and

correcting the corrected image in accordance with said second defect map.

43. One or more digital storage devices having digital code embodied therein for programming one or more processors to perform a method of correcting acquired digital images for detected sensor defects, the method comprising:

initiating an optical enclosure of an image sensor and a light source thereby preventing exposure of the sensor by ambient light, including closing an aperture;

during said optical enclosure of said image sensor due to said closing of said aperture, illuminating the sensor with the light source on multiple occasions;

detecting a calibration image acquired by the sensor when illuminated by the light source; and

creating a sensor defect map based on the calibration image; and

generating a corrected image based on an acquired image and the sensor defect map;

constructing a second defect map based on a further calibration image acquired by the sensor on a second occasion; and

correcting the corrrected image in accordance with said second defect map.

44. The one or more storage devices of claim 43, wherein preventing exposure of the sensor by ambient light by enclosing the, light source is controlled by a camera body shutter, a lens aperture, a lens cap, an on/off camera switch, a predetermined schedule, or a lens assembly coupling detection switch, or combinations thereof.

45. The one or more storage devices of claim 43, the method further comprising coupling the lens assembly to the camera body.

46. The one or more storage devices of claim 43, wherein the light source is disposed between a lens aperture and a camera body shutter, and the shutter is disposed between the light source and the sensor, and wherein the optical enclosing comprises closing the aperture, and the illuminating further comprises opening the shutter.

48. The one or more storage devices of claim 43, the method further comprising mapping an acquired calibration image to a real sensor defect position map.

49. The one or more storage devices of claim 43, the method further comprising automatically updating an existing sensor defect map upon creating a new sensor defect map.

47. One or more digital storage devices having digital code embodied therein for programming one or more processors to perform a method of correcting acquired digital images for detected sensor defects, the method comprising:

initiating an optical enclosure of an image sensor and a light source thereby preventing exposure of the sensor by ambient light, wherein the light source is disposed between a camera body shutter and the sensor, and wherein the optical enclosing comprises closing the shutter;

illuminating the sensor with the light source periodically at equal intervals of at least one day;

detecting a calibration image acquired by the sensor when illuminated by the light source; and

creating a sensor defect map based on the calibration image; and

generating a corrected image based on an acquired image and the sensor defect map;

constructing a second defect map based on a further calibration image acquired by the sensor on a second occasion; and

correcting the corrected image in accordance with said second defect map.