Camera with image enhancement functions

• US 20060256226A1
• Filed: 01/15/2004
• Published: 11/16/2006
• Est. Priority Date: 01/16/2003
• Status: Active Grant

• Current Assignee: DigitalOptics Corporation (Adeia Inc.)
• Original Assignee: Dblur Technologies Ltd. (Adeia Inc.)
• Inventors: Alon, Alex, Alon, Irina
• Granted Patent: US 7,627,193 B2
• Time in Patent Office: Days
• US Class Current: 348/335
• CPC Class Codes: G02B 27/0012 Optical design, e.g. procedures, algorithms, optimisation routines
• CPC Class Codes: G02B 27/0025 for optical correction, e.g. distorsion, aberration
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: H04N 25/68 applied to defects
• CPC Class Codes: H04N 25/683 by defect estimation performed on the scene signal, e.g. real time or on the fly detection

Abstract

Imaging apparatus (20, 44) includes an array (22) of optical sensing elements (24), characterized by a pitch, which is adapted to generate a signal in response to optical radiation that is incident on the elements. Objective optics (26, 46), which have an optical axis (134) and are characterized by a cylindrical symmetry about the axis, are arranged to focus the optical radiation from an object onto the array with a point spread function (PSF) having an extent greater than twice the pitch of the array at an optimal focus of the objective optics.

Claims

1. Imaging apparatus, comprising:

an array of optical sensing elements, characterized by a pitch, which is adapted to generate an input image comprising pixels having respective pixel values in response to optical radiation that is incident on the elements;

objective optics, which have an optical axis and are characterized by a cylindrical symmetry about the axis, and which comprise one or more lenses that are arranged to focus the optical radiation from an object onto the array with a point spread function (PSF) having an extent greater than twice the pitch of the array at an optimal focus of the objective optics; and

an image processor, which is coupled to process the input image by applying a deconvolution filter, based on the PSF, to the input image so as to generate an electronic image of the object while reducing a blur of the electronic image,

wherein the one or more lenses have optical characteristics that are selected, responsively to the PSF, so as to optimize a quality of the electronic image following application of the deconvolution filter.

2. The apparatus according to claim 1, wherein the extent of the PSF is at least three times the pitch.

5. The apparatus according to claim 1, wherein the image processor is adapted to identify one or more bad elements in the array and to replace the pixel values of the pixels in a vicinity of the bad elements in the input image with substitute values so as to produce a corrected blurred image, and to apply the deconvolution filter to the corrected blurred image so as to generate a restored image.

7. The apparatus according to claim 1, wherein the image processor is adapted to modify the deconvolution filter that is applied to the input image responsively to a change in a focal relationship between the object and the array.

8. The apparatus according to claim 7, wherein the image processor is adapted to select a macro deconvolution filter when the object is less than a predetermined minimum distance from the objective optics.

60. The apparatus according to claim 1, wherein the optical characteristics of at least one of the lenses are selected by determining an enhanced PSF that can be achieved by application of a given image restoration function by the image processor to the input image produced by a given design of the objective optics, and modifying the design of the objective optics to reduce an extent of the enhanced PSF.

61. The apparatus according to claim 1, wherein the objective optics have a given aperture, and wherein the PSF is non-optimal with respect to the given aperture.

62. The apparatus according to claim 61, wherein the optical characteristics and deconvolution filter are chosen so as to increase a depth of field of the apparatus relative to the given aperture.

3-4. -4. (canceled)

6. (canceled)

9-16. -16. (canceled)

17. Imaging apparatus, comprising:

objective optics, which are arranged to focus optical radiation from an object onto a focal plane with a characteristic point spread function (PSF);

an array of optical sensing elements, which is positioned at the focal plane and is adapted to generate a signal in response to the optical radiation that is incident on the elements, the signal corresponding to an electronic image of the object comprising pixels having respective pixel values;

an image processor, which is coupled to receive the signal from the array and is adapted to identify one or more bad elements in the array and, responsively to the PSF, to modify the pixel values of the pixels in a vicinity of the bad elements in the input image so as to produce a corrected blurred image while reducing propagation of artifacts due to the bad elements, and to apply a deconvolution filter, based on the PSF, to the corrected blurred image so as to generate a restored output image.

18. The apparatus according to claim 17, wherein the image processor is adapted to iteratively modify the pixel values of the pixels in the vicinity of the bad elements in the restored image, so as to produce a new blurred image, and to apply the deconvolution filter to the new blurred image so as to generate a new restored image, until the electronic image with the reduced blur is generated for output.

19. The apparatus according to claim 17, wherein the image processor is adapted to determine substitute values with which to replace the pixel values of the bad pixels responsively to the PSF.

20. The apparatus according to claim 19, wherein the image processor is adapted to determine the substitute values using a correction map M[x] substantially of a form given by

21. The apparatus according to claim 17, wherein the image processor is adapted to add a frame of dummy pixels surrounding the input image, and to treat the dummy pixels similarly to the pixels associated with the bad elements in producing the corrected blurred image and generating the electronic image for output.

22-23. -23. (canceled)

24. A method for producing an optical system, comprising:

producing a design of the optical system;

computing an optical point spread function (PSF) of the optical system according to the design;

determining an enhanced PSF that can be achieved by application of a given image restoration function to an image produced by the optical system according to the design; and

modifying the design of the optical system to reduce an extent of the enhanced PSF.

25. The method according to claim 24, wherein producing the design comprises:

assigning respective merit scores to each of a plurality of optical aberrations;

determining the aberrations that are characteristic of the design;

computing a design score responsively to the aberrations and the respective merit scores; and

optimizing the design responsively to the design score.

26. The method according to claim 25, wherein assigning the respective merit scores comprises determining respective effects of each of the aberrations on the PSF, and calculating the respective merit scores responsively to the respective effects.

27. The method according to claim 25, wherein determining the aberrations comprises generating a Zernike polynomial expansion of the aberrations, and wherein computing the design score comprises calculating a weighted sum of the respective merit scores responsively to coefficients of the polynomial expansion.

28. The method according to claim 24, wherein determining the enhanced PSF comprises determining coefficients of a deconvolution filter responsively to the computed optical PSF, and applying the deconvolution filter to a representative image that is characterized by the computed optical PSF.

29. The method according to claim 24, wherein modifying the design comprises estimating effects of variations in the design on the enhanced PSF, and modifying the design of the optical system responsively to the estimated effects.

30. The method according to claim 29, wherein determining the enhanced PSF comprises computing an image quality score responsively to the enhanced PSF, and wherein estimating the effects of the variations comprises finding a dependence of the image quality score on the variations.

31. The method according to claim 30, wherein computing the image quality score comprises determining respective local point spread functions at a plurality of points in an image field of the optical system, and determining the image quality score responsively to the local point spread functions.

32. The method according to claim 31, wherein determining the image quality score comprises finding respective local scores at the plurality of the points, and combining the local scores to generate the image quality score.

33. The method according to claim 31, wherein determining the enhanced PSF comprises calculating a weighted PSF by taking a weighted sum of the local point spread functions, and wherein estimating the effects of the variations comprises calculating the effects of the variations on the a weighted PSF.

34. The method according to claim 30, wherein computing the image quality score comprises finding a resolution score, which is indicative of a resolution of an image produced with the enhanced PSF, and finding an artifact score, which is indicative of a level of artifacts in the image, and combining the resolution score and the artifact score to determine the image quality score.

35. The method according to claim 30, wherein computing the image quality score comprises determining the image quality score responsively to aberrations of the optical system, and wherein finding the dependence of the image quality score comprises calculating a gradient of the score with respect to changes in the aberrations.

36. The method according to claim 35, wherein finding the dependence of the image quality score comprises determining a response matrix, indicative of the changes in the aberrations that will result as a function of specific modifications to the design of the optical system, and wherein modifying the design comprises determining a modification to be applied to the design responsively to the gradient and the response matrix.

37. The method according to claim 24, and comprising iteratively repeating, with respect to the modified design of the optical system, computing the optical PSF, determining the enhanced PSF, and modifying the design.

38. The method according to claim 37, wherein iteratively repeating comprises performing iterations through computing the optical PSF, determining the enhanced PSF, and modifying the design until a modification to the design that is indicated by one of the iterations is less than a predetermined tolerance that is to be applied in manufacturing the optical system.

39. The method according to claim 24, wherein the optical system comprises one or more optical elements, and wherein the variations in the design comprise a change in at least one of a curvature of an optical surface of one of the elements, a thickness of one of the elements and a spacing between the elements.

40. The method according to claim 39, wherein the one or more optical elements comprise at least one aspheric element having at least one surface defined by a polynomial function, and wherein modifying the design comprises optimizing coefficients of the polynomial function.

43. The system according to claim 40, wherein the optical system comprises one or more optical elements, and wherein the variations in the design comprise a change in at least one of a curvature of an optical surface of one of the elements, a thickness of one of the elements and a spacing between the elements.

44. The system according to claim 43, wherein the one or more optical elements comprise at least one aspheric element having at least one surface defined by a polynomial function, and wherein modifying the design comprises optimizing coefficients of the polynomial function.

41. An optical system, comprising one or more optical elements, which are produced and assembled in accordance with a design that is optimized by computing an optical point spread function (PSF) of the optical system according to the design, determining an enhanced PSF that can be achieved by application of a given image restoration function to an image produced by the optical system according to the design, and modifying the design of the optical system to reduce an extent of the enhanced PSF.

42. The system according to claim 41, wherein the design is optimized by iteratively repeating, with respect to the modified design of the optical system, computing the optical PSF, determining the enhanced PSF, and modifying the design.

45. (canceled)

46. A method for electronic imaging, comprising:

generating an input image comprising pixels having respective pixel values, using an array of optical sensing elements that is characterized by a pitch, in response to optical radiation that is incident on the elements;

focusing the optical radiation from an object onto the array using objective optics, which comprise one or more lenses and have an optical axis and are characterized by a cylindrical symmetry about the axis, to form an optical image with a point spread function (PSF) having an extent greater than twice the pitch of the array; and

processing the input image responsively to the PSF by applying a deconvolution filter, based on the PSF, to the input image so as to generate an electronic image of the object while reducing a blur of the electronic image,

wherein the one or more lenses have optical characteristics that are selected, responsively to the PSF, so as to optimize a quality of the electronic image following application of the deconvolution filter.

47. The method according to claim 46, wherein the extent of the PSF is at least three times the pitch.

50. The method according to claim 46, wherein processing the signal comprises identifying one or more bad elements in the array, and replacing the pixel values of the pixels in a vicinity of the bad elements in the input image with substitute values so as to produce a corrected blurred image, and applying the deconvolution filter to the corrected blurred image.

48-49. -49. (canceled)

51-59. -59. (canceled)