Super-resolving imaging system

• US 6,344,893 B1
• Filed: 06/19/2000
• Issued: 02/05/2002
• Est. Priority Date: 06/19/2000
• Status: Expired due to Term

• Current Assignee: Ramot University Authority For Applied Research and Industrial Development Limited
• Original Assignee: Ramot University Authority For Applied Research and Industrial Development Limited
• Inventors: Konforti, Naim, Marom, Emanuel, Shabtay, Gal, Levy, Uriel, Karako, Sharon, Mendlovic, David, Zalevsky, Zeev
• Primary Examiner(s): Tarcza, Thomas H.
• Assistant Examiner(s): Mull, Fred H
• Application Number:
  US09/596,445
• Time in Patent Office: 596 Days
• Field of Search: 356/4.04, 356/3.14, 356/3.15, 359/558, 359/559, 235/462.22, 235/462.23, 235/462.24, 235/462.32, 235/462.35
• US Class Current: 356/3.14
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G02B 27/42 Diffraction optics , i.e. systems including a diffractive element being designed for providing a diffractive effectG02B27/60 takes precedence
• CPC Class Codes: G02B 27/4205 having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
• CPC Class Codes: G02B 27/58 Optics for apodization or superresolution; Optical synthetic aperture systems
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A super-resolving imaging apparatus employs diffractive optical elements placed on the imaging lens. This element, and the use of a modified Scheimpflug arrangement allow the conversion of degrees of freedom in one axis of a field of view to a larger degree of freedom in another axis in order to obtain a high resolution image with a wide depth of focus and large field of view. Replicas created by the diffractive elements are mutually shifted by subpixel amounts, and are combined using a Gabor transform, which is facilitated by a spatial mask placed over the detector array. The apparatus is suitable for performing distance estimation on an object within the field of view.

Claims

1. A method of imaging, comprising the steps of:

tilting an image plane with respect to an object plane;

defining pixel units on said image plane;

defining a field of view on said object plane, wherein a vertical dimension of said field of view differs from a horizontal dimension thereof;

forming a plurality of images of said field of view on said image plane, said images being spatially interrelated by sub-pixel shifts; and

super-resolving said images into an enhanced image.

2. The method according to claim 1, wherein said step of super-resolving is geometrically performed by transforming said images according to a Gabor transform.

3. The method according to claim 2, wherein said step of super-resolving further comprises applying a spatial mask to said image plane.

4. The method according to claim 3, wherein said spatial mask is given by a function $m$

(x)=1-∑

i=1M

rect

(xδ



 



xi)⊗

δ



(x-xi),

wherein M is a total number of slits of said mask, X₁, X₂, . . . , X_M, are locations of said slits, δ

x_i is a width of an i^th slit,and {circle around (x)} denotes convolution; and

rect( ) is given by $\mathrm{rect}<br><br>\left(z\right)=\{\begin{array}{cc}1,& -1/2<<br><br>z<<br><br>1/2,\\ 0,& \mathrm{otherwise}.\end{array}$

5. The method according to claim 1, wherein said images are oriented according to a larger of said horizontal dimension and said vertical dimension.

6. The method according to claim 1, further comprising deconvolving out-of-focus point spread functions of said images.

7. The method according to claim 6, wherein said step of deconvolving is performed by a Wiener filter.

8. The method according to claim 1, wherein said plurality of images are formed by time multiplexing.

9. The method according to claim 1, wherein said plurality of images are formed by wavelength multiplexing.

10. The method according to claim 1, wherein said plurality of images are formed by diffractive gratings.

11. The method according to claim 1, further comprising the steps of transforming said plurality of images according to a Mellin transform to define transformed images, and performing a time-to-impact analysis on said transformed images.

12. The method according to claim 1, wherein said step of forming a plurality of images further comprises:

spatially transforming coordinates of said plurality of images to establish a panoramic field of view, said panoramic field or view comprising a horizontal field of view and a vertical field of view, wherein a magnitude of said horizontal field of view differs from said magnitude of said vertical field of view.

13. A method of imaging comprising the steps of

simultaneously viewing a plurality of elongated fields of view on a target, wherein a focus of each of said fields of view has a different optical displacement from an imaging surface;

diffracting beams of radiant energy that travel along a path extending between said fields of view and said imaging surface to form elongated replicas of each of said fields of view, said replicas being mutually shifted in a direction of elongation thereof, and being mutually displaced in a direction that is substantially transverse to said direction of elongation; and

combining said replicas of each of said fields of view into corresponding enhanced images, each of said enhanced images having a higher resolution than resolutions of its associated replicas.

14. The method according to claim 13, further comprising the steps of:

forming an image plane that is tilted with respect to planes of said fields of view; and

determining defocus of said fields of view.

15. The method according to claim 14, wherein said step of combining is performed by performing a Gabor transform on said replicas.

16. The method according to claim 14, wherein said step of combining is performed by performing a Mellin transform on an image comprising all of said replicas.

17. The method according to claim 14, further comprising deconvolving out-of-focus point spread functions of said images.

18. The method according to claim 17, wherein said step of deconvolving is performed by a Wiener filter.

19. The method according to claim 14, further comprising the step of conducting said beams through a spatial mask that is constructed such that zero values of a Fourier transform of a function representing spatial responsivity of pixels on said image plane are eliminated.

20. A method of imaging, comprising the steps of:

tilting an image plane with respect to a first object plane and a second object plane, wherein a displacement of said second object plane from said image plane is greater than a displacement of said first object plane from said image plane;

defining pixel units on said image plane;

defining a first field of view on said first object plane, and defining a second field of view on said second object plane, wherein vertical dimensions of said first field of view and said second field of view differ from respective horizontal dimensions thereof;

forming a plurality of first images of said first field of view on said image plane, said first images being spatially interrelated by sub-pixel shifts;

simultaneously forming a plurality of second images of said second field of view on said image plane, said second images being spatially interrelated by sub-pixel shifts;

super-resolving said first images into a first enhanced image, and super-resolving said second images into a second enhanced image; and

comparing a defocus magnitude of said first field of view with a defocus magnitude of said second field of view.

21. The method according to claim 20, wherein said first field of view is obtained through a first region of a lens, and said second field of view is obtained through a second region of said lens.

22. The method according to claim 20, wherein said step of super-resolving is geometrically performed by transforming said first images and said second images according to a Gabor transform.

23. The method according to claim 22, wherein said step of super-resolving further comprises applying a spatial mask to said image plane.

24. The method according to claim 23, wherein said spatial mask is given by a function $m$

(x)=1-∑

i=1M

rect

(xδ



 



xi)⊗

δ



(x-xi),

wherein M is a total number of slits of said mask, X₁, X₂, . . . , X_M, are locations of said slits, δ

x_i is a width of an i^th slit, and {circle around (x)} denotes convolution; and

rect( ) is given by $\mathrm{rect}<br><br>\left(z\right)=\{\begin{array}{cc}1,& -1/2<<br><br>z<<br><br>1/2,\\ 0,& \mathrm{otherwise}.\end{array}$

25. The method according to claim 20, wherein said plurality of first images and said plurality of second images are oriented according to a larger of said horizontal dimension and said vertical dimension.

26. The method according to claim 20, further comprising determining a displacement between said image plane and said target responsive to said step of comparing a defocus magnitude.

27. The method according to claim 20, further comprising deconvolving out-of-focus point spread functions of said first images and said second images.

28. The method according to claim 27, wherein said step of deconvolving is performed by a Wiener filter.

29. The method according to claim 20, wherein said first images and said second images are formed by time multiplexing.

30. The method according to claim 20, wherein said first images and said second images are formed by wavelength multiplexing.

31. The method according to claim 20, wherein said first images and said second images are formed by diffractive gratings.

32. The method according to claim 20, further comprising the steps of by transforming said first images and said second images according to a Mellin transform to produce a transformed first image and a transformed second image,and performing a time-to-impact analysis on said transformed first image and said transformed second image.

33. The method according to claim 20, wherein said first images and said second images are spaced apart, and further comprising determining a position of a target with respect to said first images and said second images by triangulation.

34. An imaging apparatus, comprising:

a lens having a field of view that includes an object plane, wherein a horizontal dimension of said field of view differs from a vertical dimension thereof;

a detector of radiant energy having an image formed thereon by said lens, said detector comprising a plurality of pixel elements;

a spatial mask disposed proximate said detector, said mask having a plurality of subpixel apertures formed therein;

a diffractive element disposed proximate said lens, wherein a diffracted light beam forms said image on said detector, and said image comprises a plurality of focused replicas of said field of view, said focused replicas being offset from one another by subpixel shifts; and

prism disposed between said lens and said detector for refracting radiant energy passing therethrough onto said detector.

35. The apparatus according to claim 34, wherein said spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of said pixel elements.

36. The apparatus according to claim 34, wherein said diffractive element comprises a one dimensional zone plate strip.

37. The apparatus according to claim 34, wherein said lens comprises means for spatially transforming said plurality of focused replicas to establish said horizontal dimension and said vertical dimension as a panoramic field of view on said detector.

38. The apparatus according to claim 34, wherein said replicas are mutually shifted in a direction of elongation of said field of view, and are spaced apart from one another in a direction that is substantially transverse to said direction of elongation.

39. The apparatus according to claim 34, further comprising a signal processing unit which is responsive to said detector, and is programmed to combine said replicas into a single image having a greater resolution than said replicas.

40. The apparatus according to claim 39, wherein said signal processing unit performs a Gabor transform on a signal output of said detector.

41. The apparatus according to claim 39, wherein said signal processing unit performs a Mellin transform on a signal output of said detector.

42. The apparatus according to claim 39, wherein said lens comprises a first lens and a second lens spaced apart therefrom.

43. The apparatus according to claim 42, wherein said signal processing unit is programmed to determine a displacement by triangulation of a target using a first image thereof obtained from said first lens and a second image thereof obtained from said second lens.

44. The apparatus according to claim 43, wherein said signal processing unit is responsive to said first image and said second image on said detector.

45. The apparatus according to claim 39, wherein an aperture of said lens is divided into a first region and a second region, wherein said signal processing unit is programmed to determine a displacement by triangulation of a target, using a first image thereof obtained from said first region of said lens and a second image thereof obtained from said second region of said lens.

46. An imaging apparatus, comprising:

a plurality of lenses each having a corresponding field of view that includes an object plane, wherein a horizontal dimension of said field of view differs from a vertical dimension thereof, said lenses having different focal lengths;

a detector of radiant energy having a plurality of corresponding images formed thereon by said lenses, said detector comprising a plurality of pixel elements;

a spatial mask disposed proximate said detector, said mask having a plurality of subpixel apertures formed therein;

a diffractive element disposed proximate each of said lenses, wherein each said corresponding image on said detector comprises a plurality of focused replicas of said corresponding field of view, said focused replicas being offset from one another by subpixel shifts; and

a prism disposed between said lens and said detector for refracting radiant energy passing therethrough onto said detector.

47. The apparatus according to claim 46, wherein said prism comprises a plurality of prisms, each said prism refracting radiant energy that is received from a corresponding said lens.

48. The apparatus according to claim 46, wherein said spatial mask is formed so as to eliminate zero values from a Fourier transform of a response function of said pixel elements.

49. The apparatus according to claim 46, wherein said diffractive element comprises a one dimensional zone plate strip.

50. The apparatus according to claim 46, wherein said replicas are mutually shifted in a direction of elongation of said corresponding field of view, and are spaced apart from one another in a direction that is substantially transverse to said direction of elongation.

51. The apparatus according to claim 46, further comprising a signal processing unit which is responsive to said detector, and is programmed to combine said replicas of each said field of view into a corresponding single image having a greater resolution than said replicas.

52. The apparatus according to claim 51, wherein said signal processing unit performs a Gabor transform on a signal output of said detector.

53. The apparatus according to claim 51, wherein said signal a processing unit performs a Mellin transform on a signal output of said detector.

54. The apparatus according to claim 51, wherein said signal processing unit deconvolves out-of-focus point spread functions of said images.

55. The method according to claim 54, wherein said signal processing unit employs a Wiener filter for deconvolving said point spread functions.