Device, image processing device and method for optical imaging

• US 8,629,930 B2
• Filed: 03/29/2011
• Issued: 01/14/2014
• Est. Priority Date: 10/14/2009
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Brueckner, Andreas, Duparr, Jacques, Wippermann, Frank, Dannberg, Peter, Braeuer, Andreas
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): NGUYEN, CHAN T H
• Application Number:
  US13/074,089
• Publication Number:
  US 20110228142A1
• Time in Patent Office: 1,022 Days
• Field of Search: 348/208.11, 348/E3.031, 348/335, 348/340, 359619-624
• US Class Current: 348/340
• CPC Class Codes: G02B 27/0025 for optical correction, e.g. distorsion, aberration
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An optical device for imaging is disclosed having at least one micro lens field with at least two micro lenses and one image sensor with at least two image detector matrices. The at least two image detector matrices each include a plurality of image detectors and there is an allocation between the image detector matrices and the micro lenses, so that each micro lens together with an image detector matrix forms an optical channel. The center points of the image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that the optical channels have different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices. Further, an image processing device and a method for optical imaging are described.

Claims

1. Device for optical imaging, comprising:

at least one micro lens field comprising at least two micro lenses;

an image sensor comprising at least two image detector matrices;

wherein the at least two image detector matrices each comprise a plurality of image detectors;

wherein an image detector corresponds to one pixel of the image sensor;

wherein an allocation between the image detector matrices and the micro lenses exists, so that each micro lens together with an image detector matrix forms an optical channel;

wherein center points of different image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that at least two of the optical channels comprise different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices;

wherein the device is implemented to generate, in each optical channel, a micro image of the respective detection area;

wherein each of the micro images comprises a plurality of pixels, wherein each of the plurality of pixels is formed by an imaging of an object cell onto an image detector; and

wherein the micro image of a first optical channel is interleaved with the micro image of a second optical channel such that along a line, neighboring object cells of the overlapping area of the first and the second channel are alternatingly associated to different micro images.

2. The device according to claim 1, wherein the overlapping area comprises a plurality of object cells, wherein each of the object cells may be imaged onto at least one image detector of the image sensor;

and wherein a sampling grid is associated with each optical channel;

wherein a sampling grid of the first optical channel describes which object cells of the overlapping area are to be imaged onto which image detectors of the image detector matrix of the first optical channel; and

wherein a sampling grid of the second optical channel describes which object cells of the overlapping area are to be imaged onto which image detectors of the image detector matrix of the second optical channel;

and wherein the sampling grid of the first optical channel is selected such that a first object cell is imaged onto a first image detector of the image detector matrix of the first optical channel and that a second object cell neighboring the first object cell is imaged onto a gap between the first image detector of the image detector matrix of the first optical channel and a second image detector of the image detector matrix of the first optical channel neighboring the first image detector;

and wherein the sampling grid of the second optical channel is selected such that the second object cell is imaged onto a first image detector of the image detector matrix of the second optical channel and that the first object cell is imaged onto a gap between the first image detector of the image detector matrix of the second optical channel and a second image detector, neighboring the first image detector, of the image detector matrix of the second optical channel.

3. The device according to claim 2, wherein the sampling grid of the first optical channel is shifted by a non-integer multiple of a sampling interval with respect to the sampling grid of the second optical channel,

so that a third object cell neighboring the second object cell is imaged on the second image detector of the image detector matrix of the first optical channel;

and

wherein the sampling interval is equal to the distance of the center point of the first object cell to the center point of the third object cell.

4. The device according to claim 1, wherein the image sensor is arranged in a focal plane of the micro lenses of the micro lens field;

wherein between the image sensor and the micro lens field a spacer layer is located;

wherein the spacer layer comprises a plurality of channel separation structures which are implemented to suppress optical crosstalk between neighboring optical channels;

and wherein the spacer layer comprises at least one aperture field which is implemented to suppress scattered light through the gaps between the micro lenses.

5. The device according to claim 1, wherein a plurality of the optical channels comprise different spectral transmission fields, so that in each of the optical channels one associated micro image results, wherein micro images of different channels are associated with different spectral colors;

wherein the optical channels are separated by channel separation structures to suppress color crosstalk

wherein neighboring channels comprise different spectral transmission fields and the micro images of the neighboring channels are interleaved such that along the line, neighboring object cells of the overlapping area of the optical channels are alternatingly associated with the different spectral micro images.

6. The device according to claim 1 including a sensor for measuring the distance between the device and an object to be detected;

wherein the device is implemented to correct an offset of an imaging of a first object cell onto a first image detector of an image detector matrix of the first optical channel with respect to an imaging of a second object cell adjacent to the first object cell onto a first image detector of an image detector matrix of the second optical channel, and to consider, in the correction, the lateral distance of the image detector matrix of the first optical channel to the image detector matrix of the second optical channel.

7. The device according to claim 1, wherein different micro lenses are different regarding at least one of their structural parameters;

wherein the micro lenses are formed such that they all image into the same focal plane.

8. The device according to claim 1, wherein at least one micro lens is implemented as an achromatic lens.

9. The device according to claim 1, wherein the micro lens field and/or an interface between a transparent material and air comprises an anti-reflection layer.

10. The device according to claim 1, wherein the micro lens field and the image detector matrices are arranged in a hexagonal or a square or a rectangular distribution.

11. The device according to claim 1, wherein the micro lens field is implemented as a field of refractive free-form surfaces.

12. The device according to claim 1, wherein the device is implemented such that in each optical channel the micro image with the plurality of pixels results;

wherein the device comprises an image processor for correcting the distortion of the micro images and for the reconstruction of an overall image from the micro images of the optical channels;

wherein the image processor is implemented to execute the correction of the distortion for each micro image separately and parallel in time;

and wherein the image processor is implemented to execute the reconstruction of the overall image by alternatingly arranging the pixels of the interleaved micro images in order to thus acquire the overall image.

13. The device according to claim 12, wherein the image processor is implemented on a chip comprising the image sensor;

wherein the image processor comprises a plurality of micro image processors for correcting distortions which are arranged in the gaps of the image detector matrices of the image sensor.

14. An image processing device, comprising:

an image sensor comprising a plurality of image detectors,

wherein a first plurality of image detectors is associated with a first image detector matrix;

wherein a second plurality of image detectors is associated with a second image detector matrix; and

wherein the individual images of neighboring channels are interleaved;

an image processor for correcting image errors and for the reconstruction of an overall image from the images on the image detectors of the image detector matrices;

wherein the image processor comprises a plurality of processors for correcting distortions;

wherein the image processor is implemented to execute the correction of the image errors of the individual images of all image detector matrices independently of each other parallel in time and to reconstruct an overall image from the individual images such that the interleaving of the individual images is considered;

and a unit for receiving a micro lens field which is implemented to mount a micro lens field on the image processing device such that the image sensor is located in the focal plane of the micro lens field.

15. The image processing device according to claim 14, wherein the image processor is implemented on a chip with the image sensor.

16. The image processing device according to claim 15, wherein the plurality of processors are arranged for correcting distortions in the gaps of the image detector matrices of the image sensor.

17. A method for optical imaging of an object using a micro lens field comprising at least two micro lenses and an image sensor, the image sensor including a plurality of image detector matrices and each image detector matrix including a plurality of image detectors, wherein an image detector matrix is associated with each micro lens so that each micro lens together with the associated image detector matrix forms an optical channel, comprising:

a) imaging the object through the micro lens field onto the image sensor such that each object cell is imaged onto an image detector,

wherein neighboring object cells are imaged onto image detectors of neighboring optical channels so that micro images are formed on the image detector matrices of the optical channels, wherein each micro image is interleaved with the micro images of the neighboring channels;

wherein each micro image comprises a plurality of pixels, wherein each of the plurality of pixels is formed by an imaging of an object cell onto an image detector; and

wherein the micro image of a first optical channel is interleaved with the micro image of a second optical channel such that along a line, neighboring object cells of the overlapping area of the first and the second channel are alternatingly associated to different micro images.

18. The method for optical imaging according to claim 17, the method further including:

b) horizontal and vertical mirroring of the individual micro images of the optical channels, wherein the mirroring of all micro images is executed parallel in time;

c) converting the pixel values of the micro images of different optical channels from their discrete grid structure by means of a location transformation onto a continuous coordinate plane, wherein a correction of image distortions of the micro images of different optical channels is performed;

d) imaging the pixel values from the continuous coordinate plane onto a new discrete grid to acquire undistorted micro images, wherein different optical channels are processed parallel in time; and

e) resorting the pixel values of the undistorted micro images into an image matrix according to a given scheme which is given by the interleaving of the micro images in order to acquire a non-interleaved overall image.

19. The method according to claim 18, wherein the conversion of the pixel values of the micro images of different optical channels from their discrete grid structure by means of a location transformation is executed onto a continuous coordinate plane parallel in time.

20. A non-transitory computer readable medium including a computer program comprising a program code for executing a method for optical imaging of an object using a micro lens field comprising at least two micro lenses and an image sensor, the image sensor including a plurality of image detector matrices and each image detector matrix including a plurality of image detectors, wherein an image detector matrix is associated with each micro lens so that each micro lens together with the associated image detector matrix forms an optical channel, the method comprising:

a) imaging the object through the micro lens field onto the image sensor such that each object cell is imaged onto an image detector,

wherein neighboring object cells are imaged onto image detectors of neighboring optical channels so that micro images are formed on the image detector matrices of the optical channels, wherein each micro image is interleaved with the micro images of the neighboring channels;

wherein each micro image comprises a plurality of pixels, wherein each of the plurality of pixels is formed by an imaging of an object cell onto an image detector; and

wherein the micro image of a first optical channel is interleaved with the micro image of a second optical channel such that along a line, neighboring object cells of the overlapping area of the first and the second channel are alternatingly associated to different micro images.

21. A device for optical imaging, comprising:

at least one micro lens field comprising at least two micro lenses;

an image sensor comprising at least two image detector matrices;

wherein the at least two image detector matrices each comprise a plurality of image detectors;

wherein one image detector corresponds to one pixel of the image sensor;

wherein an allocation between the image detector matrices and the micro lenses exists so that each micro lens together with an image detector matrix forms an optical channel;

wherein center points of different image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that at least two of the optical channels comprise different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices;

wherein the overlapping area comprises a plurality of object cells, wherein each of the object cells may be imaged onto at least one image detector of the image sensor;

and wherein a sampling grid is associated with each optical channel;

wherein a sampling grid of the first optical channel describes which object cells of the overlapping area are to be imaged onto which image detectors of the image detector matrix of the first optical channel; and

wherein a sampling grid of the second optical channel describes which object cells of the overlapping area are to be imaged onto which image detectors of the image detector matrix of the second optical channel;

and wherein the sampling grid of the first optical channel is selected such that a first object cell is imaged onto a first image detector of the image detector matrix of the first optical channel and that a second object cell neighboring the first object cell is imaged onto a gap between the first image detector of the first image detector matrix of the first optical channel and a second image detector, neighboring the first image detector, of the image detector matrix of the first optical channel;

and wherein the sampling grid of the second optical channel is selected such that the second object cell is imaged onto a first image detector of the image detector matrix of the second optical channel and that the first object cell is imaged onto a gap between the first image detector of the image detector matrix of the second optical channel and a second image detector, neighboring the first image detector, of the image detector matrix of the second optical channel.

22. A device for optical imaging, comprising:

at least one micro lens field comprising at least two micro lenses;

an image sensor comprising at least two image detector matrices;

wherein the at least two image detector matrices each comprise a plurality of image detectors;

wherein an allocation between the image detector matrices and the micro lenses exists, so that each micro lens together with an image detector matrix forms an optical channel;

wherein center points of different image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that at least two of the optical channels comprise different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices;

which comprises a sensor for measuring the distance between the device and an object to be detected;

wherein the device is implemented to consider an offset of an imaging of a first object cell onto a first image detector of an image detector matrix of a first optical channel with respect to an imaging of a second object cell neighboring the first object cell onto a first image detector of an image detector matrix of a second optical channel, and to consider the lateral distance of the image detector matrix of the first optical channel to the image detector matrix of the second optical channel in the correction.

23. A method for the optical imaging of an object using a micro lens field comprising at least two micro lenses and an image sensor, the image sensor including a plurality of image detector matrices and each image detector matrix including a plurality of image detectors and an image detector matrix being associated with each micro lens so that each micro lens together with the associated image detector matrix forms an optical channel, comprising:

a) imaging the object through the micro lens field onto the image sensor such that each object cell is imaged onto an image detector,

wherein neighboring object cells are imaged onto image detectors of neighboring optical channels so that micro images are formed on the image detector matrices of the optical channels, wherein each micro image is interleaved with the micro images of the neighboring channels;

b) horizontal and vertical mirroring of the individual micro images of the optical channels, wherein the mirroring of any micro images is executed parallel in time;

c) converting the pixel values of the micro images of different optical channels from their discrete grid structure by means of a location transformation onto a continuous coordinate plane, wherein a correction of image distortions of the micro images of different optical channels is executed;

d) imaging the pixel values from the continuous coordinate plane onto a new discrete grid to acquire undistorted micro images, wherein different optical channels are processed parallel in time; and

e) resorting the pixel values of the undistorted micro images into an image matrix according to a predefined scheme which is given by the interleaving of the micro images in order to acquire a non-interleaved overall image.

24. Device for optical imaging, comprising:

at least one micro lens field comprising at least two micro lenses;

an image sensor comprising at least two image detector matrices;

wherein the at least two image detector matrices each comprise a plurality of image detectors;

wherein an image detector corresponds to one pixel of the image sensor;

wherein an allocation between the image detector matrices and the micro lenses exists, so that each micro lens together with an image detector matrix forms an optical channel;

wherein center points of different image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that at least two of the optical channels comprise different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices;

wherein a plurality of the optical channels comprise different spectral transmission fields, so that in each of the optical channels one associated micro image results, wherein micro images of different channels are associated with different spectral colors;

wherein the optical channels are separated by channel separation structures to suppress color crosstalk;

wherein neighboring channels comprise different spectral transmission fields and the micro images of the neighboring channels are interleaved such that along a line, neighboring object cells of the overlapping area of the optical channels are alternatingly associated with the different spectral micro images.

25. Device for optical imaging, comprising:

at least one micro lens field comprising at least two micro lenses;

an image sensor comprising at least two image detector matrices;

wherein the at least two image detector matrices each comprise a plurality of image detectors;

wherein an image detector corresponds to one pixel of the image sensor;

wherein an allocation between the image detector matrices and the micro lenses exists, so that each micro lens together with an image detector matrix forms an optical channel;

wherein center points of different image detector matrices are shifted laterally by different distances, with respect to centroids, projected onto the image detector matrices, of the micro lens apertures of the associated optical channels, so that at least two of the optical channels comprise different partially overlapping detection areas and so that an overlapping area of the detection areas of two channels is imaged onto the image detector matrices offset with respect to an image detector raster of the image detector matrices;

wherein the device is implemented such that in each optical channel a micro image with a plurality of pixels results;

wherein the device comprises an image processor arranged and programmed to correct distortion of the micro images and to reconstruct an overall image from the micro images of the optical channels;

wherein the image processor is implemented to execute the correction of the distortion for each micro image separately and parallel in time; and

wherein the image processor is implemented to execute the reconstruction of the overall image by alternatingly arranging the pixels of the interleaved micro images in order to thus acquire the overall image.