Optoelectronic camera and method for image formatting in the same

• US 6,765,617 B1
• Filed: 07/08/1999
• Issued: 07/20/2004
• Est. Priority Date: 11/14/1997
• Status: Expired due to Fees

• Current Assignee: Geirr I. Leistad, Hans Gude Gudesen, Per-Erik Nordal, Reidar E. Tangen
• Original Assignee: Geirr I. Leistad, Hans Gude Gudesen, Per-Erik Nordal, Reidar E. Tangen
• Inventors: Gudesen, Hans Gude, Tangen, Reidar E., Nordal, Per-Erik, Leistad, Geirr I.
• Primary Examiner(s): Christensen, Andrew
• Assistant Examiner(s): Harris, Tia M.
• Application Number:
  US09/341,311
• Time in Patent Office: 1,839 Days
• Field of Search: 348/219.1, 348/340
• US Class Current: 348/340
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H04N 2209/046 Colour interpolation to calculate the missing colour values
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/611 Correction of chromatic aberration

Abstract

An optoelectronic camera comprises an objective system formed by a number of optical active structures (L), particularly refractive structures in the form of microlenses or lenslets provided in an array. A detector device (D) is assigned to the lens array and comprises detectors (D_n) formed by sensor elements (E) which define pixels in the optical image. Each detector (D_n) defines a sample of the optical image and optimally all samples are used to generate a digital image. The optoelectronic camera may be realized as a color image camera, particularly for recording images in an RGB system. In a method for digital electronic formatting of an image recorded with the optoelectronic camera, zoom and pan functions are implemented in the camera.

Claims

1. An optoelectronic camera, comprising:

an optical objective system for imaging a scene recorded by the camera as an optical image substantially in an image plane of the objective system, including;

an optoelectronic detector device substantially provided in the image plane for detecting the optical image and on basis of the detection outputting output signals, a processor device connected with the detector device for converting and processing the output signals of the detector device in order to reproduce the detected image in digital form, and a memory device connected with the processor device for storing the digital image or for storing, displaying or possible additional processing on external devices adapted for these purposes and whereto the camera may be connected,

an array of two or more optical active structures, each optical active structure is adapted for generating optical images of the recorded scene on areas of the objective system image plane uniquely assigned to the respective optical active structure,

wherein at least one optoelectronic detector is provided for each optical active structure in its respective assigned area or image plane, all detectors being included in the detector device of the camera, each detector including at least one sensor element uniquely defining a pixel of the optical image, the area of the pixel substantially being determined by the area of the separate defining sensor element, wherein each detector is adapted to define a sample of the optical image with a number of pixels in each sample determined by a number of sensor elements in the defining detector, the digital image optimally being generated by all samples and with a spatial resolution determined by the number of pixels in distinct positions in the optical image defined by the sensor elements, wherein the ratio between the sensor element area of the detector with a total of the detector area is substantially less than 1.

2. Optoelectronic camera according to claim 1,

wherein the optical active structures are refractive structures or diffractive structures or reflective structures or combinations of such structures.

3. Optoelectronic camera according to claim 2,

wherein the refractive or the diffractive structures are formed as lenslets with a diameter of at most 3 mm.

4. Optoelectronic camera according to claim 1,

wherein the objective system with the optical active structures forms a substantially rigid or flexible surface.

5. Optoelectronic camera according to claim 4,

wherein the substantially rigid or flexible surface is a plane, curved or double-curved surface.

6. Optoelectronic camera according to claim 1,

wherein each detector comprises an array of two or more sensor elements such that each sensor element in the array defines a spatially distinct pixel in the optical image.

7. Optoelectronic camera according to claim 1,

wherein the sensor elements all have identical form factor, and that the area of the optical image expressed in pixels hence is given by the ratio between the geometric area of the optical image and the geometric area of a single sensor element.

8. Optoelectronic camera according to claim 1,

wherein the total number of distinctly defined pixels in the optical image is equal to the total number of sensor elements in the detector device, such that a one-to-one relation between a given pixel and its defining sensor element is present, whereby the digital image may be generated by a full sampling of the optical image.

9. Optoelectronic camera according to claim 1,

wherein the total number of distinctly defined pixels in the optical image is smaller than the total number of sensor elements in the detector device, such that a one-to-many relation between a given pixel and its defining sensor element or sensor elements is present, whereby the digital image may be generated by an oversampling of the optical image.

10. Optoelectronic camera according to claim 1,

wherein two or more detectors define identical spatial samples of the optical image.

11. Optoelectronic camera according to claim 1,

further comprising:

one or more spatial filters provided in front of the objective system and/or between the objective system and the detector device.

12. Optoelectronic camera according to claim 11,

wherein the spatial filter is a spatial light modulator.

13. Optoelectronic camera according to claim 12,

wherein the spatial light modulator is a controllable electrooptical light modulator.

14. Optoelectronic camera according to claim 1,

further comprising:

one or more optical filter means provided in front of the objective system and/or between the objective system and the detector device.

15. Optoelectronic camera according to claim 14,

wherein the optical filter means comprises separate, spectral selective filters which separately are assigned to either each optical active structure or groups of optical active structures, or to the detector or detectors of the detector device assigned to each optical active structure.

16. Optoelectronic camera according to claim 15,

wherein the optical filter means is adapted for transmitting in two or more separate wavelength bands by each spectral-selective filter transmitting in a separate wavelength band, the number of filters which transmit in each of the separate wavelength bands substantially being identical.

17. Optoelectronic camera according to claim 16,

wherein the separate wavelength bands in adjacent or not adjacent bandwidth relationship combined at least cover the visual part of the spectrum.

18. Optoelectronic camera according to claim 16,

wherein the separate wavelength bands are selected such that the optical filter means forms a primary colour filter means or an RGB filter means.

19. Optoelectronic camera according to claim 16,

wherein the separate wavelength bands are selected such that the optical filter means forms a complementary colour filter means.

20. Optoelectronic camera according to claim 17,

wherein the spectral-selective filters of the filter means separately are assigned or superpositioned to the detector or detectors provided for each optical active structure, and

wherein the spectral-selective filter is a strip filter which is adapted for transmitting in two or more separate wavelength bands by each strip transmitting in a given wavelength band, the number of strips which transmits in each of the wavelength bands substantially being identical.

21. Optoelectronic camera according to claim 10,

wherein each strip in a strip filter is assigned to a respective row or column of sensor elements in the detector or detectors.

22. Optoelectronic camera according to claim 20,

wherein each strip filter is a primary colour filter or an RGB filter.

23. Optoelectronic camera according to claim 15, wherein the spectral-selective filters of the filter means separately are assigned or superpositioned to the detector or detectors provided for each optical active structure, and the spectral selective filter is a mosaic filter which is adapted for transmitting in two or more separate wavelength bands by each filter segment in the mosaic filter transmitting in a given wavelength band, the number of filter segments which transmit in each of the wavelength bands substantially being identical.

24. Optoelectronic camera according to claim 23,

wherein each filter segment in the mosaic filter is assigned to a respective sensor element or respective sensor elements in the detector or detectors.

25. Optoelectronic camera according to claim 23,

wherein each mosaic filter is a complementary colour filter.

26. Optoelectronic camera according to claim 1,

wherein the detector device comprises detectors realized in one of the following technologies, viz. CCD (charge-coupled device) technology, CID (charge-injected device) technology, APS (active pixel sensor) technology or PMSA (sensor array in passive matrix) technology.

27. Optoelectronic camera according to claim 26, wherein the detector is realized in PMSA technology, and the detector is realized as a thin-film component or a hybrid component, and that the detector is adapted for parallel read-out of the output signals from the sensor elements over a passive electrode array for unique addressing of each separate sensor element.

28. Optoelectronic camera according to claim 27,

wherein the detector is made wholly or partly of organic semiconducting or electrical isolating materials, including plastic materials and semiconducting oligomers or polymers.

29. Optoelectronic camera according to claim 28,

wherein the organic materials wholly or partly transmit light in at least the visual part of the spectrum, and that the detector in the area between its sensor elements is transparent or translucent in this spectral range.

30. Optoelectronic camera according to claim 28,

wherein the electrode array of the detector wholly or partly is transparent or translucent in at least the visual range of the spectrum.

31. Optoelectronic camera according to claim 1,

wherein the optical active structures are realized with a determined chromatic aberration or dispersion, such that each optical active structure for two or more separate wavelength bands spectral-selectively generates the optical image in each wavelength band substantially on corresponding substantially congruent image planes superpositioned spaced apart in the ray direction, and that for each optical active structure in each of these image planes a detector for spectral-selective detection of the optical image is provided such that for each optical active structure on each image plane a sample in the spatial domain and a sample in the frequency domain are defined, the resolution in the frequency domain substantially being determined by the number of separate wavelength bands with a respective assigned detector, whereby the optical image detected by the detector device may be generated as a multispectral digital colour image with the use of a selected suitable colour system.

32. Optoelectronic camera according to claim 31,

wherein three separate superpositioned detectors are provided for each optical active structure, respectively in the image plane for three separate wavelength bands assigned to a three-colour system.

38. A method for digital electronic formatting of a recorded full-format optical image in an optoelectronic camera according to claim 1, wherein the recorded optical image is stored as a digital image in a memory in a processor device provided in the camera and may be displayed on a display device connected to the processor device,

further comprising, generating a section or field of the full-format digital image by substantially continuous or stepwise radial or axial contraction of the image towards respectively a convergence point or a convergence axis in the image, the contraction of the image taking place digitally in a data processor provided in the processor device and according to one or more determined pixel-subtracting protocols and being effected by an in-camera or externally provided operating device which is manoeuvred manually by a camera operator and automatically according to predetermined criteria, and by once again expanding formatted field radially or axially in this way stepwise or continuously from respectively the convergence point or from the convergence axis towards a full-format image.

39. A method according to claim 38,

wherein the formatting is visualised on the display device, the section or field caused by the formatting at any instant being displayed as a synthetic full-format image on the display device, but with a real spatial resolution given by the corresponding pixel subtraction value of the formatting.

40. A method according to claim 38,

wherein a digital electronic zoom function is being implemented in the camera by the radial contraction or expansion, the field format being determined as respectively a telephoto, wide angle or macro format depending on the distance between the scene and the image plane in the camera, and by a digital electronic pan function being implemented in the camera by the axial contraction or expansion.

41. Method according to claim 38,

wherein the convergence point and the convergence axis default automatically and are chosen respectively by the intersection between the optical axis and the image plane or the horizontal bisecting line of the image plane, and by the convergence point and the convergence axis being manually selected over the operating device as respectively an arbitrary point or an arbitrary axis of the image.

42. The optoelectronic camera of claim 1, wherein the detected image is displayed in real time on a display device provided in the camera and connected with the processor device.

33. Optoelectronic camera, particularly for recording colour images and even more particularly for recording colour images in an RGB system, comprising

an optical objective system for imaging a scene recorded by the camera as an optical image substantially in an image plane of the objective system, an optoelectronic detector device substantially provided in the image plane for detecting the optical image and on basis of the detection outputting output signals, a processor device connected with the detector device for converting and processing the output signals of the detector device in order to reproduce the detected image in digital form and possibly for displaying this in real time on a display device optionally provided in the camera and connected with the processor device, and a memory device connected with the processor device for storing the digital image for displaying on the optional display device which also may be connected with the memory device, or for storing, displaying or possible additional processing on external devices adapted for these purposes and whereto the camera temporarily or permanently is connected,

wherein the camera objective system is formed by an array of two or more optical active structures, each optical active structure has a determined chromatic aberration or dispersion such that the location of its focal point depends on the wavelength of the light, each optical structure is adapted for generating spectral-selectively optical images of the recorded scene on areas of three separate superpositioned image planes of the objective system, said areas being uniquely assigned to respective optical active structures, a first image plane forming a first optical image on a wavelength band in the blue portion of the spectrum, and a second image plane forming a second optical image on a wavelength band in the green portion of the spectrum and a third image plane forming a third optical image on a wavelength band in the red portion of the spectrum, that for each optical active structure an optoelectronic detector is provided in each of the respective assigned image planes for detection of the optical image generated by the optical active structure in each of the wavelength bands blue, green and red, that each detector including at least one sensor element, such that at least one sensor element uniquely defines a pixel of the optical image, the area of the pixel being substantially determined by the area of the separate defining sensor element, that each detector in one of the image planes is adapted for defining a sample of the optical image in the wavelength band corresponding to this image plane and with a number of pixels in each sample determined by the number of sensor elements E in the defining detector (D_n), the digital image optimally being generated as an RGB colour image with a spatial resolution determined by the number of pixels, by the sensor elements defined positions in the optical image.

34. Optoelectronic camera according to claim 33,

wherein the optical active structures are refractive structures with a determined chromatic aberration or diffractive structures with a determined dispersion or combinations of such structures.

35. Optoelectronic camera according to claim 34,

wherein the refractive or diffractive structures are realized as lenslets with a diameter of at most 3 mm.

36. Optoelectronic camera according to claim 33,

wherein the total number of distinctly defined pixels in the optical image in one of the wavelength bands is equal to the total number of sensor elements in the detectors for this wavelength band provided in the detector device such that in this case a one-to-one relation between a given pixel and its defining sensor element is present, whereby the digital RGB colour image can be generated with a full sampling of the optical image in each wavelength band and with three times oversampling of the whole optical image in colours.

37. Optoelectronic camera according to claim 33,

wherein the total number of distinctly defined pixels of the optical image in one of the wavelength bands is smaller than the total number of sensor elements in the detectors for this wavelength band provided in the detector device, such that in this case a one-to-many relation between a given pixel and its defining sensor element or sensor elements in this case is present, whereby the digital RGB colour image can be generated with an oversampling in each wavelength band and with a total oversampling of the optical image equal to the sum of the oversampling factor in each wavelength band.