SYNTHETIC IMAGE FORMATION SIGNAL PROCESSING HARDWARE FOR VIGNETTED OPTOELECTRONIC ARRAYS, LENSLESS CAMERAS, AND INTEGRATED CAMERA-DISPLAYS

• US 20110032370A1
• Filed: 06/30/2010
• Published: 02/10/2011
• Est. Priority Date: 05/25/2008
• Status: Active Grant

• Current Assignee: NRI R&D Patent Licensing, LLC (Advanced Touchscreen and Gesture Technologies, LLC)
• Original Assignee: Avistar Communications Corporation (Avistar)
• Inventors: LUDWIG, Lester F.
• Granted Patent: US 8,284,290 B2
• Time in Patent Office: Days
• US Class Current: 348/218.1
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 25/702 SSIS architectures characterised by non-identical, non-equidistant or non-planar pixel layout
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 5/30 Transforming light or analogous information into electric information scanning details H04N3/00; cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00; circuitry of solid-state image sensors [SSIS] or control thereof H04N25/00

Abstract

Hardware arrangement performing electronic image formation and refinement from overlapping measurement vignettes captured by an array of image sensors and associated micro-optics are presented. The invention is directed to a new type of image formation system that combines readily-fabricated micro-optical structures, a two-dimensional image sensor array with electronic or digital image processing to actually construct the image. Image formation is performed without a conventional large shared lens and associated separation distance between lens and image sensor, resulting in a “lensless camera.”

In an application, a readily fabricatable LED array is used as a light-field sensor. In an application, the LED array further serves as a color “lensless camera.” In an application, the LED array also serves as an image display. In an application, the LED array further serves as a color image display. In an embodiment, one or more synergistic features of an integrated camera/display surface are realized.

Claims

1. A system for numerical image formation and for refinement from overlapping measurement vignettes captured by an image sensor array with associated micro-optical arrangements, the image sensor creating a plurality of light detection signals responsive to a plurality optically vignetted portions of an incoming light field associated with an observed scene, the system comprising:

a multiplexing arrangement for gathering said plurality of light detection signals into a stream of measurement values responsive to an incoming light field associated with an observed scene;

a processor for receiving the stream of measurement values, the processor executing the numerical image formation algorithm on the stream of measurement values to produce at least one synthesized digital image; and

an output for transferring said at least one synthesized digital image to an external system;

wherein the processor produces a visual image responsive to the incoming light field associated with an observed scene

2. The system of claim 1 wherein the light detection signals are produced by circuits measuring light-induced electrical behavior of LEDs.

3. The system of claim 2 wherein the light-induced electrical behavior is a photocurrent.

4. The system of claim 2 wherein the light-induced electrical behavior is a light-induced voltage.

5. The system of claim 2 wherein the light-induced electrical behavior is a light-induced charge.

6. The system of claim 2 wherein the light-emitting diode light sensors additionally configured to emit light act as a visual image display.

7. The optical-electronic array of claim 6 wherein the multiplexing arrangement is further configured to multiplex individual optical-electronic elements in at least a display mode and a sensor mode.

8. The system of claim 1 wherein the light detection signals are further processed by a nonlinear “

gamma correction”

element.

9. The system of claim 1 wherein the two-dimensional array of light sensors comprises a color LED array as a color light-field sensor.

10. The system of claim 7 wherein the light detection signals are further processed by a nonlinear “

gamma correction”

element applied separately for each color detection signal.

11. The system of claim 7 wherein the light detection signals are further processed by a nonlinear “

gamma correction”

element applied to the plurality of color detection signals, the treated as a plurality of color detection signals vector of color detection signals.

12. The system of claim 1 wherein the linear mathematical transformations employ linear combinations of measurements from a plurality of measurement values.

13. The system of claim 12 wherein the envelopes of at least two apertures overlap.

14. The system of claim 1 wherein the numerical image formation algorithm is adjustable, the adjustment responsive to configuration commands.

15. The system of claim 1 wherein the numerical image formation algorithm is adjustable, the adjustment responsive to control signals.

16. The system of claim 1 additionally comprising subsequent image processing operations.

17. The system of claim 16 wherein the subsequent image processing operations are adjustable, the adjustment responsive to configuration commands.

18. The system of claim 16 wherein the subsequent image processing operations are adjustable, the adjustment responsive to control signals.