Decreasing Image Acquisition Time for Compressive Imaging Devices

• US 20120038786A1
• Filed: 08/03/2011
• Published: 02/16/2012
• Est. Priority Date: 08/11/2010
• Status: Active Grant

• Current Assignee: Inview Technology Corporation
• Original Assignee: Inview Technology Corporation
• Inventors: Kelly, Kevin F., Baraniuk, Richard G., McMackin, Lenore, Bridge, Robert F., Chatterjee, Sujoy, Weston, Tyler H.
• Granted Patent: US 8,860,835 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

Mechanisms for increasing the rate of acquisition of compressed/encoded image representations are disclosed. An imaging system may deliver subsets of a modulated light stream onto respective light sensing devices. The light sensing devices may be sampled in parallel. Samples from each light sensing device may be used to construct a respective sub-image of a final image. The parallelism allows compressed images to be acquired at a higher rate. The number of light sensing devices and/or the number of pixels per image may be selected to achieve a target image acquisition rate. In another embodiment, spatial portions of the incident light stream are separated and delivered to separate light modulators. In yet another embodiment, the incident light stream is split into a plurality of beams, each of which retains the image present in the incident light stream and is delivered to a separate light modulator.

Claims

1. A system comprising:

a light modulation unit configured to modulate an incident stream of light with a sequence of spatial patterns in order to produce a modulated light stream, wherein the light modulation unit includes an array of light modulating elements;

a plurality of light sensing devices;

an optical subsystem configured to deliver light from each of a plurality of spatial subsets of the modulated light stream onto a respective one of the plurality of light sensing devices, wherein the spatial subsets of the modulated light stream are produced by respective subsets of the array of light modulating elements, wherein each subset of the array of light modulating elements produces the respective spatial subset of the modulated light stream by modulating a respective spatial subset of the incident light stream, wherein the light sensing devices are configured to generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial subset of the modulated light stream as a function of time; and

a sampling subsystem configured to acquire samples of the electrical signals, wherein the samples include sample subsets that correspond respectively to the electrical signals, wherein each sample subset includes a plurality of samples of the respective electrical signal.

2. The system of claim 1, wherein each of the sample subsets is usable to construct a respective sub-image of an image, wherein each sub-image represents a respective one of the spatial subsets of the incident light stream.

3. The system of claim 2, wherein N is the number of the light modulating elements, wherein L is the number of the light sensing devices, wherein n is the number of pixels in said image, wherein n is less than or equal to N, wherein the light modulation unit is configured to modulate the incident light stream with the spatial patterns at a maximum rate of R_P spatial patterns per unit time, wherein m_SS is the number of samples in each sample subset, wherein a minimum number m_n/L of compressive-sensing samples required to reconstruct an n/L-pixel image with given accuracy decreases as L increases, wherein m_SS and L are selected so that m_SS is greater than or equal to m_n/L, and m_SS/R_P is less than or equal to a target acquisition time.

4. The system of claim 2, wherein N is the number of the light modulating elements, wherein L is the number of the light sensing devices, wherein n is the number of pixels in said image, wherein n is less than or equal to N, wherein each sample subset is usable to construct the respective sub-image with n/L pixels, wherein the sampling subsystem is configured so that the number of samples in each sample subset is dynamically programmable to allow different tradeoffs between image quality and acquisition time of the samples of the electrical signals.

5. The system of claim 2, further comprising a processing unit configured to operate on each sample subset to construct the respective sub-image.

6. The system of claim 1, wherein the sampling subsystem is configured to acquire the samples of the electrical signals in groups, wherein each of the groups includes at least one sample of each of the electrical signals, wherein each of the groups corresponds to a respective one of the spatial patterns.

7. The system of claim 6, wherein said sampling subsystem is configured to acquire the groups of samples of the electrical signals at a maximum group rate of R_G groups per unit time, wherein m_SS is the number of samples in each sample subset, wherein N is the number of the light modulating elements, wherein n is less than or equal to N, wherein L is the number of the light sensing devices, wherein a number m_n/L of the compressive sensing samples required to reconstruct an n/L-pixel image decreases as L increases, wherein L is selected so that m_SS is greater than or equal to m_n/L, and, m_SS/R_G is less than or equal to a target acquisition time.

8. The system of claim 1, wherein the sampling subsystem includes a plurality of analog-to-digital converters (ADCs), wherein each of the ADCs is configured to acquire the samples of the electrical signal generated by a respective one of the light sensing devices.

9. The system of claim 8, wherein at least a first and a second of the ADCs have different bit resolutions.

10. The system of claim 1, wherein the optical subsystem includes a plurality of lenses, wherein each of the lenses is configured to direct the light from a respective one of the spatial subsets of the modulated light stream onto a light sensing surface of a respective one of the light sensing devices.

11. The system of claim 1, wherein the optical subsystem includes a fiber coupling device and a plurality of fiber bundles, wherein the fiber coupling device is configured to couple the light from each of the spatial subsets of the modulated light stream onto a respective one of the fiber bundles.

12. The system of claim 1, wherein the optical subsystem includes a plurality of light pipes whose internal surfaces are reflective.

13. The system of claim 1, wherein the optical subsystem includes a focusing lens, wherein the plurality of light sensing devices are located at an image plane of the focusing lens.

14. The system of claim 1, wherein the subsets of the array of the light modulating elements are contiguous subsets.

15. The system of claim 1, wherein at least a first and a second of the light sensing devices have different light sensitivities.

16. The system of claim 1, wherein the plurality of light sensing devices are arranged in an array.

17. The system of claim 16, wherein the array is a two-dimensional rectangular array, wherein the sampling subsystem includes one analog-to-digital converter per column of the array.

18. The system of claim 1, wherein the plurality of light sensing devices are incorporated into a first integrated circuit.

19. The system of claim 18, wherein the sampling subsystem includes a readout integrated circuit, wherein the readout integrated circuit is coupled to the first integrated circuit.

20. The system of claim 19, wherein the readout integrated circuit is bump bonded to the first integrated circuit.

21. A method comprising:

modulating an incident stream of light with a sequence of spatial patterns in order to produce a modulated light stream, where said modulating is performed by an array of light modulating elements;

delivering light from each of a plurality of spatial subsets of the modulated light stream onto a respective one of a plurality of light sensing devices, wherein the spatial subsets of the modulated light stream are produced by respective subsets of the array of light modulating elements, wherein each subset of the array of light modulating elements produces the respective spatial subset of the modulated light stream by modulating a respective spatial subset of the incident light stream;

generating electrical signals, wherein each of the electrical signals is generated by a respective one of the light sensing devices, wherein each of the electrical signals represents intensity of a respective one of the spatial subsets of the modulated light stream as a function of time; and

acquiring samples of the electrical signals, wherein the samples include sample subsets that correspond respectively to the electrical signals, wherein each sample subset includes a plurality of samples of the respective electrical signal.

22. The method of claim 21, wherein each of the sample subsets is usable to construct a respective sub-image of an image, wherein each sub-image represents a respective one of the spatial subsets of the incident light stream.

23. The method of claim 22, wherein N is the number of the light modulating elements, wherein L is the number of the light sensing devices, wherein n is the number of pixels in said image, wherein n is less than or equal to N, wherein the incident light stream is modulated with the spatial patterns at a maximum rate of R_P spatial patterns per unit time, wherein m_SS is the number of samples in each sample subset, wherein a minimum number m_n/L of compressive-sensing samples required to reconstruct an n/L-pixel image with given accuracy decreases as L increases, wherein m_SS and L are selected so that m_SS is greater than or equal to m_n/L, and m_SS/RP is less than or equal to a target acquisition time.

24. The method of claim 21, further comprising:

operating on the sample subsets to construct the respective sub-images.

25. The method of claim 21, further comprising:

transmitting the sample subsets to a receiver through a communication channel.

26. The method of claim 21, wherein said acquiring the samples of the electrical signals includes acquiring the samples in groups, wherein each of the groups includes at least one sample of each of the electrical signals, wherein each of the groups corresponds to a respective one of the spatial patterns.

27. The method of claim 26, wherein the groups are acquired at a rate of R_G groups per unit time, wherein N is the number of the light modulating elements, wherein n is less than or equal to N, wherein L is the number of the light sensing devices, wherein m_SS is the number of samples in each sample subset, wherein a number m_n/L of the compressive sensing samples required to reconstruct an n/L-pixel image decreases as L increases, wherein L is selected so that m_SS is greater than or equal to m_n/L, and, m_SS/R_G is less than or equal to a target acquisition time.

28. A system comprising:

a light modulation unit configured to modulate an incident stream of light with a sequence of spatial patterns in order to produce a modulated light stream, wherein the light modulation unit includes an array of light modulating elements;

a plurality of light sensing devices;

an optical subsystem configured to direct spatial subsets of the modulated light stream to respective ones of the light sensing devices, wherein the spatial subsets of the modulated light stream are produced by respective non-overlapping regions of the array of light modulating elements, wherein each region of the array of light modulating elements produces the respective spatial subset of the modulated light stream by modulating a respective spatial subset of the incident light stream, wherein each of the light sensing devices is configured to generate a respective electrical signal that represents intensity of the respective spatial subset of the modulated light stream as a function of time;

a plurality of analog-to-digital converters (ADCs), wherein each of the ADCs is configured to capture a respective sequence of samples of the electrical signal generated by a respective one of the light sensing devices, wherein the sequence of samples captured by each of the ADCs is usable to construct a respective sub-image that represents the corresponding spatial subset of the incident light stream.

29. The system of claim 28, wherein the optical subsystem includes a plurality of lenses, wherein each of the lenses is configured to direct a respective one of the spatial subsets of the modulated light stream onto a light sensing surface of a respective one of the light sensing devices.

30. The system of claim 28, wherein the optical subsystem includes a fiber coupling device and a plurality of fiber bundles, wherein the fiber coupling device is configured to couple light from each of the spatial portions of the modulated light stream onto a respective one of the fiber bundles, wherein each of the fiber bundles is configured to deliver the light from the respective spatial subset of the modulated light stream onto a light sensing surface of a respective one of the light sensing devices.

31. The system of claim 28, wherein the plurality of light sensing devices are incorporated into a single integrated circuit.

32. The system of claim 28, wherein the optical subsystem includes a focusing lens, wherein the plurality of light sensing devices are located at an image plane of the focusing lens.

33. The system of claim 28, wherein the plurality of ADCs are configured to capture their respective sample sequences synchronously.

34. The system of claim 28, wherein the plurality of light sensing devices are arranged in an array.

35. The system of claim 28, wherein each of the sample sequences has a sample size that is smaller than the number of pixels in the respective sub-image.

36. A system comprising:

a light modulation unit configured to modulate an incident stream of light with a sequence of spatial patterns in order to produce a modulated light stream, wherein the light modulation unit includes an array of N light modulating elements;

an array of L light sensing elements, wherein L is greater than one but less than N;

an optical subsystem configured to focus light from each of a plurality of spatial portions of the modulated light stream onto a respective one of the array of light sensing elements, wherein the spatial portions of the modulated light stream are produced by respective non-overlapping regions of the array of light modulating elements, wherein each region of the array of light modulating elements produces the respective spatial portion of the modulated light stream by modulating a respective spatial portion of the incident light stream, wherein the L light sensing elements are configured to generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial portion of the modulated light stream as a function of time; and

circuitry configured to read groups of samples from the array of L light sensing elements, wherein each group of samples includes L samples, wherein the L samples of each group include one sample for each of the electrical signals.

37. The system of claim 36, wherein the circuitry is configured to read the groups at a maximum rate of R_G groups per unit time, wherein each of the groups corresponds to a respective one of the spatial patterns, wherein the sequence of spatial patterns is configured so that M of said groups are usable to construct an N-pixel image representing the incident light stream, wherein M is greater than one and less than or equal to N/L.

38. The system of claim 37, wherein L is selected to achieve an acquisition time for the M groups that is less than or equal to a target acquisition time.

39. The system of claim 37, wherein M is less than N/L.

40. The system of claim 36, wherein the circuitry is configured to read the groups at a maximum rate of R_G groups per unit time, wherein each of the groups corresponds to a respective one of the spatial patterns, wherein the sequence of spatial patterns is configured so that subsets of the groups, each subset of groups being M groups in length, are usable to construct respective N-pixel images, wherein M is greater than one and less than or equal to N/L, wherein L is selected so that a rate of acquisition of the group subsets is greater than or equal to a target acquisition rate.

41. The system of claim 36, wherein the circuitry is included in a readout integrated circuit, wherein the array of L light sensing elements are included in a second integrated circuit.

42. The system of claim 41, wherein the readout integrated circuit is bump bonded to the second integrated circuit.

43. The system of claim 36, wherein the circuitry and the light sensing elements are included in a single integrated circuit.