Focusing Mechanisms for Compressive Imaging Device

• US 20120038817A1
• Filed: 08/11/2011
• Published: 02/16/2012
• Est. Priority Date: 08/11/2010
• Status: Active Grant

• Current Assignee: Inview Technology Corporation
• Original Assignee: Inview Technology Corporation
• Inventors: McMackin, Lenore, Hewitt, Donna E., Weston, Tyler H., Zollars, Byron
• Granted Patent: US 8,717,492 B2
• Time in Patent Office: Days
• US Class Current: 348/345
• CPC Class Codes: G02B 26/0833 the reflecting element being a micromechanical device, e.g. a MEMS mirror, DMD G02B26/0825 takes precedence; micromechanical devices in general B81B
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/75 by influencing optical camera components
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof

Abstract

A compressive imaging (CI) device including a light modulator and a light sensor (e.g., a single-element sensor or a sensor array). The CI device may support the focusing of light on the light modulator and/or on the light sensor in a number of ways: (1) determining a focus indicator value by analyzing a 1D or 2D image in the incident light field; (2) measuring light spillover between modulator regions and light sensing elements—either at the level of voltage measurements or the level of reconstructed images; (3) measure noise in reconstructed sub-images; (4) measuring an amount high-frequency content in the incident light field; (5) incorporating a range finder to measure distance to the object being imaged; (6) incorporating an image sensor downstream from the modulator; and (7) splitting a portion of the incident light onto an image sensor, prior to the modulator.

Claims

1. A method for focusing a compressive imaging (CI) device, the method comprising:

(a) supplying a sequence of spatial patterns to a light modulation unit of the CI device;

(b) the light modulation unit modulating an incident light stream with the sequence of spatial patterns to produce a modulated light stream;

(c) a light sensing device of the CI device receiving the modulated light stream and acquiring a sequence of samples representing an intensity of the modulated light stream as a function of time, wherein the sequence of samples includes at least one sample for each of the spatial patterns;

(d) constructing an image using the sequence of spatial patterns and the acquired samples;

(e) computing a focus indicator value based on the image, wherein the focus indicator value indicates an extent to which the incident light stream is in focus at the light modulation unit.

2. The method of claim 1, wherein each of the spatial patterns is restricted to a subset of the array of light modulating elements.

3. The method of claim 2, wherein the subset of the array of light modulating elements is a convex region located at the center of the array of light modulating elements.

4. The method of claim 1, wherein the CI device includes an optical subsystem that receives and operates on the incident light stream prior to the incident light stream arriving at the light modulation unit.

5. The method of claim 4, further comprising:

changing a focus setting of the optical subsystem based on data including the focus indicator value.

6. The method of claim 5, further comprising repeatedly performing a set of operations that includes:

changing a focus setting of the optical subsystem; and

performing (a), (b), (c), (d) and (e), wherein the set of operations is repeated until the focus indicator value becomes less than a threshold value.

7. The method of claim 4, further comprising:

changing an optical distance between the optical subsystem and the light modulation unit based on data including the focus indicator value.

8. The method of claim 4, further comprising repeatedly performing a set of operations that includes:

changing an optical distance between the optical subsystem and the light modulation unit; and

performing (a), (b), (c), (d) and (e), wherein the set of operations is repeated until the focus indicator value becomes less than a threshold value.

9. The method of claim 8, further comprising:

changing an optical distance between the light modulation unit and the light sensing device based on a final distance between the optical subsystem and the light modulation unit after the focus indicator value becomes less than the threshold value.

10. The method of claim 1, further comprising:

changing an angle of orientation of the light modulation unit based on data including the focus indicator value.

11. The method of claim 4, further comprising:

directing an actuator to change an optical distance between the optical subsystem and the light modulation unit according to a first displacement value;

performing (a) through (e) a second time, to obtain an additional image and a focus indicator value for the additional image;

computing a second displacement value for the optical distance based on data including the focus indicator value for the image, the focus indicator value for the additional image and the first displacement value.

12. The method of claim 11, further comprising:

directing the actuator to change the optical distance according to the second displacement value.

13. The method of claim 4, further comprising:

directing an actuator to change a focus setting of the optical subsystem according to a first displacement value;

performing (a) through (e) a second time, to obtain an additional image and a focus indicator value for the additional image;

computing a second displacement value for the focus setting based on data including the focus indicator value for the image, the focus indicator value for the additional image and the first displacement value.

14. The method of claim 13, further comprising:

directing the actuator to change the focus setting of the optical subsystem according to the second displacement value.

15. The method of claim 4, further comprising:

directing an actuator to change an optical distance between the optical subsystem and the light modulation unit through a range of distances;

performing (a) through (e) for each of a plurality of distances within the range in order to obtain a corresponding plurality of images and a corresponding plurality of focus indicator values;

determining an optimal value for the optical distance based on an analysis of data including the plurality of focus indicator values.

16. The method of claim 15, further comprising:

directing the actuator to change the optical distance between the optical subsystem and the light modulation unit to the optimal value.

17. The method of claim 4, further comprising:

directing an actuator to change the focus setting of the optical subsystem through a range of settings;

performing (a) through (e) for each of a plurality of focus settings within the range in order to obtain a corresponding plurality of images and a corresponding plurality of focus indicator values; and

determining an optimal value for the focus setting based on an analysis of data including the plurality of focus indicator values.

18. The method of claim 17, further comprising:

directing the actuator to change the focus setting to the optimal value.

19. The method of claim 1, wherein the light sensing device includes a plurality of light sensing elements, wherein each of the light sensing elements generates a respective electrical signal representing intensity of a respective spatial portion of the modulated light stream, wherein said acquiring the sequence of samples comprises adding the electrical signals to obtain a sum signal and sampling the sum signal.

20. The method of claim 1, wherein the light sensing device includes a plurality of light sensing elements, wherein each of the light sensing elements generates a respective electrical signal representing intensity of a respective spatial portion of the modulated light stream, wherein said acquiring the sequence of samples comprises sampling each of the electrical signals and adding the sampled versions of the electrical signals in the digital domain.

21. The method of claim 1, wherein the light sensing device includes only one light sensing element.

38. The method of claim 37 2.31, further comprising:

directing the actuator to change the optical distance between the light modulation unit and the array of light sensing elements according to the second displacement value.

39. The method of claim 31 2.27, further comprising:

directing an actuator to change an angle of orientation of the array of light sensing elements according to a first displacement value;

performing (a) through (e) a second time, to obtain an additional composite spillover value;

computing a second displacement value for the angular orientation based on data including the composite spillover value, the additional composite spillover value and the first displacement value.

40. The method of claim 39, further comprising:

directing the actuator to change the angle of orientation of the array of light sensing elements according to the second displacement value.

22. A method for focusing a compressive imaging (CI) device, the method comprising:

(a) supplying a sequence of spatial patterns to a light modulation unit of the CI device;

(b) the light modulation unit modulating an incident light stream with the sequence of spatial patterns to produce a modulated light stream;

(c) acquiring a sequence of the frames from an array of light sensing elements of the CI device, wherein each of the light sensing elements receives a respective spatial portion of the modulated light stream, wherein each of the frames corresponds to respective one of the spatial patterns, wherein each of the frames includes one sample from each of the light sensing elements;

(d) determining a sequence of sum values, wherein each of the sum values is determined by adding the samples in a respective one of the frames of the acquired sequence;

(e) constructing an image using the sequence of spatial patterns and the sequence of sum values;

(f) computing a focus indicator value based on the image, wherein the focus indicator value indicates an extent to which the incident light stream is in focus at the light modulation unit.

23. A method for determining focus-related information for a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with the sequence of spatial patterns, thereby generating a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device;

(b) generating an electrical signal representing intensity of the modulated light stream as a function of time, wherein the electrical signal is generated by a light sensing device of the CI device;

(c) acquiring samples of the electrical signal;

(d) constructing an image using the sequence of spatial patterns and a subset of the acquired samples that corresponds to the sequence of spatial patterns;

(e) computing focus information based on the image, wherein the focus information comprises information regarding at extent to which the incident light stream is in focus at the light modulation unit.

24. The method of claim 23, wherein the number of spatial patterns in the sequence of spatial patterns is m, wherein the image is an n-pixel image with n less than the number N of light modulating elements in the light modulation unit, wherein m is less than n.

25. The method of claim 23, further comprising:

changing an optical distance between an optical subsystem of the CI device and the light modulation unit based on the focus information.

26. The method of claim 23, further comprising:

changing a focus setting of an optical subsystem of the CI device based on the focus information.

27. A method for focusing a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatial patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device;

(b) generating an electrical signal representing intensity of the modulated light stream as a function of time, wherein the electrical signal is generated by a light sensing device of the CI device;

(c) acquiring samples of the electrical signal, wherein the acquired samples include at least one sample for each of the spatial patterns;

(d) constructing an image using the sequence of spatial patterns and the acquired samples;

(e) displaying the image on a display;

(f) changing a focus setting of the CI device based on user input.

28. The method of claim 27, wherein the focus setting of the CI device is an optical distance between an optical subsystem of the CI device and the light modulation unit.

29. The method of claim 27, wherein the focus setting of the CI device is a focus setting of an optical subsystem of the CI device.

30. The method of claim 27, further comprising:

performing (a) through (f) one or more additional times.

31. A method for focusing a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with calibration patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, where the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each electrical signal represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions within the array of light modulating elements, wherein each of the calibration patterns specifies that the light modulating elements inside a respective one of the regions are to be set to an off state and specifies that at least a subset of the light modulating elements outside the respective region are to be set to an on state;

(c) for each of the calibration patterns, acquiring a respective group of samples from the array of light sensing elements, wherein each of the sample groups includes at least one sample of each of the electrical signals;

(d) determining spillover values corresponding respectively to the calibration patterns, wherein each of the spillover values is determined based on the sample group corresponding to the respective calibration pattern, wherein each of the spillover values indicates an extent to which modulated light from outside the respective region of the array of light modulating elements reaches the light sensing element corresponding to the respective region.

(e) computing a composite spillover value for the array of light sensing elements based on the spillover values.

32. The method of claim 31, further comprising:

directing an actuator to change an optical distance between the light modulation unit and the array of light sensing elements based on data including the composite spillover value.

33. The method of claim 31, further comprising repeatedly performing a set of operations that includes:

changing an optical distance between the light modulation unit and the array of light sensing elements; and

performing (a) through (e), wherein the set of operations is repeated until the composite spillover value becomes less than a threshold value.

34. The method of claim 31, further comprising:

directing an actuator to change an angle of orientation of the array of light modulating elements based on data including the composite spillover value.

35. The method of claim 31, further comprising repeatedly performing a set of operations that includes:

changing an angle of orientation of the array of light sensing elements; and

performing (a) through (e), wherein the set of operations is repeated until the composite spillover value becomes less than a threshold value.

36. The method of claim 31, wherein each of the calibration patterns specifies that all the light modulating elements outside the respective region are to take the ON state.

37. The method of claim 31, further comprising:

directing an actuator to change an optical distance between the light modulation unit and the array of light sensing elements according to a first displacement value;

performing (a) through (e) a second time, to obtain an additional composite spillover value;

computing a second displacement value for the optical distance between the light modulation unit and the array of light sensing elements based on data including the composite spillover value, the additional composite spillover value and the first displacement value.

41. The method of claim 31, further comprising:

directing an actuator to change an optical distance between the light modulation unit and the array of light sensing elements through a range of distances;

performing (a) through (e) for each of a plurality of optical distances within the range in order to obtain a corresponding plurality of composite spillover values;

determining an optimal value for the optical distance based on an analysis of data including the plurality of composite spillover values.

42. The method of claim 41, further comprising:

directing the actuator to change the optical distance to the optimal value.

43. The method of claim 31, further comprising:

directing an actuator to change an orientation angle of the array of light sensing elements through a range of angles;

performing (a) through (e) for each of a plurality of angles within the range in order to obtain a corresponding plurality of composite spillover values;

determining an optimal value for the angle based on an analysis of data including the plurality of composite spillover values.

44. The method of claim 43, further comprising:

directing the actuator to change the angular orientation to the optimal value.

45. A method for focusing a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a calibration pattern to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, where the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each electrical signal represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions within the array of light modulating elements, wherein the calibration pattern specifies that the light modulating elements inside a first one of the regions are to be set to an off state and specifies that at least a subset of the light modulating elements outside the first region are to be set to an on state, wherein the first region corresponds to a first of the light sensing elements;

(c) acquiring samples of the electrical signal generated by the first light sensing element in response to said modulating the incident light stream with the calibration pattern, wherein each of the samples indicates an extent to which modulated light from outside the first region of the array of light modulating elements reaches the first light sensing element.

46. A method for determining focus-related information for a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatial patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, wherein the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions of the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements, wherein the measurements include sample sets that correspond respectively to the light sensing elements, wherein each sample set includes samples of the electrical signal generated by the respective light sensing element, with the samples including at least one sample for each of the spatial patterns;

(d) computing a focus indicator value based on two or more noise values for two or more respective ones of the light sensing devices, wherein the focus indicator value indicates an extent to which the modulated light stream is in focus at the sensor array, wherein each of the two or more noise values is computed by;

constructing a respective sub-image based on the sample set of the respective light sensing element and also on a restriction of the spatial patterns to the region of the array of light modulating elements corresponding to the respective light sensing element; and

computing an amount of noise present in the respective sub-image.

47. The method of claim 46, further comprising:

determining a displacement value for the array of light sensing elements based on data including the focus indicator value; and

directing the array of light sensing elements to be translated according to the displacement value.

48. The method of claim 46, further comprising:

determining an angular displacement value for the array of light sensing elements based on data including the focus indicator value; and

directing the array of light sensing elements to be rotated according to the angular displacement value.

49. The method of claim 46, further comprising:

changing an optical distance between the light modulation unit and the array of light sensing elements; and

performing said changing and the operations (a) through (d) one or more times until the focus indicator value is optimized.

50. The method of claim 46, further comprising:

changing an angular orientation of the array of light sensing elements; and

performing said changing and the operations (a) through (d) one or more times until the focus indicator value is optimized.

51. A method for determining focus-related information for a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatial patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, wherein the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions of the array of light modulating elements;

(c) acquiring a set of samples from a first of the light sensing elements, wherein the sample set includes samples of the electrical signal generated by the first light sensing element, with the samples including at least one sample for each of the spatial patterns;

(d) computing a focus indicator value for the first light sensing device, wherein the focus indicator value indicates an extent to which the modulated light stream is in focus at the sensor array, wherein the focus indicator value is computed by;

constructing a sub-image based on the sample set and on a restriction of the spatial patterns to a the region of the array of light modulating elements corresponding to the first light sensing element; and

computing an amount of noise present in the sub-image.

52. A method for determining focus-related information for a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatial patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, wherein the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions on the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements, wherein the measurements include sample sets that correspond respectively to the light sensing elements, wherein each sample set includes samples of the electrical signal generated by the respective light sensing element, with the samples including at least one sample for each of the spatial patterns;

(d) computing a focus indicator value based on two or more spillover values for two or more respective ones of the light sensing devices, wherein the focus indicator value indicates an extent to which the modulated light stream is in focus at the sensor array, wherein each of the two or more spillover values is computed by;

constructing a respective image based on the sample set of the respective light sensing element and also on the spatial patterns;

summing pixels in the image outside the region corresponding to the respective light sensing element.

53. A method for determining focus-related information for a compressive imaging (CI) device, the method comprising:

(a) modulating an incident light stream with a sequence of spatial patterns to obtain a modulated light stream, wherein said modulating is performed by a light modulation unit of the CI device, wherein the light modulation unit includes an array of light modulating elements;

(b) an array of light sensing elements of the CI device receiving the modulated light stream, wherein the light sensing elements receive respective spatial portions of the modulated light stream and generate respective electrical signals, wherein each of the electrical signals represents intensity of the respective spatial portion as a function of time, wherein the light sensing elements correspond to respective non-overlapping regions on the array of light modulating elements;

(c) acquiring measurements from the array of light sensing elements, wherein the measurements include sample sets that correspond respectively to the light sensing elements, wherein each sample set includes samples of the electrical signal generated by the respective light sensing element, with the samples including at least one sample for each of the spatial patterns;

(d) computing a spillover value for a first of the light sensing devices, wherein the spillover value indicates an extent to which the modulated light stream is in focus at the sensor array, wherein the spillover value is computed by;

constructing an image based on the sample set corresponding to the first light sensing element and also on the spatial patterns;

summing pixels in the image outside the region corresponding to the first light sensing element.