MULTIPLEXED IMAGING

• US 20110267482A1
• Filed: 01/15/2010
• Published: 11/03/2011
• Est. Priority Date: 01/19/2009
• Status: Active Grant

• Current Assignee: Guangdong OPPO Mobile Telecommunications Corporation Limited
• Original Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Inventors: Ihrke, Ivo Bodo, Heidrich, Wolfgang, Wetzstein, Gordon
• Granted Patent: US 8,860,856 B2
• Time in Patent Office: Days
• US Class Current: 348/218.1
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 3/4015 Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

An imaging method comprises acquiring image data in which image components are spatially modulated at distinct spatial frequencies, transforming the image data into the Fourier domain and separating the image components in the Fourier domain. The image components may be transformed into the spatial domain. The image components may comprise different colors. In some embodiments saturated pixels are reconstructed by performing an optimization based on differences between image copies in the Fourier domain. Imaging apparatus may perform the imaging methods.

Claims

1. A method for obtaining image data, the method comprising:

acquiring image data by exposing an imaging array to optical radiation and operating the imaging array, wherein acquiring the image data comprises spatially modulating a response of the imaging array to each of a plurality of components of the optical radiation according to a corresponding basis function for an invertible transformation;

applying the transformation to the image data to yield transformed image data, the transformed image data comprising spatially-separated image copies corresponding respectively to the plurality of components;

extracting the spatially-separated image copies from the transformed image data; and

,

applying an inverse of the transformation to each of the extracted image copies.

2. A method according to claim 1 wherein acquiring the image data comprises allowing the optical radiation to pass through an optical filter before interacting with pixels of the imaging array.

3. A method according to claim 2 wherein the optical filter has a spatial variation having at least first and second spatial frequencies.

4. A method according to claim 3 wherein the optical filter comprises tiles that repeat in a multiple of a pitch of pixels of the imaging array.

5. A method according to claim 2 wherein the optical filter has a transmissivity in excess of 45%.

6. A method according to claim 1 wherein the basis functions corresponding to the plurality of components are mutually orthogonal.

7. A method according to claim 6 wherein the components comprise primary colors.

8. A method according to claim 6 wherein the components comprise a red component a blue component and a green component and acquiring the image data comprises modulating each of the red, blue and green components at a distinct spatial frequency.

9. A method according to claim 1 wherein acquiring the image data comprises band-limiting a spatial frequency of the optical radiation prior to allowing the optical radiation to interact with the imaging array.

10. A method according to claim 1 wherein a direction of the spatial modulation is not aligned with rows or columns of the imaging array.

11. A method according to claim 1 wherein the modulation is spatially periodic and has a spatial frequency that is different for each of the components.

12. A method according to claim 1 wherein the basis functions comprise sinusoids.

13. A method according to claim 11 wherein acquiring the image data comprises spatially modulating a response of the imaging array to a first one of the plurality of components of the optical radiation according to a first sinusoid having a first spatial frequency and spatially modulating a response of the imaging array to a second one of the plurality of components of the optical radiation according to a second sinusoid having the first spatial frequency and a phase difference of ¼

wave with the first sinusoid.

14. A method according to claim 13 comprising extracting a pair of spatially-separated copies each corresponding to the first spatial frequency, computing a real part of a sum of the pair of spatially-separated copies and an imaginary part of a difference of the pair of spatially-separated copies, and applying the inverse of the transformation to the real part and the imaginary part.

15. A method according to claim 1 wherein the components comprise polarization states.

16. An automated method for reconstructing pixel values for saturated pixels in an image, the method comprising:

obtaining image data comprising a band-limited exposure of an image having some saturated pixels wherein the exposure is spatially modulated at two or more spatial frequencies by functions that differently attenuate the exposure;

identifying the saturated pixels in the image data;

computing a Fourier transform of the image data;

setting up an optimization problem in which pixel values for the saturated components are unknown and an error measure to be minimized comprises a difference between Fourier domain image copies corresponding to the two or more spatial frequencies; and

numerically solving the optimization problem to obtain pixel values for the saturated pixels.

17. A method according to claim 16 comprising inserting the pixel values for the saturated pixels into the image data.

18. A method according to claim 16 wherein the optimization problem comprises a spatial curvature constraint.

19. A method according to claim 16 wherein setting up the optimization problem comprises generating an initial estimate of the pixel values for the saturated pixels.

20. A method according to claim 19 wherein the initial estimate sets a pixel value at a centroid of a saturated region to a maximum value.

21. An imaging array comprising a filter wherein the filter transmissivity for each of a plurality of spectral bands varies spatially with a distinct spatial frequency.

22. An imaging array according to claim 21 wherein the transmissivity for each of the spectral bands varies sinusoidally.

23. An imaging array according to claim 21 wherein the plurality of spectral bands comprise a band passing red light, a band passing blue light and a band passing green light.

24. An automated image processing system comprising:

a processor;

software instructions for execution by the processor, the software instructions comprising instructions that configure the processor to;

obtain image data comprising a band-limited exposure of an image having some saturated pixels wherein the exposure is spatially modulated at two or more spatial frequencies by functions that differently attenuate the exposure;

identify the saturated pixels in the image data;

compute a Fourier transform of the image data;

set up an optimization problem in which pixel values for the saturated components are unknown and an error measure to be minimized comprises a difference between Fourier domain image copies corresponding to the two or more spatial frequencies; and

numerically solve the optimization problem to obtain pixel values for the saturated pixels.

25-26. -26. (canceled)