Cameras with varying spatio-angular-temporal resolutions

• US 8,229,294 B2
• Filed: 09/18/2009
• Issued: 07/24/2012
• Est. Priority Date: 12/10/2007
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Agrawal, Amit Kumar, Veeraraghavan, Ashok
• Primary Examiner(s): Fuller, Rodney
• Assistant Examiner(s): Chang, Fang-Chi
• Application Number:
  US12/562,485
• Publication Number:
  US 20100003024A1
• Time in Patent Office: 1,040 Days
• Field of Search: 396/340, 398/207, 356/310, 348/499
• US Class Current: 396/340
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A camera includes a lens and a sensor. A dynamic mask is arranged at an aperture plane between the lens and the sensor, and a static mask is arranged immediately adjacent to the sensor. Angular, temporal or spatial variations in light rays acquired of a scene by the sensor are mapped to individual pixels of the sensor.

Claims

1. An apparatus, comprising:

a lens;

a sensor having a spatial resolution of P×

P pixels;

a dynamic mask having an angular resolution of K×

K locations arranged at an aperture plane between the lens and the sensor; and

a static mask arranged immediately adjacent to the sensor, wherein the sensor acquires a light field with a spatial resolution of

2. The apparatus of claim 1, wherein the static mask is a high frequency heterodyne mask.

3. The apparatus of claim 1, further comprising:

a processor connected to the sensor, wherein the sensor acquires a single input image for the processor, and the processor produces a set of output images.

4. The apparatus of claim 3, wherein the input image is acquired of a dynamic scene, and temporal variations in light rays from the scene are mapped to angular variations by the dynamic mask, and further comprising

acquiring the angular variations at the sensor as a light field using the static mask.

5. The apparatus of claim 4, wherein moving parts of the dynamic scene are recovered.

6. The apparatus of claim 3, wherein the input image is acquired of a static scene, and variations in angular and temporal dimensions of light rays from the scene are mapped by the static mask to spatial intensity variations on the sensor.

7. The apparatus of claim 6, wherein the output images of the set are different for different parts of the scene.

8. The apparatus of claim 6, wherein in-focus parts of the static scene have a higher spatial resolution in the set of output images.

9. The apparatus of claim 1, wherein the dynamic mask is configured to rotate and includes a pattern of pin holes.

10. The apparatus of claim 1, wherein the dynamic mask is configured to rotate and includes a slit.

11. The apparatus of claim 10, wherein angular variations in light rays from a scene are mapped to a vertical dimension and temporal variation in the light rays are mapped to a horizontal dimension.

12. The apparatus of claim 1, wherein the dynamic mask includes low resolution liquid crystal diodes.

13. The apparatus of claim 1, further comprising:

a glass plate separating the static mask and the sensor.

14. The apparatus of claim 1, wherein the static mask is a sum of cosine masks for the static mask.

15. The apparatus of claim 1, wherein the static mask is a tiled broadband mask.

16. The apparatus of claim 1, wherein the static mask is a lenslet array.

17. The apparatus of claim 1, wherein the dynamic mask use Hadamard codes.

18. The apparatus of claim 1, wherein the static mask is an array of uniformly spaced pinholes.

19. The method for acquiring an input image of a scene, comprising:

modulating a light field from a scene entering a camera through a lens at an aperture plane using a dynamic mask; and

acquiring the modulated light field at a sensor as a single image using a static mask placed immediately adjacent to the sensor, wherein temporal variations of the light field are mapped to angular variations by partitioning an exposure time T into K² intervals of duration T/K², and during each of the K² intervals, only one of the K² locations of the dynamic mask is open, while all other locations are closed.

20. The method of claim 19, further comprising:

producing a set of output images from the single image using a processor connected to the sensor.

21. The method of claim 19, wherein the scene is static, and further comprising:

converting the input image to a high resolution image.

22. The method of claim 19, further comprising:

converting the input image to a four-dimensional plenoptic function.

23. The method of claim 19, further comprising:

refocusing the single image at different depth planes of the scene.

24. The method of claim 19, wherein the scene is dynamic, and further comprising:

converting the input image to a low spatial resolution video.

25. The method of claim 19, further comprising:

mapping angular, temporal or spatial variations in light field to individual pixels of the sensor.