Actively Addressable Aperture Light Field Camera

• US 20110128412A1
• Filed: 11/23/2010
• Published: 06/02/2011
• Est. Priority Date: 11/25/2009
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Milnes, Thomas B., Hart, Douglas P.
• Granted Patent: US 8,497,934 B2
• Time in Patent Office: Days
• US Class Current: 348/231.99
• CPC Class Codes: G03B 35/04 with movement of beam-selecting members in a system defining two or more viewpoints
• CPC Class Codes: H04N 13/211 using temporal multiplexing
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems

Abstract

An actively addressable aperture disposed ahead of the rear surface of a camera lens allows the camera to capture a full-resolution, five-dimensional (5D) light field that describes every possible view from every possible angle of the scene being imaged. Shifting the aperture over the entire aperture plane and acquiring an image at each step yields a 2D grid of 2D images of the scene, otherwise known as a 4D parameterized light field. Estimating the 3D depth of the objects in the imaged scene yields a 3D model with 2D surface irradiance patterns, which is the full, non-parameterized 5D light field. The 5D light field can be used to display perspective changes in a way that mimics cognitive processing of the same scene or object. 5D light fields can also be used to create high-precision, 3D depth maps suitable for 3D movies, interactive displays, machine vision, and other applications.

Claims

1. A camera, comprising:

a lens arranged to produce an image of an object in an image plane located a distance L from the lens;

an actively addressable aperture defined by a spatial light modulator (SLM) disposed between the object and the image plane, the SLM including elements spaced at a pitch of Λ

;

a detector disposed in the image plane and configured to sense two-dimensional images transmitted or reflected through the actively addressable aperture, the detector having an active area with a maximum transverse dimension of about D<

2 Lλ

/Λ

, where λ

is a wavelength of the two-dimensional images sensed by the detector; and

a processor operably coupled to the SLM and to the detector, the processor configured to vary at least one of a position, size, and shape of the actively addressable aperture and to generate a representation of a four-dimensional light field associated with the object based on the two-dimensional images sensed by the detector.

2. A camera as in claim 1 wherein the lens has a focal length F that is within a range of distances from about L to about 2 L.

3. A camera as in claim 1 further including:

a stop configured to limit the angular field of view of the lens to less than arcsin(λ

/Λ

).

4. A camera as in claim 1 further including:

a stop configured to limit the angle of incidence of light onto the SLM to within a range of angles from about 0 degrees to about 20 degrees about an axis perpendicular to a plane defined by the actively addressable aperture.

5. A camera as in claim 4 further including:

a relay lens disposed between the object and the lens and configured to produce another image of the object at an intermediate plane between the object and the image plane, and wherein the stop is disposed in the intermediate plane.

6. A camera as in claim 1 wherein the lens includes a first optical element disposed between the SLM and the object and a second optical element disposed between the SLM and the image plane.

7. A camera as in claim 6 wherein at least of the first and second optical elements is a Fresnel lens.

8. A camera as in claim 6 further including:

a relay lens disposed between the object and the first lens element, wherein the relay lens is configured to produce another image of the object at an intermediate plane between the object and the image plane.

9. A camera as in claim 1 wherein the SLM is bonded to the lens.

10. A camera as in claim 1 wherein the elements in the SLM are transmissive elements configured to define the actively addressable aperture by selectively transmitting incident light in response to a signal from the processor.

11. A camera as in claim 10 wherein at least some of the transmissive elements are further configured to define an edge of the actively addressable aperture by selectively partially transmitting light in response to a signal from the processor.

12. A camera as in claim 1 wherein the elements in the SLM are reflective elements configured to define the actively addressable aperture by selectively reflecting incident light in a desired direction in response to a signal from the processor.

13. A camera as in claim 12 wherein at least some of the reflective elements are further configured to only partially reflect at least some of the light from the object.

14. A camera as in claim 1 wherein the processor is further configured to vary at least one of a position, size, and shape of the actively addressable aperture in response to one or more of the two-dimensional images sensed by the detector.

15. A camera as in claim 1 wherein the processor is further configured to vary the actively addressable aperture according to a predetermined pattern.

16. A camera as in claim 1 wherein the processor is further configured to produce a representation of a five-dimensional light field associated with the object based on the two-dimensional images sensed by the detector

17. A camera as in claim 1, further including:

memory operably coupled to the detector and configured to store plural sets of data, each set of data indicative of a two-dimensional image corresponding to a particular actively addressable aperture position, size, or shape.

18. A camera as in claim 1 in combination with a display configured to render an image based on an orientation of the object relative to the camera.

19. A method of acquiring a light field associated with an object, comprising:

projecting two-dimensional images of an object to an image plane via a lens located a distance L from the image plane;

spatially modulating the light field with an actively addressable aperture defined by a spatial light modulator (SLM) disposed between the object and the image plane, the SLM including elements spaced at a pitch of Λ

;

detecting a first two-dimensional image with a detector disposed in the image plane, the detector having an active area with a maximum transverse dimension of about D<

2 Lλ

/Λ

, where λ

is a wavelength of the two-dimensional images sensed by the detector;

varying at least one of position, size, and shape of the actively addressable aperture;

detecting a second two-dimensional image with the detector; and

processing data indicative of the detected modulated light fields to generate a representation of a four-dimensional light field associated with the object based on the two-dimensional images sensed by the detector.

20. A method as in claim 19 wherein the lens has a focal length F that is within a range of distances from about L to about 2 L.

21. A method as in claim 19 further including:

limiting the angular field of view of the lens to less than arcsin(λ

/Λ

).

22. A method as in claim 19 further including:

limiting the angle of incidence of light onto the SLM to within a range of angles from about 0 degrees to about 20 degrees about an axis perpendicular to a plane defined by the actively addressable aperture.

23. A method as in claim 22 further including:

forming another image of the object at an intermediate plane between the object and the image plane, and wherein limiting the angle of incidence of light onto the SLM includes blocking light with a stop disposed in the intermediate plane.

24. A method as in claim 19 wherein the lens includes a first optical element disposed between the SLM and the object and a second optical element disposed between the SLM and the image plane.

25. A method as in claim 24 wherein at least one of the first and second optical elements is a Fresnel lens.

26. A method as in claim 24 further including:

forming another image of the object at an intermediate plane between the object and the first lens element.

27. A method as in claim 19 wherein the SLM is bonded to the lens.

28. A method as in claim 19 wherein spatially modulating the light field with the actively addressable aperture includes selectively transmitting the light field in response to a signal from the processor.

29. A method as in claim 28 wherein spatially modulating the light field with the actively addressable aperture further includes selectively partially transmitting at least a portion of the light field.

30. A method as in claim 19 wherein spatially modulating the light field with the actively addressable aperture includes selectively reflecting the light field in a desired direction in response to a signal from the processor.

31. A method as in claim 30 wherein spatially modulating the light field with the actively addressable aperture further includes only partially reflecting at least a portion the light field.

32. A method as in claim 19 wherein varying at least one of position, size, and shape of the actively addressable aperture occurs in response to one or more of the two-dimensional images sensed by the detector.

33. A method as in claim 19 wherein varying at least one of position, size, and shape of the actively addressable aperture occurs according to a predetermined pattern.

34. A method as in claim 19 further including:

producing a representation of a five-dimensional light field associated with the object based on the two-dimensional images sensed by the detector.

35. A method as in claim 19 further including:

rendering an image on a display based on the representation of the four-dimensional light field associated with the object.