Bi-Directional Screen

• US 20110019056A1
• Filed: 11/20/2009
• Published: 01/27/2011
• Est. Priority Date: 07/26/2009
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Lanman, Douglas, Holtzman, Henry, Hirsch, Matthew, Raskar, Ramesh
• Granted Patent: US 8,654,234 B2
• Time in Patent Office: Days
• US Class Current: 348/333.01
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 5/2226 Determination of depth image, e.g. for foreground/background separation determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10

Abstract

A bidirectional screen alternately switches between a display mode showing conventional graphics and a capture mode in which the LCD backlight is disabled and the LCD displays a pinhole array or a tiled-broadband code. A large-format image sensor is placed behind the liquid crystal layer. Together, the image sensor and LCD function as a mask-based light field camera, capturing an array of images equivalent to that produced by an array of cameras spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points from focus.

Claims

1. Apparatus comprising, in combination:

at least one screen for displaying images,

at least one light-sensing device for measuring light that comes from at least one object and that falls upon, or passes through, said screen; and

a processing unit for using said measurements to calculate the direction-dependent intensity of at least some of the light rays that comprise said light.

2. Apparatus as set forth in claim 1, wherein at least one of said screens comprises a liquid crystal display.

3. Apparatus as set forth in claim 1, wherein at least one said light-sensing device is comprised of an array of optical sensors.

4. Apparatus as set forth in claim 1, wherein at least one of said screens is further adapted for displaying said images perceptible to a human viewer part of the time and for displaying at least one spatial light modulator pattern part of the time.

5. Apparatus as set forth in claim 4, wherein said at least one screen is further adapted for displaying a spatial light modulator pattern comprised of tiled broadband code.

6. Apparatus as set forth in claim 4, further adapted for adjusting at least one said spatial light modulator pattern in real time in response to measured properties of the imaged scene.

7. Apparatus as set forth in claim 6, further adapted for reconfiguring the spatial light modulator pattern to locally optimize the tradeoff of spatial resolution vs. angular resolution for imaging an object.

8. Apparatus as set forth in claim 4, wherein at least one said screen comprises an interface for human interaction with a computer or phone.

9. Apparatus as set forth in claim 1, wherein at least one surface of at least one said screen is flat.

10. Apparatus as set forth in claim 1, wherein at least one surface of at least one said screen is curved.

11. Apparatus as set forth in claim 1, wherein said screen or screens are on or comprise at least part of at least two surfaces of a polyhedron.

12. Apparatus as set forth in claim 1, further comprising a spatial light attenuation mask or a device for displaying said mask, which mask is transparent in the humanly-visible light spectrum but not transparent in at least one other wavelength.

13. Lensless apparatus for capturing a light field, comprising:

at least one screen

for displaying, at some times, a pattern that spatially attenuates light from external objects that passes through said screen and

for displaying, at other times, images other than said pattern, and

an array of sensors for measuring said light, wherein at least some of said sensors are displaced from the display plane, and

a processing unit for using said measurements to calculate the direction of at least some of the light rays that comprise said light.

14. Apparatus as set forth in claim 13, wherein said light attenuation pattern is adjustable.

15. Apparatus as set forth in claim 13, wherein at least one said screen comprises a liquid crystal display.

16. Apparatus as set forth in claim 13, wherein said images other than said pattern are perceptible to a human viewer.

17. A system comprising, in combination:

at least one screen, each said screen for displaying a spatial light attenuator at some times and for displaying images other than said light attenuator at other times, and

at least one light-measuring device,

for measuring light that passes through said screen from external objects and that is attenuated by said light attenuator, and

for outputting data indicative of said measurements for transmission to a processing unit for calculating vectors of at least some light rays comprising said light.

18. A system as set forth in claim 17, wherein said light attenuator is adjustable in real time.

19. A system as set forth in claim 17, further comprising a diffuser, which is adapted to be substantially transparent during at least part of time when images are being captured and to be diffuse during at least part of the time when images are displayed for perception by a human viewer.

20. A system as set forth in claim 17, wherein at least part of at least one said screen is displaced by at least 1.5 centimeters from at least part of at least one said light-measuring device.

21. Apparatus comprising, in combination:

at least one light-sensitive device for

measuring light that is from external objects, that falls upon, or passes through, a surface of said apparatus, and that is attenuated by a spatial light modulation pattern, and

outputting data indicative of said light measurements for transmission to a processing unit, which processing unit is adapted for calculating, based on said measurements, a light field of four or more dimensions and is further adapted for calculating the depth in an imaged scene of at least one said external object; and

at least one display device for displaying images on or through said surface.

22. Apparatus as set forth in claim 21, further comprising a liquid crystal display for displaying said light modulation pattern.

23. A method comprised of the following steps:

displaying images perceptible by a human viewer on or through a surface,

attenuating light from external objects that falls upon or passes through said surface,

measuring at least some of said light, and

using said light measurements to calculate multi-view orthographic imagery of at least one said external object and to calculate the position in three dimensions of said object.

24. A method as set forth in claim 23, wherein said spatial light modulator is in the pattern of tiled broadband code.