Method and system for acquiring and displaying 3D light fields

• US 20070229653A1
• Filed: 04/04/2006
• Published: 10/04/2007
• Est. Priority Date: 04/04/2006
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Zwicker, Matthias, Matusik, Wojciech, Pfister, Hanspeter, Durand, Fredo
• Granted Patent: US 7,609,906 B2
• Time in Patent Office: Days
• US Class Current: 348/51
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence

Abstract

A method and system acquire and display light fields. A continuous light field is reconstructed from input samples of an input light field of a 3D scene acquired by cameras according to an acquisition parameterization. The continuous light is reparameterized according to a display parameterization and then prefiltering and sampled to produce output samples having the display parametrization. The output samples are displayed as an output light field using a 3D display device.

Claims

1. A computer implemented method for acquiring and displaying light fields, comprising, the steps of:

reconstructing, according to an acquisition parameterization, a continuous light field from input samples of an input light field of a three dimensional scene acquired by a plurality of cameras;

reparameterizing, according to a display parameterization, the continuous light field; and

prefiltering the reparameterized light field and sampling the prefiltered light field to produce output samples having the display parametrization, and displaying the output samples as an output light field using a three dimensional display device.

2. The method of claim 1, in which the prefiltering matches a Nyquist limit of a display bandwidth according to the display parameterization.

3. The method of claim 1, in which the three dimensional display device is an automultiscopic display device.

4. The method of claim 1, in which the input samples are in a form of sequences of images acquired by the plurality of cameras.

5. The method of claim 1, in which the plurality of cameras are arranged as a horizontal linear array.

6. The method of claim 1, in which the three dimensional display device includes a plurality of projectors, and a display screen includes a parallax-barrier mounted on a front side of a vertically oriented lenticular screen.

7. The method of claim 1, in which the display parameterization is defined in part by a t-plane plane of the parallax-barrier defining t coordinates, and a v-plane of the lenticular screen defining v coordinates, and a pixel spacing of the lenticular screen is Δ

v, a spacing of the gaps in the parallax-barrier is Δ

t, a separation between the lenticular screen and parallax-barrier is f and depth is z.

8. The method of claim 7, in which a ray of the output light field intersecting the t-plane at one location corresponds to one multi-view pixel, and each intersection of the ray with the v-plane is a view-dependent subpixel, and the a number of the multi-view pixels defines a spatial resolution and a number of view-dependent subpixels per multi-view pixel defines the angular resolution of the display parameterization.

9. The method of claim 7, in which a display bandwidth is limited according to

10. The method of claim 7, in which input coordinates of a particular camera and the focal plane are, respectively, t_in and v_in;

a distance from the t_in axis is z_in;

inter-sampling distances are Δ

t_in and Δ

v_in;

corresponding display coordinates are t_d, v_d, z_d, Δ

t_d, and Δ

v_d;

a distance between the t- and v-planes for both input light field and the output light is normalized to 1; and

the reparameterization uses a relation between the acquisition parametrization and the display parametrization defined by

11. The method of claim 11, in which an input sample of the input light field is ξ

_i,j, and the reconstructing is according to

12. The method of claim 11, in which a band-limited signal as a weighted sum of input samples uses a resampling filter ρ

and performed according to

13. The method of claim 12, in which p is

ρ

(v_d,t_d)=(r(M[·

])h)(v_d,t_d).

14. A system for acquiring and displaying light fields, comprising:

a plurality of cameras configured to acquire an input light field of a three dimensional scene;

means for reconstructing, according to an acquisition parameterization, a continuous light field from input samples of the input light field;

means for reparameterizing, according to a display parameterization, the continuous light field;

means for prefiltering the reparameterized light field and sampling the prefiltered light field to produce output samples having the display parametrization; and

a three dimensional display device configured to display the output samples as an output light field.