Multi-projector geometric calibration

• US 7,306,341 B2
• Filed: 02/28/2005
• Issued: 12/11/2007
• Est. Priority Date: 02/28/2005
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chang, Nelson Liang An
• Primary Examiner(s): Lee, Diane I.
• Assistant Examiner(s): Cruz, Magda
• Application Number:
  US11/068,195
• Publication Number:
  US 20060192925A1
• Time in Patent Office: 1,016 Days
• Field of Search: 353/94, 353/5, 353/11, 353/30, 356/603, 356/391
• US Class Current: 353/94
• CPC Class Codes: G03B 21/14 Details
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

In one aspect, a first sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a first projection plane is projected onto a scene. A second sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a second projection plane is projected onto the scene. Light patterns corresponding to the first and second sequences of patterns of light symbols reflected from the scene are captured at a capture plane. Captured light symbol sequence codes are determined from the captured light patterns. A correspondence mapping of the first projector plane and the second projector plane with respect to a reference coordinate system is generated based at least in part on correspondence between the captured light symbol sequence codes and the first and second sequences of light symbol patterns.

Claims

1. A method of multi-projector correspondence mapping, comprising:

projecting onto a scene a first sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a first projection plane;

projecting onto the scene a second sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a second projection plane;

capturing at a capture plane light patterns corresponding to the first and second sequences of patterns of light symbols reflected from the scene;

determining captured light symbol sequence codes from the captured light patterns; and

generating a correspondence mapping of the first projection plane and the second projection plane with respect to a reference coordinate system based at least in part on correspondence between the captured light symbol sequence codes and the first and second sequences of light symbol patterns.

2. The method of claim 1, wherein the first and second sequences of light symbol patterns are projected concurrently.

3. The method of claim 1, wherein the first and second sequences of light symbol patterns are projected consecutively.

4. The method of claim 1, wherein the first sequence of light symbol patterns is identical to the second sequence of light symbol patterns.

5. The method of claim 1, wherein among the projected light symbol patterns are light patterns respectively comprising different spatial variations of light.

6. The method of claim 1, wherein the generating comprises mapping corner points of light symbols captured in the capture plane to corresponding corner points of light symbols in the first and second projection planes.

7. The method of claim 1, wherein the generating comprises determining a point-to-point correspondence mapping between regions in the first projection plane and corresponding regions of the second projection plane.

8. The method of claim 7, wherein the generating comprises fitting the point-to-point correspondence mapping to a model of the scene.

9. The method of claim 1, wherein the scene is modeled by a planar surface and the correspondence mapping corresponds to a planar perspective transformation H_k=[h_hk] such that points (u,v) in the reference coordinate system relate to coordinates (x_k,y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=(\frac{h_k}{}$

⁢

1 ⁢

u + h k ⁢

⁢

2 ⁢

v + h k ⁢

⁢

3 h k ⁢

⁢

7 ⁢

u + h k ⁢

⁢

8 ⁢

v + h k ⁢

⁢

9 , h k ⁢

⁢

4 ⁢

u + h k ⁢

⁢

5 ⁢

v + h k ⁢

⁢

6 h k ⁢

⁢

7 ⁢

u + h k ⁢

⁢

8 ⁢

v + h k ⁢

⁢

9 ) .

10. The method of claim 1, wherein the scene is modeled by a curved surface, and the correspondence mapping corresponds to a transformation H_k=[h_hk] such that points (u,v) in the reference coordinate system relate to coordinates (x_k,y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=(\frac{h_k}{}$

⁢

1 ⁢

u 2 + h k ⁢

⁢

2 ⁢

v 2 + h k ⁢

⁢

3 ⁢

uv + h k ⁢

⁢

4 ⁢

u + h k ⁢

⁢

5 ⁢

v + h k ⁢

⁢

6 h k ⁢

⁢

7 ⁢

u 2 + h k ⁢

⁢

8 ⁢

v 2 + h k ⁢

⁢

9 ⁢

uv + h k ⁢

⁢

10 ⁢

u + h k ⁢

⁢

11 ⁢

v + h k ⁢

⁢

12 , h k ⁢

⁢

13 ⁢

u 2 + h k ⁢

⁢

14 ⁢

v 2 + h k ⁢

⁢

15 ⁢

uv + h k ⁢

⁢

16 ⁢

u + h k ⁢

⁢

17 ⁢

v + h k ⁢

⁢

18 h k ⁢

⁢

19 ⁢

u 2 + h k ⁢

⁢

20 ⁢

v 2 + h k ⁢

⁢

21 ⁢

uv + h k ⁢

⁢

22 ⁢

u + h k ⁢

⁢

23 ⁢

v + h k ⁢

⁢

24 ) .

11. The method of claim 1, wherein the scene is modeled by a general three-dimensional surface, and the correspondence mapping corresponds to a transformation H_k=[hklm] such that points (u,v) in the reference coordinate system relate to coordinates (x_k,y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=\sum _{}^{}$

l = 0 L ⁢

∑

m = 0 M ⁢

h klm ⁢

u l ⁢

v m

where L and M are integers.

12. The method of claim 1, further comprising, combining images projected from the first and second projection planes based at least in part on the correspondence mapping.

13. The method of claim 12, further comprising, before the combining, warping coordinates of the first projection plane to corresponding coordinates of the second projection plane based at least in part on the correspondence mapping.

14. The method of claim 13, further comprising storing the warped coordinates in a lookup table.

15. A machine-readable medium storing machine-readable instructions for causing a machine to perform operations comprising:

projecting onto a scene a first sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a first projection plane;

projecting onto the scene a second sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a second projection plane;

capturing at a capture plane light patterns corresponding to the first and second sequences of patterns of light symbols reflected from the scene;

determining captured light symbol sequence codes from the captured light patterns; and

generating a correspondence mapping of the first projection plane and the second projection plane with respect to a reference coordinate system based at least in part on correspondence between the captured light symbol sequence codes and the first and second sequences of light symbol patterns.

16. The machine-readable medium of claim 15, wherein the machine-readable instructions cause the machine to perform operations comprising determining a point-to-point correspondence mapping between regions in the first projection plane and corresponding regions of the second projection plane.

17. The machine-readable medium of claim 16, wherein the machine-readable instructions cause the machine to perform operations comprising fitting the point-to-point correspondence mapping to a model of the scene.

18. The machine-readable medium of claim 15, wherein the scene is modeled by a planar surface, and the machine-readable instructions cause the machine to perform operations comprising generating a correspondence mapping corresponding to a planar perspective transformation H_k=[h_ki] such that points (u,v) in the reference coordinate system relate to coordinates (x_k,y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=(\frac{h_k}{}$

⁢

1 ⁢

u + h k ⁢

⁢

2 ⁢

v + h k ⁢

⁢

3 h k ⁢

⁢

7 ⁢

u + h k ⁢

⁢

8 ⁢

v + h k ⁢

⁢

9 , h k ⁢

⁢

4 ⁢

u + h k ⁢

⁢

5 ⁢

v + h k ⁢

⁢

6 h k ⁢

⁢

7 ⁢

u + h k ⁢

⁢

8 ⁢

v + h k ⁢

⁢

9 ) .

19. The machine-readable medium of claim 15, wherein the scene is modeled by a curved surface, and the machine-readable instructions cause the machine to perform operations comprising generating a correspondence mapping corresponding to a transformation H_k=[h_ki] such that points (u,v) in the reference coordinate system relate to coordinates (x_k, y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=(\frac{h_k}{}$

⁢

1 ⁢

u 2 + h k ⁢

⁢

2 ⁢

v 2 + h k ⁢

⁢

3 ⁢

uv + h k ⁢

⁢

4 ⁢

u + h k ⁢

⁢

5 ⁢

v + h k ⁢

⁢

6 h k ⁢

⁢

7 ⁢

u 2 + h k ⁢

⁢

8 ⁢

v 2 + h k ⁢

⁢

9 ⁢

uv + h k ⁢

⁢

10 ⁢

u + h k ⁢

⁢

11 ⁢

v + h k ⁢

⁢

12 , h k ⁢

⁢

13 ⁢

u 2 + h k ⁢

⁢

14 ⁢

v 2 + h k ⁢

⁢

15 ⁢

uv + h k ⁢

⁢

16 ⁢

u + h k ⁢

⁢

17 ⁢

v + h k ⁢

⁢

18 h k ⁢

⁢

19 ⁢

u 2 + h k ⁢

⁢

20 ⁢

v 2 + h k ⁢

⁢

21 ⁢

uv + h k ⁢

⁢

22 ⁢

u + h k ⁢

⁢

23 ⁢

v + h k ⁢

⁢

24 ) .

20. The machine-readable medium of claim 15, wherein the scene is modeled by a general three-dimensional surface, and the correspondence mapping corresponds to a transformation H_k=[h_klm] such that points (u,v) in the reference coordinate system relate to coordinates (x_k,y_k) in the k^th projection plane by

$\left(x_k,y_k\right)=\sum _{}^{}$

l = 0 L ⁢

∑

m = 0 M ⁢

h klm ⁢

u l ⁢

v m

where L and M are integers.

21. A multi-projector system, comprising:

a first projector;

a second projector;

an imaging device; and

a processing system operable to

control the first projector to project onto a scene a first sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a first projection plane,

control the second projector to project onto the scene a second sequence of patterns of light symbols that spatio-temporally encode two-dimensional position information in a second projection plane,

control the imaging device to capture at a capture plane light patterns corresponding to the first and second sequences of patterns of light symbols reflected from the scene,

determine captured light symbol sequence codes from the captured light patterns, and

generate a correspondence mapping of the first projection plane and the second projection plane with respect to a reference coordinate system based at least in part on correspondence between the captured light symbol sequence codes and the first and second sequences of light symbol patterns.

22. The system of claim 21, wherein the processing system is operable to control the first and second projectors to project light patterns respectively comprising different spatial variations of light.

23. The system of claim 21, wherein the processing system is operable to determine a point-to-point correspondence mapping between regions in the first projection plane and corresponding regions of the second projection plane.

24. The system of claim 23, wherein the processing system is operable to fit the point-to-point correspondence mapping to a model of the scene.

25. The system of claim 21, wherein the processing system comprises a server processing unit controlling operation of the imaging device and the first projector, and a client processing unit controlling operation of the second projector,

the client processing unit being operable to generate a client-generated correspondence mapping between the capture plane and the second projection plane and to pass the client-generated correspondence mapping to the server processing unit, and

the server processing unit being operable to generate the correspondence mapping of the first projection plane and the second projection plane with respect to the reference coordinate system based at least in part on the client-generated correspondence mapping.

26. The system of claim 21, wherein the processing system comprises a first processing unit controlling operation of the imaging device, a second processing unit controlling operation of the first projector, and a third processing unit controlling operation of the second projector,

the first processing unit passing image data corresponding to images captured by the imaging device to each of the second and third processing units,

the second processing unit being operable to generate a first correspondence mapping between the capture plane and the first projection plane based on the image data received from the first processing unit, and

the third processing unit being operable to generate a second correspondence mapping between the capture plane and the second projection plane based on the image data received from the first processing unit.