Projecting warped images onto curved surfaces

• US 6,793,350 B1
• Filed: 06/10/2003
• Issued: 09/21/2004
• Est. Priority Date: 03/21/2003
• Status: Expired due to Term

• Current Assignee: Trivale Technologies, LLC (Vector Capital Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Rao, Srinivasa G., Raskar, Ramesh, van Baar, Jeroen, Willwacher, Thomas H.
• Primary Examiner(s): Adams, Russell
• Assistant Examiner(s): Koval, Melissa J
• Application Number:
  US10/458,386
• Publication Number:
  US 20040184013A1
• Time in Patent Office: 469 Days
• Field of Search: 353/69, 353/94, 353/121, 353/122, 353/30, 353/48, 353/82, 353/89, 353/70, 345/421, 345/422, 345/419, 345/423, 345/424, 345/425, 345/426, 345/427, 352/47, 348/383, 382/154, 702/152
• US Class Current: 353/121
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 3/153 using elastic snapping
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A method projects one or more image onto a curved display surface. First, a predetermined structured light pattern is projected onto the display surface. A stereo pair of images is acquired of the projected images on the display surface. Then, a quadric transfer function between the predetermined images and the stereo pair of images, via the display surface, is determined. Thus, an arbitrary output image can be warped according to the quadric transfer function so that when it is projected onto the display surface it appears correct.

Claims

1. A method for projecting an image onto a display surface, comprising:

projecting a predetermined image onto a display surface;

acquiring a stereo pair of images of the predetermined image;

determining a quadric transfer function between the predetermined image and the stereo pair of images via the display surface;

warping an output image according to the quadric transfer function; and

projecting the warped output image onto the display surface.

2. The method of claim 1 wherein the display surface is quadric.

3. The method of claim 2 wherein the display surface is selected from a group consisting of a sphere, hemisphere, spheroid, dome, cylinder, cone, paraboloid, hyperboloid, hyperbolic parabaloid, and ellipsoid.

4. The method of claim 2 wherein the quadric surface is concave.

5. The method of claim 2 wherein the quadric surface is convex.

6. The method of claim 1 further comprising:

projecting a plurality of predetermined images onto the display surface from a plurality of viewpoints;

acquiring a stereo pair of input images of the plurality of predetermined images;

determining a quadric transfer function between each predetermined image and the stereo pair of input images via the display surface;

warping a plurality of output images according to the corresponding quadric transfer function; and

projecting the plurality of warped output images onto the display surface from the plurality of viewpoints.

7. The method of claim 6 wherein the surface is quadric.

8. The method of claim 6 wherein the plurality of warped output images overlap and appear as a single seamless image.

9. The method of claim 1 further comprising:

determining three-dimensional correspondences between features of the predetermined image and corresponding features in the stereo pair of images;

fitting a quadric surface to the three-dimensional correspondences;

determining a pose of the predetermined image with respect to the stereo pair of images; and

determining the quadric transfer function from the pose and the correspondences.

10. The method of claim 6 further comprising:

attenuating the plurality of warped output images according to blending weights before projecting.

11. The method of claim 1 wherein the predetermined image is a checkerboard pattern.

12. The method of claim 1 further comprising:

applying non-linear minimization to an objective function to obtain the quadric transfer function.

13. The method of claim 12 wherein the objective function takes a total squared transfer error for all pixels in predetermined image.

14. The method of claim 1 wherein the output image is rendered for an arbitrary virtual viewpoint.

15. The method of claim 10 further comprising:

cropping the plurality of attenuated and warped output images to an arbitrary view frustum.

16. The method of claim 1 wherein the quadric transfer function is determined parametrically.

17. The method of claim 1 wherein the warping is implemented with a vertex shader of a graphic processing unit.

18. A system for projecting an image onto a display surface, comprising:

a projector configured to project a predetermined image onto a display surface;

a camera configured to acquire a stereo pair of images of the predetermined image;

means for determining a quadric transfer function between the predetermined image and the stereo pair of images via the display surface;

means for warping an output image according to the quadric transfer function; and

means for projecting the warped output image onto the display surface.