Method and system for calibrating projectors to arbitrarily shaped surfaces with discrete optical sensors mounted at the surfaces

• US 20050030486A1
• Filed: 08/06/2003
• Published: 02/10/2005
• Est. Priority Date: 08/06/2003
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Information Technology Center America, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Information Technology Center America, Inc. (Mitsubishi Electric Corporation)
• Inventors: Raskar, Ramesh, Lee, Johnny Chung, Dietz, Paul H., Maynes-Aminzade, Daniel
• Granted Patent: US 7,001,023 B2
• Time in Patent Office: Days
• US Class Current: 353/69
• CPC Class Codes: H04N 17/04 for receivers
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A system determines correspondence between locations on a display surface and pixels in an output image of a projector. The display surface can have an arbitrary shape and pose. Locations with know coordinates are identified on the display surface. Each location is optically coupled to a photo sensor by an optical fiber installed in a throughhole in the surface. Known calibration patterns are projected, while sensing directly an intensity of light at each location for each calibration pattern. The intensities are used to determine correspondences between the of locations and pixels in an output image of the projector so that projected images can warped to conform to the display surface.

Claims

1. A method for determining correspondence between locations on a display surface having an arbitrary shape and pixels in an output image of a projector, comprising:

projecting a set of known calibration patterns onto the display surface;

sensing directly an intensity of light at each of a plurality of locations on the display surface for each calibration pattern, there being one discrete optical sensor associated with each location; and

correlating the intensities at the locations to determine correspondences between the plurality of locations and pixels in an output image of the projector.

2. The method of claim 1, in which each location has known coordinates.

3. The method of claim 1, in which the calibration patterns are in a form of Gray codes.

4. The method of claim 1, in which the correspondences are used to determine parameters of the projector.

5. The method of claim 4, in which the parameters include internal and external parameters and non-linear distortions of the projector.

6. The method of claim 1, further comprising:

warping an input image to the projector according to the correspondences; and

projecting the warped input image on the display surface to appear undistorted.

7. The method of claim 1, in which the projector is casually aligned with the planar display surface.

8. The method of claim 1, in which the display surface is planar.

9. The method of claim 1, in which the display surface is quadric.

10. The method of claim 1, in which a viewer and the projector are on a same side of the display surface.

11. The method of claim 8, in which the display surface is planar and a number of locations is four.

12. The method of claim 1, in which the optical sensor is a photo transistor.

13. The method of claim 12, in which the optical sensor is coupled to the corresponding location by an optical fiber.

14. The method of claim 1, in which the intensity is quantized to zero or one.

15. The method of claim 1, further comprising:

warping a sequence of input images to the projector according to the correspondences; and

projecting the warped sequence of input image on the display surface to appear undistorted as a video.

17. The method of claim 1, in which the display surface is an external surface of a 3D model of a real-world object.

18. The method of claim 1, in which the display surface includes a backdrop on which the 3D model is placed.

19. The method of claim 1, in which the light is infrared.

20. The method of claim 1, in which each calibration image is projected as a pair, a second image of the pair being an inverse of the calibration image.

21. The method of claim 1, in which the correspondences are used to relocate the projector.

22. The method of claim 1, in which the correspondences are used to deform the display surface.

16. The method of clam 15, in which the display surface and the projector are moving with respect to each other while determining the correspondences, warping the sequence of images, and projecting the warped sequence of input images.

23. A system for determining correspondence between locations on a display surface having an arbitrary shape and pixels in an output image of a projector, comprising:

a display surface having a plurality of locations with known coordinates;

a plurality of known calibration patterns;

means for sensing directly an intensity of light at each of the plurality of locations on the display surface for each calibration pattern; and

means for correlating the intensities at the locations to determine correspondences between the plurality of locations and pixels in an output image of the projector.

24. The system of claim 24, in which each location is optically coupled to a discrete photo sensor by an optical fiber.

25. The system of claim 24, in which the optical fiber is located in a throughhole in the display surface.

26. A method for determining correspondence between locations on a display surface having an arbitrary shape and pixels in an output image of a projector, comprising:

sensing directly an intensity of light at each of a plurality of locations on a display surface for each of a plurality of calibration patterns projected on the display surface, there being one discrete optical sensor associated with each location; and

correlating the intensities at the locations to determine correspondences between the plurality of locations and pixels in an output image of the projector.