System and method for calibrating a display system using a short throw camera

• US 9,860,494 B2
• Filed: 03/14/2014
• Issued: 01/02/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Scalable Display Technologies, Inc.
• Original Assignee: Scalable Display Technologies, Inc.
• Inventors: Johnson, Tyler M., Amaratunga, Kevin, Surati, Rajeev J., Timoner, Samson J.
• Primary Examiner(s): Le, Bao-Luan
• Application Number:
  US14/213,789
• Publication Number:
  US 20140313423A1
• Time in Patent Office: 1,390 Days
• CPC Class Codes: G03B 21/13 Projectors for producing special effects at the edges of picture, e.g. blurring
• CPC Class Codes: G03B 35/20 using two or more projectors
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G09G 2320/0693 Calibration of display systems
• CPC Class Codes: G09G 2360/145 the light originating from the display screen
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

This invention overcomes disadvantages of the prior art by providing a system and method that allows for the calibration of a display system using one or more cameras whose lenses are not modeled, or are modeled with limited accuracy, and are potentially incapable of imaging the entire projector system. The illustrative system and method generally relies on well-established models of the projectors, rather than well-established models of the cameras. Illustratively, the cameras serve mainly to map the projectors into one coordinate system, where the overlap between the projectors can be established very precisely using a camera, even if the model of the camera lens properties is not known, or known only minimally.

Claims

1. A method of calibrating a projector system having two or more projectors, with one or more cameras, comprising the steps of:

capturing, with the one or more cameras, one or more projected patterns from the projectors in an overlap region on a screen;

mapping projected pixels from each projector into a common coordinate system;

mapping the common coordinate system to the screen to find a homography between the two or more projectors;

measuring a difference between the projected pixels and a projector model generated from the homography and a projector distortion map to yield a displacement field; and

selectively applying the displacement field outside of the overlap region.

2. The method as set forth in claim 1 wherein the projectors are pre-calibrated.

3. The method as set forth in claim 1 wherein either of the mapping steps are formed using the projector model.

4. The method as set forth in claim 1 wherein the projector distortion map includes a model of distortions from an idealized model of a projector mapping.

5. The method as set forth in claim 1 wherein the displacement field is applied between the two or more projectors and the common coordinate system using edge-blending techniques.

6. The method as set forth in claim 1 wherein mapping the common coordinate system to the screen further comprises locating fiducials on the screen using the one or more cameras.

7. The method as set forth in claim 1 further comprising storing intensity and color information and balancing intensity and color using regions of the projector system that are imaged by the one or more cameras.

8. The method as set forth in claim 1, wherein the one or more cameras image a first portion of the entire screen, thereby defining a second portion of the entire screen not imaged by the one or more cameras.

9. The method as set forth in claim 3 wherein the projector model is pre-calibrated by measurement.

10. The method as set forth in claim 3 wherein the projectors are projecting onto a flat screen.

11. The method as set forth in claim 3 wherein the projectors are projecting onto a curved screen with known or parameterized geometry.

12. The method as set forth in claim 9 wherein the pre-calibrated measurement is deduced while the projector is part of the calibrated system.

13. The method as set forth in claim 9, wherein the measurement includes measuring data from the projectors in a controlled environment.

14. A system for calibrating a projector system having two or more projectors, with one or more cameras, comprising:

one or more cameras that capture images of one or more projected patterns from the projectors in an overlap region on a screen, and

a processor configured to;

map projected pixels from each projector into a common coordinate system;

map the common coordinate system to the screen to find a homography between the two or more projectors;

measure a difference between the projected pixels and a projector model generated from the homography and a projector distortion map to yield a displacement field; and

selectively apply the displacement field outside of the overlap region.

15. The system as set forth in claim 14, wherein the projectors are pre-calibrated.

16. The system as set forth in claim 14, wherein the projectors are pre-calibrated and the step of mapping to a common coordinate system includes using the projector model of the projector to surface mapping of the pre-calibrated projectors.

17. A method of calibrating a projector system having two or more projectors, with one or more cameras, comprising the steps of:

capturing, with the one or more cameras, one or more projected patterns from the projectors in an overlap region on a screen, free of requiring the one or more cameras to image the entire screen;

mapping projected pixels from each projector into a common coordinate system, the step of mapping being free of depending on a model of lens properties of the camera;

mapping the common coordinate system to the screen to find a homography between the two or more projectors, free of depending on a model of lens properties of the one or more cameras;

measuring a difference between the projected pixels and a projector model generated from the homography and a projector distortion map to yield a displacement field; and

selectively applying the displacement field outside of the overlap region.