Method and apparatus for calibration of multiple projector systems

• US 9,547,222 B2
• Filed: 02/06/2014
• Issued: 01/17/2017
• Est. Priority Date: 02/08/2013
• Status: Active Grant

• Current Assignee: University of South Australia (University of Adelaide)
• Original Assignee: University of South Australia (University of Adelaide)
• Inventors: Smith, Ross Travers, Webber, Guy, Sugimoto, Maki, Marner, Michael Robert, Thomas, Bruce Hunter
• Primary Examiner(s): Dowling, William C
• Application Number:
  US14/174,109
• Publication Number:
  US 20140226167A1
• Time in Patent Office: 1,076 Days
• Field of Search: 353/69, 353/70, 353/121, 353/122, 356/614
• US Class Current: 1/1
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G03B 21/14 Details
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Disclosed is a method for improving the calibration of multiple projector systems in Spatial Augmented Reality systems where multiple projectors are used to project images directly onto objects of interest. The methods and system described herein improve the calibration of multiple projector systems in order to improve the alignment and clarity of projected images by reducing ghosting that can occur with poorly aligned projectors. The system uses a planar photodiode and the projector is used to project a plurality of projection regions, such as scan lines, across the planar photodetector and calculating the position based on weighting measurements by the measured light intensity and projected images in SAR.

Claims

1. A method for estimating a location of a photodetector in a projector based two dimensional coordinate system, the projector projecting a two dimensional array of pixels and the photodetector comprising a sensing area, the method comprising:

projecting a plurality of projection regions from the projector, and taking a photodetector measurement for each projection region, wherein the projected area of each projection region on the photodetector is less than the sensing area of the photodetector; and

estimating the location of the photodetector in the projector based two dimensional coordinate system using a weighted mean of the locations of the projection regions, wherein the location of each projection region in the projector based two dimensional coordinate system is weighted by the photodetector measurement for the respective projection region.

2. The method as claimed in claim 1, wherein estimating the location of the photodetector further comprises the step of combining one or more of the plurality of projected regions into one or more scanning set, wherein a scanning line for each scanning set is defined based upon the locations of the one or more of the projected regions comprising the scanning set, and the combined projected area on the photodetector is less than the sensing area of the photodetector, and the photodetector measurements are combined to generate a photodetector set measurement.

3. The method as claimed in claim 2, wherein estimating the location of the photodetector further comprises the step of combining a plurality of scanning sets to form two or more scanning collections, wherein for each scanning collection, the scanning lines of the scanning sets forming the scanning collection are substantially parallel, and define a scanning direction line which is substantially orthogonal to the scanning lines.

4. The method as claimed in claim 3, wherein for each scanning collection, a measurement set location is defined for each scanning line in the scanning collection by the intersection of the scanning direction line with the respective scanning line.

5. The method as claimed in claim 3, wherein estimating the location of the photodetector in the projector based two dimensional coordinate system is performed using a weighted mean of the locations of the measurement set locations of a scanning collection, wherein the location of each measurement set location is weighted by the photodetector set measurement for the respective measurement set locations.

6. The method as claimed in claim 5, wherein the two or more scanning collections include a first scanning collection and a second scanning collection, and the scanning direction line of the first scanning collection is substantially parallel to a first axis of the projector based two dimensional coordinate system, and the scanning direction line of the second scanning collection is substantially parallel to a second axis of the projector based two dimensional coordinate system, and the coordinate of the photodetector on the first axis is estimated from the measurement set locations and photodetector set measurements of the first scanning collection, and the coordinate of the photodetector on the second axis is estimated from the measurement set locations and photodetector set measurements of the second scanning collection.

7. The method as claimed in claim 6, wherein each projection region is a line of pixels aligned with one of the axes of the two dimensional array of pixels and each line is a scanning set and each scanning collection is comprised of a plurality of projected lines separated by a scanning step in the scanning direction line.

8. The method as claimed in claim 7, wherein the coordinate of the photodetector on the i^th axis, where the i^th axis is the first axis or the second axis, is obtained using the equation:

9. The method as claimed in claim 7, further comprising a step of obtaining a coarse estimate of the photodetector location prior to the step of projecting a plurality of projection regions, and each scanning collection is obtained by projecting a plurality of lines over a range from a start pixel location to an end pixel location, wherein the start pixel location is a pixel location estimated from the coarse location to be before the edge of the photodetector and the end pixel location is a pixel location estimated from the coarse location to be after an opposite edge of the photodetector.

10. The method as claimed claim 1, further comprising the step of background correcting the photodetector measurements, wherein the step of background correcting the photodetector measurements comprises:

measuring one or more background light measurements when the photodetector is not illuminated by the projector to obtain a background estimate; and

subtracting the background estimate from each photodetector measurement.

11. The method as claimed claim 1, wherein the estimated location of the photodetector is the centroid of the sensing area.

12. The method as claimed in claim 1, wherein at least one of the plurality of projection regions partially illuminates the sensing area.

13. The method as claimed in claim 1, wherein the photodetector is a composite photodetector comprised of a plurality of light sensing elements, the projected area of each projection region on the photodetector is less than the sensing area of each light sensing element.

14. The method as claimed in claim 1, wherein the photodetector is a composite photodetector comprised of a plurality of light sensing elements, wherein the gap between each of the light sensing elements is less than 10% of the width of the projected area of each projection region on the photodetector.

15. A method for calibrating a multiple projector system to align a plurality of projectors, the method comprising:

receiving a physical location of at least one photodetector;

pairing each physical location with a location in a virtual model of the multiple projector system;

estimating the pixel based location of each the at least one photodetector for each of the plurality of projectors using the method as claimed in claim 1 for each of the plurality of projectors; and

aligning the plurality of projectors from the estimated pixel based location for each of the plurality of projectors.

16. The method as claimed in claim 15, wherein the step of aligning comprises adjusting at least one extrinsic parameter of at least one of the plurality of projectors.

17. The method as claimed in claim 16, wherein the at least one extrinsic parameter comprises a position and orientation of the projector in the virtual model of the multiple projector system.

18. The method as claimed in claim 17, wherein the calibration is performed continuously during use of the multiple projector system to maintain alignment of the plurality of photodetectors.

19. The method as claimed in claim 18, wherein the multiple projector system is a spatial augmented reality (SAR) system.

20. A non transitory processor readable medium comprising instructions for causing at least one processor to perform a method for estimating a location of a photodetector in a projector based two dimensional coordinate system, the projector projecting a two dimensional array of pixels and the photodetector comprising a sensing area, the method comprising:

projecting a plurality of projection regions from the projector, and taking a photodetector measurement for each projection region, wherein the projected area of each projection region on the photodetector is less than the sensing area of the photodetector; and

estimating the location of the photodetector in the projector based two dimensional coordinate system using a weighted mean of the locations of the projection regions, wherein the location of each projection region in the projector based two dimensional coordinate system is weighted by the photodetector measurement for the respective projection region.

21. The non-transitory processor readable medium as claimed in claim 20, further comprising instructions for causing at least one processor to perform a method for calibrating a multiple projector system to align a plurality of projectors, the method comprising:

receiving a physical location of at least one photodetector;

pairing each physical location with a location in a virtual model of the multiple projector system;

estimating the pixel based location of the at least one photodetector for each of the plurality of projectors using the method for estimating a location of a photodetector in a projector based two dimensional coordinate system for each of the plurality of projectors; and

aligning the plurality of projectors from the estimated pixel based location for each of the plurality of projectors.

22. A system for estimating a location of a photodetector in a projector based two dimensional coordinate system, the system comprising:

a projector wherein the projector projects a two dimensional array of pixels;

at least one photodetector comprising a sensing area; and

a computing apparatus comprising at least one processor and a memory, the memory comprising instructions to perform a method for estimating a location of each of the at least one photodetector in a projector based two dimensional coordinate system, the method comprising;

projecting a plurality of projection regions from the projector, and taking a photodetector measurement for each projection region, wherein the projected area of each projection region on the photodetector is less than the sensing area of the photodetector; and

estimating the location of the photodetector in the projector based two dimensional coordinate system using a weighted mean of the locations of the projection regions, wherein the location of each projection region in the projector based two dimensional coordinate system is weighted by the photodetector measurement for the respective projection region.

23. The system as claimed in claim 22, wherein the photodetector comprises:

a housing;

a light sensor comprising a sensing area;

a signal processing module for generating a measurement from the light sensor;

a power supply module; and

a communications module.

24. The system as claimed in claim 22, wherein the system further comprises a plurality of projectors wherein each projector projects a two dimensional array of pixels, and the memory further comprises instructions to perform a method for calibrating a multiple projector system to align a plurality of projectors, the method comprising:

receiving a physical location of each of the at least one photodetector;

pairing each physical location with a location in a virtual model of the multiple projector system;

estimating the pixel based location of each of the at least one photodetectors for each of the plurality of projectors using the method for estimating a location of a photodetector in a projector based two dimensional coordinate system for each of the plurality of projectors;

aligning the plurality of projectors from the estimated pixel based location for each of the plurality of projectors.