System and method for advanced lens geometry fitting for imaging devices

• US 9,807,359 B1
• Filed: 01/18/2017
• Issued: 10/31/2017
• Est. Priority Date: 11/11/2016
• Status: Active Grant

• Current Assignee: Christie Digital Systems USA Incorporated (Ushio Incorporated)
• Original Assignee: Christie Digital Systems USA Incorporated (Ushio Incorporated)
• Inventors: Moule, Kevin, Ben-Lavi, Eliezer Azi, Scott, Derek, Urquhart, Daniel Thomson
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US15/408,923
• Time in Patent Office: 286 Days
• Field of Search: 348744-747, 356601-603, 356606, 356610, 345633, 382154
• CPC Class Codes: G02B 13/0005 having F-Theta characteristic scanning systems employing movable or deformable optical elements G02B26/10
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 27/0012 Optical design, e.g. procedures, algorithms, optimisation routines
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 9/3102 using two-dimensional electronic spatial light modulators micromechanical modulators as such G02B26/0833; liquid crystal modulators as such G02F1/13
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

A system and method for advanced lens geometry fitting is provided for projecting images using a projector with a non-planar lens. An initial estimate of intrinsic parameters of the projector can be determined, for example from a list; an initial estimate of extrinsic parameters of the projector can be determined based on the initial estimate of intrinsic parameters. The intrinsic parameters and extrinsic parameters can be optimized together by starting with a simple lens model and iteratively adding nonlinear terms, an error function evaluated at each iteration, for example for each set of intrinsic parameters in the list and the corresponding extrinsic parameters. As the values of the error function converges the optimized intrinsic parameters and the optimized extrinsic parameters can be used to generate images that are projected by the projector.

Claims

1. A system comprising:

a computing device;

a projector comprising a non-planar lens, the projector configured for projection onto a object;

the computing device configured to;

using an initial model of the non-planar lens that relates a position of projected points on the object to pixel points at an image modulator of the projector, determine at least one initial estimate of intrinsic parameters of the projector using the initial model;

determine at least one initial estimate of extrinsic parameters of the projector using the at least one initial estimate of the intrinsic parameters;

iteratively evaluate an error function that includes a comparison between the pixel points and a transformation of the projected points to the pixel points, an initial transformation determined using the at least one initial estimate of the intrinsic parameters and the at least one initial estimate of the extrinsic parameters, and iterative transformations including one or more of;

varying one or more nonlinear terms used in the initial model of the non-planar lens, in one or more iterations;

increasing a number of the one or more nonlinear terms used in the initial model of the non-planar lens, in one or more of the iterations;

varying intrinsic parameters of the projector; and

varying extrinsic parameters of the projector;

when the error function, of one or more iterations, one or more of converges, reaches a minimum, and is less than a threshold value;

store current value of the intrinsic parameters of the projector and the extrinsic parameters of the projector as pose data; and

,

control the projector to project images onto the object based on the pose data.

2. The system of claim 1, wherein the computing device is further configured to determine at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a range of estimates for each of a plurality of the intrinsic parameters.

3. The system of claim 1, wherein the computing device is further configured to determine at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a set of discrete estimates of a plurality of the intrinsic parameters.

4. The system of claim 3, wherein the computing device is further configured to determine the at least one initial estimate of extrinsic parameters for each of elements of the set of discrete estimates of the plurality of the intrinsic parameters.

5. The system of claim 4, wherein the computing device is further configured to determine the at least one initial estimate of the extrinsic parameters, for each of the elements of the set of discrete estimates of the plurality of the intrinsic parameters, by extrapolating a line between a given projection point and a corresponding point at a curved image modulator that has a curvature determined from each of the elements of the set of discrete estimates of the plurality of the intrinsic parameters, to a pixel point on flat image modulator that shares a principal point with the curved image modulator.

6. The system of claim 5, wherein the computing device is further configured to determine the at least one initial estimate of the extrinsic parameters using a direct linear transformation algorithm using a set of pixel points on the flat image modulator.

7. The system of claim 5, wherein the computing device is further configured to iteratively evaluate the error function for each element of the set of discrete estimates of the plurality of the intrinsic parameters and corresponding estimates of the extrinsic parameters.

8. The system of claim 1, wherein the non-planar lens comprises one or more of a fisheye lens and an f-theta lens, and wherein the initial model of the non-planar lens comprises one or more of a fisheye lens model and an f-theta lens model.

9. The system of claim 1, wherein the initial model of the non-planar lens comprises one or more of:

zero non-linear terms, one non-linear term, and a plurality of non-linear terms.

10. A method comprising:

at a computing device in communication with a projector comprising a non-planar lens, the projector configured for projection onto a object;

using an initial model of the non-planar lens that relates a position of projected points on the object to pixel points at an image modulator of the projector, determining, at the computing device, at least one initial estimate of intrinsic parameters of the projector using the initial model;

determining, at the computing device, at least one initial estimate of extrinsic parameters of the projector using the at least one initial estimate of the intrinsic parameters;

iteratively evaluating, at the computing device, an error function that includes a comparison between the pixel points and a transformation of the projected points to the pixel points, an initial transformation determined using the at least one initial estimate of the intrinsic parameters and the at least one initial estimate of the extrinsic parameters, and iterative transformations including one or more of;

varying one or more nonlinear terms used in the initial model of the non-planar lens, in one or more iterations;

increasing a number of the one or more nonlinear terms used in the initial model of the non-planar lens in one or more of the iterations;

varying intrinsic parameters of the projector; and

varying extrinsic parameters of the projector;

when the error function, of one or more iterations, one or more of converges, reaches a minimum, and is less than a threshold value;

store current value of the intrinsic parameters of the projector and the extrinsic parameters of the projector as pose data; and

,

controlling, at the computing device, the projector to project images onto the object based on the pose data.

11. The method of claim 10, further comprising:

determining at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a range of estimates for each of a plurality of the intrinsic parameters.

12. The method of claim 10, further comprising:

determining at least one initial estimate of intrinsic parameters of the projector using the initial model by determining a set of discrete estimates of a plurality of the intrinsic parameters.

13. The method of claim 12, further comprising:

determining the at least one initial estimate of extrinsic parameters for each element of the set of discrete estimates of the plurality of the intrinsic parameters.

14. The method of claim 13, further comprising:

determining the at least one initial estimate of the extrinsic parameters by extrapolating a line between a given projection point and a corresponding point at a curved image modulator that has a curvature similar to that of the non-planar lens, to a pixel point on flat image modulator that shares a principal point with the curved image modulator.

15. The method of claim 14, further comprising:

determining the at least one initial estimate of the extrinsic parameters using a direct linear transformation algorithm using a set of pixel points on the flat image modulator.

16. The method of claim 13, further comprising:

iteratively evaluating the error function for each element of the set of discrete estimates of the plurality of the intrinsic parameters and corresponding estimates of the extrinsic parameters.

17. The method of claim 10, wherein the non-planar lens comprises one or more of a fisheye lens and an f-theta lens, and wherein the initial model of the non-planar lens comprises one or more of a fisheye lens model and an f-theta lens model.

18. The method of claim 10, wherein the initial model of the non-planar lens comprises one or more of:

zero non-linear terms, one non-linear term, and a plurality of non-linear terms.

19. A non-transitory computer-readable medium storing a computer program, wherein execution of the computer program is for:

at a computing device in communication with a projector comprising a non-planar lens, the projector configured for projection onto a object;

using an initial model of the non-planar lens that relates a position of projected points on the object to pixel points at an image modulator of the projector, determining, at the computing device, at least one initial estimate of intrinsic parameters of the projector using the initial model;

determining, at the computing device, at least one initial estimate of extrinsic parameters of the projector using the at least one initial estimate of the intrinsic parameters;

iteratively evaluating, at the computing device, an error function that includes a comparison between the pixel points and a transformation of the projected points to the pixel points, an initial transformation determined using the at least one initial estimate of the intrinsic parameters and the at least one initial estimate of the extrinsic parameters, and iterative transformations including one or more of;

varying one or more nonlinear terms used in the initial model, in one or more iterations;

increasing a number of the one or more nonlinear terms used in the initial model of the non-planar lens in one or more of the iterations;

varying intrinsic parameters of the projector; and

varying extrinsic parameters of the projector;

when the error function, of one or more iterations, one or more of converges, reaches a minimum, and is less than a threshold value;

store current value of the intrinsic parameters of the projector and the extrinsic parameters of the projector as pose data; and

,

controlling, at the computing device, the projector to project images onto the object based on the pose data.