System and method for automated test-pattern-free projection calibration

• US 9,915,857 B2
• Filed: 12/09/2013
• Issued: 03/13/2018
• Est. Priority Date: 12/09/2013
• Status: Active Grant

• Current Assignee: Geo Semiconductor Incorporated
• Original Assignee: Geo Semiconductor Incorporated
• Inventors: Bassi, Zorawar S., Jegatheswaran, Thushyanthan
• Primary Examiner(s): Porta, David
• Assistant Examiner(s): Malevic, Djura
• Application Number:
  US14/100,524
• Publication Number:
  US 20150160539A1
• Time in Patent Office: 1,555 Days
• Field of Search: 25036307, 382286, 382276, 382289, 382215, 382255, 382275, 382293
• CPC Class Codes: G03B 21/147 Optical correction of image distortions, e.g. keystone
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

Systems and methods for calibrating an image projection system, using image capture devices are disclosed. In particular, Wide-Angle (WA) and Ultra Wide-Angle (UWA) lenses can be utilized to eliminate need for traditional test patterns and tedious alignment settings. That is particularly useful for automatic calibration of projection sources that may require frequent adjustments on the field. Geometric mapping techniques combined with image analytics allow identifying parallel lines in an image captured from a scene. Coordinates of vanishing points corresponding to pairs of parallel lines provide sufficient information to determine orientation of the image projection system in space.

Claims

1. A method to correct for rotational distortions of an image projected from an image projector on a viewing surface without using a test pattern, said method comprising:

acquiring an image of a scene containing the viewing surface, using at least one image capture device equipped with an Ultra Wide-Angle (UWA) lens with a field of view of about 180°

or larger and having a lens mapping transformation, with a known orientation relative to the image projector;

using at least one image processing unit to;

identify at least a first pair of parallel lines in the image in the UWA lens space;

identify at least a second pair of parallel lines, orthogonal to the at least first pair of parallel lines, in the image in the UWA lens space;

obtain coordinates of vanishing points corresponding to the at least first and the at least second pair of parallel lines;

compute angles of rotation of the at least one image capture device with respect to a frame of reference, using the coordinates of the vanishing points;

construct a correction transformation using the rotational angles and the orientation of the at least one image capture device relative to the image projector;

wherein the correction transformation for the rotational angles is independent of a position of lens used to capture the image, and

apply the correction transformation to correct for the rotational distortion of the image, wherein the correction transformation does not use a test pattern for correction.

2. The method of claim 1, wherein a portion of the image containing the identified lines are mapped to a projective plane, using the lens mapping transformation, to facilitate obtaining coordinates of the vanishing points.

3. The method of claim 1, wherein the angles of rotation are represented as a rotation matrix.

4. The method of claim 1, wherein the correction transformation is applied to an output image in a forward direction.

5. The method of claim 1, wherein inverse of the correction transformation is determined and applied to pre-distort an input image, such that projected output image appear substantially rotation free.

6. The method of claim 1, wherein spatial orientation of the image projector changes in time and the correction is dynamically performed.

7. The method of claim 1, wherein the viewing surface is virtual.

8. The method of claim 1, wherein one or more of the at least one image capture device is integrated with the image projector.

9. The method of claim 1, wherein one or more of the at least one image capture device is physically attached to the image projector.

10. The method of claim 1, wherein the image projector is part of a Heads-Up-Display (HUD) assembly.

11. The method of claim 1, wherein the image projector is used in a front projection configuration.

12. The method of claim 1, wherein the image projector is used in a rear projection configuration.

13. The method of claim 1, wherein said correction transformation is concatenated with correction components associated with at least one of optical distortions and geometry of the viewing surface.

14. A calibration system for use with an image projection set-up having an image source and a viewing surface, said system comprising:

at least one image capture device equipped with an Ultra Wide-Angle (UWA) lens with a field of view of about 180°

or larger and having a corresponding lens transformation, with a known orientation relative to the image projector, to acquire an image of a scene; and

at least one image processing unit to;

detect a plurality of objects in the image that are presumed vertical and horizontal objects in the scene;

identify at least a first pair of parallel lines in one or more of the plurality of the horizontal objects, and least a second pair of parallel lines orthogonal to the at least first pair of parallel lines in one or more of the plurality of the vertical objects, in the image in the UWA lens space;

obtain coordinates of vanishing points corresponding to the at least first and the at least second pair of parallel lines;

compute rotational angles of the image capture device with respect to a frame of reference, using the coordinates of the vanishing points; and

construct a correction transformation using the rotational angles and the orientation of the at least one image capture device relative to the image projector; and

apply the correction transformation to correct for the rotational distortion of the acquired image.

15. The calibration system of claim 14, further comprising means to map a portion of the image containing the identified parallel lines to a projective plane, using the lens mapping transformation, to facilitate obtaining the vanishing points.

16. The calibration system of claim 14, wherein the viewing surface is virtual.

17. The calibration system of claim 14, wherein one or more of the at least one image capture device is integrated with the image projector.

18. The method of claim 14, wherein one or more of the at least one image capture device is attached to the image projector.

19. The calibration system of claim 14, wherein the image projection set-up is used in a Heads-Up-Display (HUD) assembly.

20. The calibration system of claim 14, wherein the image projection set-up is used in a front projection configuration.

21. The calibration system of claim 14, wherein the image projection set-up is used in a rear projection configuration.

22. A calibration method for use with an image projection set-up having an image source and a viewing surface, said method being carried out by at least one electronic processor configured to:

receive input image data acquired, from a scene containing the viewing surface, by at least one image capture device equipped with an Ultra Wide-Angle (UWA) lens with a field of view of about 180°

or larger and having a corresponding lens transformation;

receive input data representing orientation of the at least one image capture device relative to the image projector;

detect a plurality of objects in the image that are presumed vertical and horizontal objects in the scene;

identify at least a first pair of parallel lines in one or more of the plurality of the horizontal objects, in the image in the UWA lens space;

identify at least a second pair of parallel lines, in one or more of the plurality of the vertical objects in the image in the UWA lens space;

compute coordinates of vanishing points corresponding to the at least first and the at least second pair of parallel lines;

compute angles of rotation of the at least one image capture device with respect to a frame of reference, using the coordinates of the vanishing points;

construct a correction transformation using the computed angles of rotation and the orientation of the at least one image capture device relative to the image projector; and

store data corresponding to a corrected image based on the correction information to a memory device.