Homography rectification

• US 10,275,863 B2
• Filed: 04/03/2015
• Issued: 04/30/2019
• Est. Priority Date: 04/03/2015
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Bogan, Nathaniel
• Primary Examiner(s): Niu, Feng
• Application Number:
  US14/678,279
• Publication Number:
  US 20160292831A1
• Time in Patent Office: 1,488 Days
• Field of Search: None
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

The present disclosure provides a multi-stage image mapping mechanism for mapping a distorted image to a rectified image. For example, the multi-stage image mapping mechanism can remove homography from a distorted image to reconstruct a rectified image in two-stages: (1) a first stage in which distortion is partially removed from a distorted image to generate an intermediate image, and (2) a second stage in which residual distortion is removed from the intermediate image to recover the rectified image.

Claims

1. A machine vision system comprising:

a processor configured to run a computer program stored in memory that is configured to cause the processor to;

receive a distorted image and a homography matrix associated with the distorted image;

determine, using a homography decomposition module, a first transform and a second transform that collectively represent the homography matrix, wherein the first transform relates the distorted image to an intermediate image, and the second transform relates a transposed intermediate image to a transposed rectified image, wherein the distorted image, the intermediate image, the transposed intermediate image, and the transposed rectified image comprise at least a first dimension and a second dimension;

wherein;

the first transform is constrained to map pixels along the first dimension of the intermediate image to locations along the first dimension of the distorted image, and wherein the locations along the first dimension of the distorted image are constrained to be separated by a same distance from adjacent locations along the first dimension of the distorted image by constraining a rate of change of (1) the first dimension of the distorted image with respect to (2) the first dimension of the intermediate image; and

the second transform is constrained to map pixels along the first dimension of the transposed rectified image to locations along the first dimension of the transposed intermediate image, and wherein the locations along the first dimension of the transposed intermediate image are constrained to be separated by a same distance from adjacent locations along the first dimension of the transposed intermediate image by constraining a rate of change of (1) the first dimension of the transposed intermediate image with respect to (2) the first dimension of the transposed rectified image;

generate, using a homography removal module, the intermediate image by removing the first transform from the distorted image;

transpose, using the homography removal module, the intermediate image prior to removing the second transform from the intermediate image to generate the transposed intermediate image; and

generate, using the homography removal module, the transposed rectified image by removing the second transform from the transposed intermediate image.

2. The machine vision system of claim 1, wherein the processor is configured, using the homography removal module, to cause a direct memory access controller to transpose the intermediate image prior to removing the second transform from the intermediate image.

3. The machine vision system of claim 2, wherein the direct memory access controller is configured to transpose the intermediate image using a two-dimensional direct memory access.

4. The machine vision system of claim 1, wherein the processor is configured, using the homography removal module, to remove the first transform from the distorted image by:

determining, for a first pixel of the intermediate image, a corresponding first location in the distorted image; and

convolving pixels surrounding the corresponding first location in the distorted image to determine an image value for the first pixel.

5. The machine vision system of claim 4, wherein the processor is configured, using the homography removal module, to determine the corresponding first location by operating the first transform on a coordinate of the first pixel.

6. The machine vision system of claim 4, wherein the processor is configured, using the homography removal module, to determine, for a second pixel of the intermediate image having an identical coordinate as the first pixel in a second dimension, a corresponding second location in the distorted image by adding a constant to a coordinate of the corresponding first location in the first dimension.

7. The machine vision system of claim 1, wherein the processor is configured, using the homography removal module, to:

transpose the transposed rectified image to generate a rectified image; and

provide the rectified image to a pattern detection system so that the pattern detection system can use the rectified image to determine whether the rectified image includes a particular pattern.

8. The machine vision system of claim 7, wherein the processor is configured to receive the homography matrix from the pattern detection system, wherein the homography matrix is determined by the pattern detection system based on a hypothesis of homography associated with the distorted image.

9. The machine vision system of claim 1, wherein the processor is configured, using the homography removal module, to select one of a set of kernels based on a sub-pixel coordinate of one of the locations along the first dimension of the distorted image, and apply the selected kernel at the one of the locations.

10. A computerized method comprising:

receiving, at a homography decomposition module, a homography matrix associated with a distorted image;

determining, using the homography decomposition module, a first transform and a second transform that collectively represent the homography matrix, wherein the first transform relates the distorted image to an intermediate image, and the second transform relates a transposed intermediate image to a transposed rectified image, wherein the distorted image, the intermediate image, the transposed intermediate image, and the transposed rectified image comprise at least a first dimension and a second dimension;

wherein;

the first transform is constrained to map pixels along the first dimension of the intermediate image to locations along the first dimension of the distorted image, and wherein the locations along the first dimension of the distorted image are constrained to be separated by a same distance from adjacent locations along the first dimension of the distorted image; and

the second transform is constrained to map pixels along the first dimension of the transposed rectified image to locations along the first dimension of the transposed intermediate image, and wherein the locations along the first dimension of the transposed intermediate image are constrained to be separated by a same distance from adjacent locations along the first dimension of the transposed intermediate image;

generating, using a homography removal module coupled to the homography decomposition module, the intermediate image by removing the first transform from the distorted image;

transposing, using the homography removal module, the intermediate image using the homography removal module prior to removing the second transform from the intermediate image to generate the transposed intermediate image; and

generating, using the homography removal module, the transposed rectified image by removing the second transform from the transposed intermediate image.

11. The method of claim 10, further comprising causing, using the homography removal module, a direct memory access controller to transpose the intermediate image prior to removing the second transform from the intermediate image.

12. The method of claim 10, wherein removing the first transform from the distorted image comprises determining, for a first pixel of the intermediate image, a corresponding first location in the distorted image, and convolving pixels surrounding the corresponding location in the distorted image to determine an image value for the first pixel.

13. The method of claim 12, further comprising determining, for a second pixel of the intermediate image having an identical coordinate as the first pixel in a second dimension, a corresponding second location in the distorted image by adding a constant to a coordinate of the corresponding first location in the first dimension.

14. The method of claim 12, further comprising:

transposing the transposed rectified image to generate a rectified image; and

providing the rectified image to a pattern detection system so that the pattern detection system can use the rectified image to determine whether the rectified image includes a particular pattern.

15. The method of claim 12, further comprising selecting one of a set of kernels based on a sub-pixel coordinate of one of the locations along the first dimension of the distorted image, and applying the selected kernel at the one of the locations.

16. A non-transitory computer readable medium having executable instructions associated with a homography decomposition module and a homography removal module, operable to cause a machine vision system to:

receive a homography matrix associated with a distorted image;

determine a first transform and a second transform that collectively represent the homography matrix, wherein the first transform relates the distorted image to an intermediate image, and the second transform relates a transposed intermediate image to a transposed rectified image, wherein the distorted image, the intermediate image, the transposed intermediate image, and the transposed rectified image comprise at least a first dimension and a second dimension;

wherein;

the first transform is constrained to map pixels along the first dimension of the intermediate image to locations along the first dimension of the distorted image, and wherein the locations along the first dimension of the distorted image are constrained to be separated by a same distance from adjacent locations along the first dimension of the distorted image; and

the second transform is constrained to map pixels along the first dimension of the transposed rectified image to locations along the first dimension of the transposed intermediate image, and wherein the locations along the first dimension of the transposed intermediate image are constrained to be separated by a same distance from adjacent locations along the first dimension of the transposed intermediate image;

generate the intermediate image by removing the first transform from the distorted image;

transpose the intermediate image prior to removing the second transform from the intermediate image to generate the transposed intermediate image; and

generate the transposed rectified image by removing the second transform from the transposed intermediate image.