Method and apparatus for removing general lens distortion from images

• US 8,265,422 B1
• Filed: 02/20/2009
• Issued: 09/11/2012
• Est. Priority Date: 02/20/2009
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Jin, Hailin
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US12/390,317
• Time in Patent Office: 1,299 Days
• Field of Search: 382/275, 382/276, 382289-297, 382/100, 348/571, 348/580, 348/583, 348/744, 345/645, 345/647
• US Class Current: 382/275
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

Method and apparatus for removing general geometric lens distortion from images. A geometric distortion removal algorithm is implemented according to a general 5-parameter distortion model that includes both radial and tangential lens distortion components. The algorithm may be implemented in a two-stage process in which one-dimensional elements in one planar dimension (rows or columns) of an input image with geometric distortion are rendered to generate an intermediate buffer, and then one-dimensional elements in the other planar dimension (columns or rows) of the intermediate buffer are rendered to generate an output image with geometric distortion reduced or removed. Thus, one-dimensional sampling is performed, and interpolation is performed in one planar dimension, rather than 2-D interpolation. An input row or column may be decimated, if necessary, before row or column interpolation is performed to generate an output row or column. Since decimation is performed in one dimension, aliasing artifacts may be prevented.

Claims

1. A computer-implemented method, comprising:

rendering, by one or more computing devices, an output image from an input image, wherein the input image has general geometric distortion, wherein the general geometric distortion includes at least one radial distortion component and at least one tangential distortion component, and wherein said rendering an output image comprises;

rendering each one-dimensional element in a first planar dimension of the input image to generate pixel values in a corresponding one-dimensional element in the first planar dimension of an intermediate buffer, wherein said rendering comprises, for each pixel in each element in the first planar dimension of the intermediate buffer;

calculating a pixel coordinate in the first planar dimension, wherein the calculated pixel coordinate in the first planar dimension accounts for at least a portion of the general geometric distortion of the input image in the first planar dimension; and

performing interpolation at the calculated pixel coordinate in the first planar dimension in the input image to generate a pixel value for the current pixel in the intermediate buffer; and

rendering each one-dimensional element in a second planar dimension of the intermediate buffer to generate pixel values in a corresponding one-dimensional element in the second planar dimension of the output image, wherein said rendering comprises, for each pixel in each element in the second planar dimension of the output image;

calculating a pixel coordinate in the second planar dimension, wherein the calculated pixel coordinate in the second planar dimension accounts for at least a portion of the general geometric distortion of the input image in the second planar dimension; and

performing interpolation at the distorted pixel coordinate in the second planar dimension in the intermediate image to generate a pixel value for the current pixel in the output image;

wherein at least a portion of the general geometric distortion present in the input image is not present in the output image.

2. The computer-implemented method as recited in claim 1, wherein the radial distortion component is one of barrel distortion and pincushion distortion.

3. The computer-implemented method as recited in claim 1, wherein the elements in the first planar dimension are rows, and wherein the elements in the second planar dimensions are columns.

4. The computer-implemented method as recited in claim 1, wherein the elements in the first planar dimension are columns, and wherein the elements in the second planar dimensions are rows.

5. The computer-implemented method as recited in claim 1, further comprising decimating the element in the first planar dimension of the input image prior to said interpolation.

6. The computer-implemented method as recited in claim 1, wherein said calculating a pixel coordinate in the first planar dimension comprises solving for x_d given coordinates (x, y_d) in the intermediate buffer, where subscript _d signifies a pixel coordinate in the distorted image, and wherein said solving for x_d given (x, y_d) comprises:

solving for y by iteratively performing

7. The computer-implemented method as recited in claim 1, further comprising decimating the element in the second planar dimension of the intermediate buffer prior to said interpolation.

8. The computer-implemented method as recited in claim 1, wherein said calculating a pixel coordinate in the second planar dimension comprises solving for y_d given coordinates (x, y) in the output buffer by performing

y_d= y(1+k₁r²+k₂r⁴+k₅r⁶)+k₃(r²+2y²)+2k₄xy+d_y

where k₁, k₂, and k₅ correspond to radial distortion parameters, k₃ and k₄ correspond to tangential distortion parameters, (d_x, d_y) is the image center for the distorted image, subscript _d signifies a pixel coordinate in the distorted image, and r=√

{square root over ( x²+ y²)}.

9. The computer-implemented method as recited in claim 1, wherein said rendering an output image from an input image with general geometric distortion is performed according to a parametric general lens distortion model that includes both radial distortion and tangential distortion components.

10. A system, comprising:

at least one processor; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to render an output image from an input image, wherein the input image has general geometric distortion, wherein the general geometric distortion includes at least one radial distortion component and at least one tangential distortion component, and wherein, to render an output image, the program instructions are executable by the at least one processor to;

render each one-dimensional element in a first planar dimension of the input image to generate pixel values in a corresponding one-dimensional element in the first planar dimension of an intermediate buffer, wherein, to perform said rendering, the program instructions are executable by the at least one processor to, for each pixel in each element in the first planar dimension of the intermediate buffer;

calculate a pixel coordinate in the first planar dimension, wherein the calculated pixel coordinate in the first planar dimension accounts for at least a portion of the general geometric distortion of the input image in the first planar dimension; and

perform interpolation at the calculated pixel coordinate in the first planar dimension in the input image to generate a pixel value for the current pixel in the intermediate buffer; and

render each one-dimensional element in a second planar dimension of the intermediate buffer to generate pixel values in a corresponding one-dimensional element in the second planar dimension of the output image, wherein, to perform said rendering, the program instructions are executable by the at least one processor to, for each pixel in each element in the second planar dimension of the output image;

calculate a pixel coordinate in the second planar dimension, wherein the calculated pixel coordinate in the second planar dimension accounts for at least a portion of the general geometric distortion of the input image in the second planar dimension; and

perform interpolation at the distorted pixel coordinate in the second planar dimension in the intermediate image to generate a pixel value for the current pixel in the output image;

wherein at least a portion of the general geometric distortion present in the input image is not present in the output image.

11. The system as recited in claim 10, wherein the radial distortion component is one of barrel distortion and pincushion distortion.

12. The system as recited in claim 10, wherein the elements in the first planar dimension are rows, and wherein the elements in the second planar dimensions are columns.

13. The system as recited in claim 10, wherein the elements in the first planar dimension are columns, and wherein the elements in the second planar dimensions are rows.

14. The system as recited in claim 10, wherein the program instructions are executable by the at least one processor to decimate the element in the first planar dimension of the input image prior to said interpolation.

15. The system as recited in claim 10, wherein, to calculate a pixel coordinate in the first planar dimension, the program instructions are executable by the at least one processor to solve for x_d given coordinates (x, y_d) in the intermediate buffer, where subscript _d signifies a pixel coordinate in the distorted image, and wherein, to solve for x_d given (x, y_d), the program instructions are executable by the at least one processor to:

solve for y by iteratively performing

16. The system as recited in claim 10, wherein the program instructions are executable by the at least one processor to decimate the element in the second planar dimension of the intermediate buffer prior to said interpolation.

17. The system as recited in claim 10, wherein, to calculate a pixel coordinate in the second planar dimension, the program instructions are executable by the at least one processor to solve for y_d given coordinates (x, y) in the output buffer according to

y_d= y(1+k₁r²+k₂r⁴+k₅r⁶)+k₃(r²+2y²)+2k₄xy+d_y

where k₁, k₂, and k₅ correspond to radial distortion parameters, k₃ and k₄ correspond to tangential distortion parameters, (d_x, d_y) is the image center for the distorted image, subscript _d signifies a pixel coordinate in the distorted image, and r=√

{square root over ( x²+ y²)}.

18. The system as recited in claim 10, wherein the program instructions are executable by the at least one processor to render the output image from the input image with general geometric distortion according to a parametric general lens distortion model that includes both radial distortion and tangential distortion components.

19. The system as recited in claim 10, wherein the system is a digital camera.

20. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:

rendering an output image from an input image, wherein the input image has general geometric distortion, wherein the general geometric distortion includes at least one radial distortion component and at least one tangential distortion component, and wherein, in said rendering an output image, the program instructions are computer-executable to implement;

rendering each one-dimensional element in a first planar dimension of the input image to generate pixel values in a corresponding one-dimensional element in the first planar dimension of an intermediate buffer, wherein in said rendering the program instructions are computer-executable to implement, for each pixel in each element in the first planar dimension of the intermediate buffer;

calculating a pixel coordinate in the first planar dimension, wherein the calculated pixel coordinate in the first planar dimension accounts for at least a portion of the general geometric distortion of the input image in the first planar dimension; and

performing interpolation at the calculated pixel coordinate in the first planar dimension in the input image to generate a pixel value for the current pixel in the intermediate buffer; and

rendering each one-dimensional element in a second planar dimension of the intermediate buffer to generate pixel values in a corresponding one-dimensional element in the second planar dimension of the output image, wherein in said rendering the program instructions are computer-executable to implement, for each pixel in each element in the second planar dimension of the output image;

calculating a pixel coordinate in the second planar dimension, wherein the calculated pixel coordinate in the second planar dimension accounts for at least a portion of the general geometric distortion of the input image in the second planar dimension; and

performing interpolation at the distorted pixel coordinate in the second planar dimension in the intermediate image to generate a pixel value for the current pixel in the output image;

wherein at least a portion of the general geometric distortion present in the input image is not present in the output image.

21. The non-transitory computer-readable storage medium as recited in claim 20, wherein the radial distortion component is one of barrel distortion and pincushion distortion.

22. The non-transitory computer-readable storage medium as recited in claim 20, wherein the elements in the first planar dimension are rows, and wherein the elements in the second planar dimensions are columns.

23. The non-transitory computer-readable storage medium as recited in claim 20, wherein the elements in the first planar dimension are columns, and wherein the elements in the second planar dimensions are rows.

24. The non-transitory computer-readable storage medium as recited in claim 20, wherein the program instructions are further computer-executable to implement decimating the element in the first planar dimension of the input image prior to said interpolation.

25. The non-transitory computer-readable storage medium as recited in claim 20, wherein, in said calculating a pixel coordinate in the first planar dimension, the program instructions are computer-executable to implement solving for x_d given coordinates (x, y_d) in the intermediate buffer, where subscript _d signifies a pixel coordinate in the distorted image, and wherein said solving for x_d given (x, y_d) comprises:

solving for y by iteratively performing

26. The non-transitory computer-readable storage medium as recited in claim 20, wherein the program instructions are computer-executable to implement decimating the element in the second planar dimension of the intermediate buffer prior to said interpolation.

27. The non-transitory computer-readable storage medium as recited in claim 20, wherein, in said calculating a pixel coordinate in the second planar dimension, the program instructions are computer-executable to implement solving for y_d given coordinates (x, y) in the output buffer by performing

y_d= y(1+k₁r²+k₂r⁴+k₅r⁶)+k₃(r²+2y²)+2k₄xy+d_y

where k₁, k₂, and k₅ correspond to radial distortion parameters, k₃ and k₄ correspond to tangential distortion parameters, (d_x, d_y) is the image center for the distorted image, subscript _d signifies a pixel coordinate in the distorted image, and r=√

{square root over ( x²+ y²)}.

28. The non-transitory computer-readable storage medium as recited in claim 20, wherein the program instructions are computer-executable to perform said rendering the output image from the input image with general geometric distortion according to a parametric general lens distortion model that includes both radial distortion and tangential distortion components.

29. A computer-implemented method, comprising:

executing instructions on a specific apparatus so that binary digital electronic signals representing an output image are rendered from binary digital electronic signals representing an input image, wherein the input image has general geometric distortion, wherein the general geometric distortion includes at least one radial distortion component and at least one tangential distortion component, and wherein said rendering the output image from the input image comprises;

rendering each one-dimensional element in a first planar dimension of the input image to generate pixel values in a corresponding one-dimensional element in the first planar dimension of binary digital electronic signals representing an intermediate buffer, wherein said rendering comprises, for each pixel in each element in the first planar dimension of the intermediate buffer;

calculating a pixel coordinate in the first planar dimension, wherein the calculated pixel coordinate in the first planar dimension accounts for at least a portion of the general geometric distortion of the input image in the first planar dimension; and

performing interpolation at the calculated pixel coordinate in the first planar dimension in the input image to generate a pixel value for the current pixel in the intermediate buffer; and

rendering each one-dimensional element in a second planar dimension of the intermediate buffer to generate pixel values in a corresponding one-dimensional element in the second planar dimension of the output image, wherein said rendering comprises, for each pixel in each element in the second planar dimension of the output image;

calculating a pixel coordinate in the second planar dimension, wherein the calculated pixel coordinate in the second planar dimension accounts for at least a portion of the general geometric distortion of the input image in the second planar dimension; and

performing interpolation at the distorted pixel coordinate in the second planar dimension in the intermediate image to generate a pixel value for the current pixel in the output image; and

storing the output image in a memory location of said specific apparatus for later use, wherein at least a portion of the general geometric distortion present in the input image is not present in the output image.