Method and apparatus for transforming a non-linear lens-distorted image

• US 8,855,441 B2
• Filed: 11/02/2012
• Issued: 10/07/2014
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: General Instrument Corporation (CommScope Holding Company, Inc.)
• Inventors: Chai, Sek M., Dwyer, Malcolm R., Lin, Ruei-Sung, Linzmeier, Daniel A., Bellas, Nikolas
• Primary Examiner(s): AHMED, SAMIR ANWAR
• Application Number:
  US13/667,604
• Publication Number:
  US 20130058589A1
• Time in Patent Office: 704 Days
• Field of Search: 382/255, 382/274, 382/275, 382/284, 348/207.99, 348/36, 348/208.11, 348/224.1
• US Class Current: 382/275
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals

Abstract

A method and apparatus for image processing a lens-distorted image (e.g., a fisheye image) is provided. The method includes partitioning coordinate points in a selected output image into tiles. The output image is an undistorted rendition of a subset of the lens-distorted image. Coordinate points on a border of the tiles in the output image are selected. For each tile, coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image are calculated. In addition, for each tile, a bounding box on the lens-distorted image is selected. The bounding box includes the calculated coordinates in the lens-distorted image. The bounding boxes are expanded so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles. Output pixel values are generated for each tile from pixel values in their corresponding expanded bounding boxes.

Claims

1. A non-transitory computer-readable storage medium comprising stored instructions which, when executed by one or more processors, cause the one or more processors to perform a method for processing a lens-distorted image, the method comprising:

for an output image corresponding to an undistorted rendition of a subset of the lens-distorted image, partitioning coordinate points in the output image into tiles;

selecting coordinate points on a border of the tiles in the output image;

for each tile, calculating coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image;

for each tile, selecting a bounding box on the lens-distorted image that includes the calculated coordinates in the lens-distorted image;

expanding the bounding boxes so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles; and

generating output pixel values for each tile from pixel values in their corresponding expanded bounding boxes.

2. The computer-readable storage medium of claim 1, wherein the partitioning the output image into tiles is further dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixel values for each tile.

3. The computer-readable storage medium of claim 1, wherein selecting coordinate points on the border of the tiles comprises selecting corner points on the border of the tiles.

4. The computer-readable storage medium of claim 1, wherein selecting the bounding boxes comprises selecting maximum and minimum coordinates of the selected coordinate points for each tile in the output image.

5. The computer-readable storage medium of claim 1, wherein expanding the bounding boxes is further dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern used to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.

6. The computer-readable storage medium of claim 1, wherein the selected output image is selected in accordance with at least one received input pan, tilt, or zoom parameters.

7. The computer-readable storage medium of claim 1, wherein generating the output pixel values comprises:

transferring pixel values from the lens-distorted image for tile K of the output image;

calculating the coordinates of the pixels in the tile K of the output image within the coordinate system of the lens-distorted image;

calculating output pixel values for tile K+1 of the output image by interpolating between pixels of the lens-distorted image for tile K+1; and

filtering the output pixel values for tile K+2 of the output image.

8. The computer-readable storage medium of claim 7, wherein the transferring step, the calculating steps, and the filtering step are performed in parallel with one another.

9. The computer-readable storage medium of claim 7, wherein filtering the output pixel values comprises of vertical filtering and horizontal filtering steps for consecutive tile K of the output image.

10. The computer-readable storage medium of claim 7, wherein the pixel values comprise luminance and chrominance color channels.

20. The computer-readable medium of claim 1, the method further comprising:

storing the output image in an output memory communicatively coupled to the one or more processors.

11. A device comprising:

a memory for storing digitized optical image data comprising a lens-distorted input image;

a processor communicatively coupled to the memory, the processor transforming a selected portion of the lens-distorted input image to produce an undistorted output image, wherein the processor is configured to perform the transformation by dividing coordinates in the output image into a plurality of tiles;

for each tile, calculating coordinate points in the lens-distorted input image corresponding to each selected coordinate point in the output image; and

for each tile, selecting a bounding box on the lens-distorted input image that includes the calculated coordinates in the lens-distorted input image;

wherein the processor is further configured to establish the bounding boxes by selecting coordinate points on a border of the tiles in the output image;

and wherein the processor is further configured to expand the bounding boxes so that they encompass all coordinate points in the lens-distorted input image that map to all coordinate points in their respective corresponding tiles.

12. The device of claim 11 wherein the processor is further configured to generate output pixel values in accordance with a pipelining process in which different tasks are performed simultaneously on different ones of the tiles.

13. The device of claim 12 wherein the different tasks include transferring input pixel values in a bounding box to the processor, inverse mapping coordinates in each tile to coordinates in a corresponding one of the bounding boxes, and interpolating and filtering transferred output pixel values.

14. The device of claim 11 wherein the lens-distorted input image comprises lens distortion from a fisheye lens.

15. The device of claim 11 wherein the processor is further configured to select coordinate points on the border of the tiles by selecting corner coordinate points on the border of the tiles.

16. The device of claim 15 wherein the processor is further configured to select the bounding boxes by selecting maximum and minimum coordinates of the selected corner coordinate points for each tile in the output image.

17. The device of claim 11, wherein the processor is further configured to divide the coordinates in the output image into a plurality of tiles in a manner dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixel values for each tile.

18. The device of claim 11, wherein the processor is further configured to establish the bounding boxes in a manner dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.

19. The device of claim 11 further comprising:

an output memory, communicatively coupled to the processor, for storing the undistorted output image.