METHOD AND APPARATUS FOR TRANSFORMING A NON-LINEAR LENS-DISTORTED IMAGE

• US 20100111440A1
• Filed: 10/31/2008
• Published: 05/06/2010
• Est. Priority Date: 10/31/2008
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Lin, Ruei-Sung, Bellas, Nikolaos, Dwyer, Malcolm R., Chai, Sek M., Linzmeier, Daniel A.
• Granted Patent: US 8,326,077 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals

Abstract

A method and apparatus for image processing a lens-distorted image (e.g., a fisheye image) is provided. The method includes partitioning coordinate points in a selected output image into tiles. The output image is an undistorted rendition of a subset of the lens-distorted image. Coordinate points on a border of the tiles in the output image are selected. For each tile, coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image are calculated. In addition, for each tile, a bounding box on the lens-distorted image is selected. The bounding box includes the calculated coordinates in the lens-distorted image. The bounding boxes are expanded so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles. Output pixel values are generated for each tile from pixel values in their corresponding expanded bounding boxes.

Claims

1. A method for image processing a lens-distorted image, the method comprising:

partitioning coordinate points in a selected output image into tiles, wherein the output image is an undistorted rendition of a subset of the lens-distorted image;

selecting coordinate points on a border of the tiles in the output image;

for each tile, calculating coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image;

for each tile, selecting a bounding box on the lens-distorted image that includes the calculated coordinates in the lens-distorted image;

expanding the bounding boxes so that they encompass all coordinate points in the lens-distorted image that map to all coordinate points in their respective corresponding tiles; and

generating output pixel values for each tile from pixel values in their corresponding expanded bounding boxes.

2. A method in accordance with claim 1, wherein the partitioning the output image into tiles is further dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixels for each tile.

3. A method in accordance with claim 1, wherein selecting coordinate points on the border of the tiles comprises selecting corner points on the border of the tiles.

4. A method in accordance with claim 1, wherein selecting the bounding boxes comprises selecting maximum and minimum coordinates of the selected coordinate points for each tile in the output image.

5. A method in accordance with claim 1, wherein expanding the bounding boxes is further dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern used to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.

6. A method in accordance with claim 1, wherein the selected output image is selected in accordance with at least one received input pan, tilt, or zoom parameters.

7. A method in accordance with claim 1, wherein generating the output pixels comprises:

transferring pixel values from the lens-distorted image for tile K of the output image;

calculating the coordinates of the pixels in the tile K of the output image within the coordinate system of the lens-distorted image

calculating output pixel values for tile K+1 of the output image by interpolating between pixels of the lens-distorted image for tile K+1; and

filtering the output pixel data for tile K+2 of the output image.

8. A method in accordance with claim 7, wherein the transferring step, the calculating steps, and the filtering step are performed in parallel with one another.

9. A method in accordance with claim 7, wherein filtering the output pixel values comprises of vertical filtering and horizontal filtering steps for consecutive tile K of the output image.

10. A method in accordance with claim 7, wherein the pixel values comprises luminance and chrominance color channels.

11. An imaging system for providing undistorted views of a selected portion of a lens-distorted optical image, comprising:

a lens for obtaining a lens-distorted input optical image;

a processor transforming a selected portion of the lens-distorted image to produce an undistorted output image, wherein the processor is configured to perform the transformation by dividing coordinates in the output image into a plurality of tiles, establishing bounding boxes on the input optical image that each correspond to one of the tiles, and generating output pixel values for each tile from pixel values in their corresponding bounding boxes.

12. The imaging system of claim 11 wherein the processor is further configured to generate the output pixel values in accordance with a pipelining process in which different tasks are performed simultaneously on different ones of the tiles.

13. The imaging system of claim 12 wherein the different tasks include transferring input pixel values in a bounding box to the processor, inverse mapping coordinates in each tile to coordinates in a corresponding one of the bounding boxes, and interpolating and filtering the transferred output pixel values.

14. The imaging system of claim 11 wherein the lens is a fisheye lens.

15. The imaging system of claim 11 wherein the processor is further configured to expand the bounding boxes so that they encompass all coordinate points in the lens-distorted input image that map to all coordinate points in their respective corresponding tiles.

16. The imaging system of claim 11 wherein the processor is further configured to establish the bounding boxes by:

selecting coordinate points on a border of the tiles in the output image;

for each tile, calculating coordinate points in the lens-distorted image corresponding to each selected coordinate point in the output image; and

for each tile, selecting a bounding box on the lens-distorted image that includes the calculated coordinates in the lens-distorted image;

17. The imaging system of claim 16 wherein the processor is further configured to select coordinate points on the border of the tiles by selecting corner coordinate points on the border of the tiles.

18. The imaging system of claim 17 wherein the processor is further configured to select the bounding boxes by selecting maximum and minimum coordinates of the selected corner coordinate points for each tile in the output image.

19. A method in accordance with claim 11, wherein the processor is further configured to divide the coordinates in the output image into a plurality of tiles in a manner dependent upon at least one item selected from the group consisting of available on-chip memory, memory controller burst size, and a complexity of a memory access pattern used to generate the output pixels for each tile.

20. A method in accordance with claim 11, wherein the processor is further configured to establish the bounding boxes in a manner dependent upon at least one item selected from the group consisting of filter size used in image processing, available on-chip memory, memory controller burst size, a complexity of a memory access pattern to fetch pixel values from memory, and a location of the bounding box with respect to a center of lens-distortion.