Method and system for correcting a distorted input image

• US 9,280,810 B2
• Filed: 06/02/2015
• Issued: 03/08/2016
• Est. Priority Date: 07/03/2012
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Stec, Piotr, Pososin, Alexei, Munteanu, Mihai Constantin, Zaharia, Corneliu, Georgescu, Vlad
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Brown, Jr., Howard D
• Application Number:
  US14/728,939
• Publication Number:
  US 20150262344A1
• Time in Patent Office: 280 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

A method for correcting a distorted input image comprises determining a local region of an image to be displayed and dividing said region into an array of rectangular tiles, each tile corresponding to a distorted tile with a non-rectangular boundary within said input image. For each tile of the local region, maximum and minimum memory address locations of successive rows of said input image sufficient to span said boundary of said distorted tile are determined. Successive rows of the distorted input from between said maximum and minimum addresses are read. Distortion of the non-rectangular portion of said distorted input image is corrected to provide a tile of a corrected output image which is stored.

Claims

1. An image acquisition system comprising:

a first memory for storing at least a portion of a distorted input image acquired from an image sensor and a lens system;

a second memory for storing a corrected output image; and

an interpolator module connected to said first memory for reading distorted input image information and to said second memory for writing corrected output image information, said interpolator comprising;

a bi-cubic interpolator; and

a pair of bi-linear interpolators and being switchable between a first high quality mode and a second high speed mode where,

in said first high quality mode, for each pixel for said output image, said interpolator is arranged to read a 4×

4 pixel window from said first memory and with said bi-cubic interpolator to interpolate said 4×

4 pixel window to provide said output pixel value, and

in said second high speed mode, for each pair adjacent output pixels for said corrected output image, said interpolator is arranged to read a 4×

4 pixel window from said first memory, said 4×

4 pixel window bounding a pair of 2×

2 pixel windows, each of which are interpolated in parallel by said pair of bi-linear interpolators to provide said output pixel values.

2. An image acquisition system according to claim 1 wherein said interpolator module is configured to select a minimum x and a minimum y value of said pair of 2×

2 pixel windows as a minimum x,y value for said 4×

4 pixel window.

3. An image acquisition system according to claim 1 further comprising a distortion correction module incorporating said an interpolator module and wherein said first memory comprises a cache memory within said distortion correction module, said system further comprising a system bus connecting said distortion correction module to a system memory arranged to store each of said distorted input image and said corrected output image.

4. An image acquisition system according to claim 1 wherein said first memory comprises a plurality of N interleaved memories, said memories being arranged so that said 4×

4 pixel window can be read from said memories in a single clock cycle.

5. An image acquisition system according to claim 3 further comprising:

a formatter configured to;

a) divide a local region of an image to be displayed into an array of tiles, each tile having a boundary defined by a plurality of nodes, each node having a set of coordinates within an image space;

b) for each tile of the local region, transform node coordinates according to at least a first local transformation to take into account a local distortion of said input image in said local region;

wherein said distortion correction module is configured to, for each tile of the local region;

i) read a non-rectangular portion of said distorted input image corresponding to at least locally transformed node coordinates of said tile into said first memory; and

ii) correct the distortion of said non-rectangular portion of said distorted input image to provide a tile of a corrected output image.

6. A method for correcting a distorted input image comprising:

a) determining a local region of an image to be displayed and dividing said region into an array of tiles, each tile having a boundary defined by a plurality of nodes, each node having a set of coordinates within an image space;

b) for each tile of the local region, transforming node coordinates according to a first local transformation to take into account a local distortion of said input image in said local region;

c) determining a perspective transformation applicable to at least a portion of the nodes of said local region of an image, said perspective transformation comprising multiplying node coordinates u,v transformed according to said first local transformation with the following matrix transformation;

7. A method according to claim 6 wherein matrix coefficients a1 . . . a6 are changed for successive different subsets of nodes of the local region to compensate for rolling shutter distortion.

8. A method according to claim 6 wherein matrix coefficients a1 . . . a6 are changed for different subsets of nodes of the local region to apply a user defined distortion to said input image.

9. A method according to claim 6 wherein matrix coefficients a1 . . . a6 are set to compensate for inter-frame camera shake.

10. A method according to claim 6 further comprising, for each tile of the local region, determining maximum and minimum memory address locations of successive rows of said input image sufficient to span a tile boundary, said maximum and minimum addresses for each of tile the input image corresponding to a non-rectangular region of the input image.

11. A method according to claim 10 further comprising extending the memory locations to be read from each row of said distorted input image to ensure at least four pixels immediately outside a distorted tile boundary are available for correcting the distortion of said tile.

12. A method according to claim 6 comprising acquiring said input image with a wide field of view, WFOV, lens system and wherein said global transformation is arranged to compensate for non-uniformities in said WFOV lens system.

13. A method according to claim 6 comprising acquiring said input image with a zoom lens system and responsive to changing image acquisition focal length, changing said global transformation to compensate for different non-uniformities in said lens system.

14. A method according to claim 6 comprising changing one of said global or said local transformation in response to adjusting an angle of an image acquisition sensor relative to a lens system.

15. A method according to claim 14 wherein said angle comprises a rotation angle about an axis normal to a plane of the image sensor.

16. A method according to claim 14 wherein said angle comprises a rotation angle about an axis parallel to a plane of the image sensor.

17. An image acquisition system comprising:

a memory for storing a distorted input image acquired from an image sensor and a lens system and a corrected output image;

a system bus connected to said memory; and

a distortion correction module connected to said system bus for reading distorted input image information from said memory and writing corrected output image information to said memory, said module being arranged to perform the steps of claim 6.