Registration of distorted images

• US 8,493,459 B2
• Filed: 09/15/2011
• Issued: 07/23/2013
• Est. Priority Date: 09/15/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Albu, Felix
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): PHAM, QUAN L
• Application Number:
  US13/234,139
• Publication Number:
  US 20130070125A1
• Time in Patent Office: 677 Days
• Field of Search: 348/218.1, 382/275, 382/284, 382/294
• US Class Current: 348/218.1
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20068 Projection on vertical or horizontal image axis
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

An image registration method involves computing horizontal and vertical integral projection vectors for first and second distorted or partially distorted images or distortion-corrected images, or both. The images are registered by applying a translation, rotation and/or scale factor estimation between the first and second images on the horizontal and vertical integral projection vectors.

Claims

1. An image registration method, comprising:

acquiring first and second distorted or partially distorted images of scenes that each include a first object;

computing horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images;

applying at least an approximation of an inverse distortion function to the horizontal and vertical integral projection vectors of the first and second distorted or partially distorted images to generate distorted horizontal and vertical integral projection vectors;

registering the first and second distorted or partially distorted images, including applying a scale factor estimation between the first and second images on the distorted horizontal and vertical integral projection vectors, wherein the registering comprises applying a scale factor estimation including;

resampling at least a portion of the first distorted or partially distorted image with multiple different resampling factors over a range of resampling factors to generate multiple resampled integral projection vectors;

computing for each of the different resampling factors an absolute sum of differences, or sum of squares, between the resampled integral projection vectors of the first image and corresponding integral projection vectors of the second image; and

determining the scaling factor between the first and second distorted or partially distorted images, including finding the resampling factor that corresponds to the minimum value of the absolute sums over the range of resampling factors; and

performing a further action using the registered first and second distorted or partially distorted images.

2. The method of claim 1, wherein the registering of the first and second distorted or partially distorted images and the performing of the further action do not first involve correcting distortion in the first and second distorted or partially distorted images.

3. The method of claim 1, wherein the computing of the horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images comprises summing columns and rows of image data, respectively.

4. The method of claim 1, further comprising determining a distortion function for at least a portion of the first or second distorted or partially distorted image, or both, including finding a model function that approximately fits the distortion of at least said portion of the first or second distorted or partially distorted image, or both.

5. The method of claim 4, wherein the distortion function determined for at least said portion of the first or second distorted or partially distorted image, or both, comprises:

1+k₁r;

or

[1+k₁r]<sup>−

1</sup>;

or

1+k₁r²;

or

[1+k₁r²]<sup>−

1</sup>;

or

1+k₁r+k₂r²;

or

(1+k₁r)/(1+k₂r²);

or

combinations thereof, wherein r comprises a radial distortion function parameter and k₁ and k₂ comprise distortion function coefficients for the first or second distorted or partially distorted image, respectively.

6. The method of claim 1, wherein the first object comprises a face.

7. One or more non-transitory processor-readable media having code embedded therein for programming one or more processors to perform an image registration method, wherein the method comprises:

acquiring first and second distorted or partially distorted images of scenes that each include a first object;

computing horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images;

applying at least an approximation of an inverse distortion function to the horizontal and vertical integral projection vectors of the first and second distorted or partially distorted images to generate distorted horizontal and vertical integral projection vectors;

registering the first and second distorted or partially distorted images, including applying a scale factor estimation between the first and second images on the distorted horizontal and vertical integral projection vectors, wherein the registering comprises applying a scale factor estimation including;

resampling at least a portion of the first distorted or partially distorted image with multiple different resampling factors over a range of resampling factors to generate multiple resampled integral projection vectors;

computing for each of the different resampling factors an absolute sum of differences, or sum of squares, between the resampled integral projection vectors of the first image and corresponding integral projection vectors of the second image; and

determining the scaling factor between the first and second distorted or partially distorted images, including finding the resampling factor that corresponds to the minimum value of the absolute sums over the range of resampling factors; and

performing a further action using the registered first and second distorted or partially distorted images.

8. The one or more non-transitory processor-readable media of claim 7, wherein the registering of the first and second distorted or partially distorted images and the performing of the further action do not first involve correcting distortion in the first and second distorted or partially distorted images.

9. The one or more non-transitory processor-readable media of claim 7, wherein the computing of the horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images comprises summing columns and rows of image data, respectively.

10. The one or more non-transitory processor-readable media of claim 7, wherein the method further comprises determining a distortion function for at least a portion of the first or second distorted or partially distorted image, or both, including finding a model function that approximately fits the distortion of at least said portion of the first or second distorted or partially distorted image, or both.

11. The one or more non-transitory processor-readable media of claim 10, wherein the distortion function determined for at least said portion of the first or second distorted or partially distorted image, or both, comprises:

1+k₁r;

or

[1+k₁r]<sup>−

1</sup>;

or

1+k₁r²;

or

[1+k₁r²]<sup>−

1</sup>;

or

1+k₁r+k₂r²;

or

(1+k₁r)/(1+k₂r²);

or

combinations thereof, wherein r comprises a radial distortion function parameter and k₁ and k₂ comprise distortion function coefficients for the first or second distorted or partially distorted image, respectively.

12. The one or more non-transitory processor-readable media of claim 7, wherein the first object comprises a face.

13. A digital image acquisition device, comprising:

a lens,

an image sensor,

a processor,

a memory having code embedded therein for programming the processor to perform an image registration method, wherein the method comprises;

acquiring first and second distorted or partially distorted images of scenes that each include a first object;

computing horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images;

applying at least an approximation of an inverse distortion function to the horizontal and vertical integral projection vectors of the first and second distorted or partially distorted images to generate distorted horizontal and vertical integral projection vectors;

registering the first and second distorted or partially distorted images, including applying a scale factor estimation between the first and second images on the distorted horizontal and vertical integral projection vectors, wherein the registering comprises applying a scale factor estimation including;

resampling at least a portion of the first distorted or partially distorted image with multiple different resampling factors over a range of resampling factors to generate multiple resampled integral projection vectors;

computing for each of the different resampling factors an absolute sum of differences, or sum of squares, between the resampled integral projection vectors of the first image and corresponding integral projection vectors of the second image; and

determining the scaling factor between the first and second distorted or partially distorted images, including finding the resampling factor that corresponds to the minimum value of the absolute sums over the range of resampling factors; and

performing a further action using the registered first and second distorted or partially distorted images.

14. The device of claim 13, wherein the registering of the first and second distorted or partially distorted images and the performing of the further action do not first involve correcting distortion in the first and second distorted or partially distorted images.

15. The device of claim 13, wherein the computing of the horizontal and vertical integral projection vectors for the first and second distorted or partially distorted images comprises summing columns and rows of image data, respectively.

16. The device of claim 13, wherein the method further comprises determining a distortion function for at least a portion of the first or second distorted or partially distorted image, or both, including finding a model function that approximately fits the distortion of at least said portion of the first or second distorted or partially distorted image, or both.

17. The device of claim 16, wherein the distortion function determined for at least said portion of the first or second distorted or partially distorted image, or both, comprises:

1+k₁r;

or

[1+k₁r]<sup>−

1</sup>;

or

1+k₁r²;

or

[1+k₁r²]<sup>−

1</sup>;

or

1+k₁r+k₂r²;

or

(1+k₁r)/(1+k₂r²);

or

combinations thereof, wherein r comprises a radial distortion function parameter and k₁ and k₂ comprise distortion function coefficients for the first or second distorted or partially distorted image, respectively.

18. The device of claim 13, wherein the first object comprises a face.