Digital aerial image mosaic method and apparatus

• US 6,694,064 B1
• Filed: 11/20/2000
• Issued: 02/17/2004
• Est. Priority Date: 11/19/1999
• Status: Expired due to Fees

• Current Assignee: Corveq LLC Imaging (Corveq LLC)
• Original Assignee: Positive System Inc.
• Inventors: Benkelman, Cody
• Primary Examiner(s): Patel, Jayanti K.
• Assistant Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US09/718,440
• Time in Patent Office: 1,184 Days
• Field of Search: 382/48, 382/54, 382/56, 382/100, 382/133, 382/154, 382/232, 382/233, 382/263, 382/268, 382/284, 382/294, 709/217, 345/420, 345/582, 345/679, 345/719, 430/313, 430/315, 348/144
• US Class Current: 382/284
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A computer-implemented method and system for use in alignment of multiple digital images to form a mosaic image involves selecting multiple search site points (SSPs) in an overlapping area of a pair of the digital images and searching for an interesting point (IP) near each of the SSPs. The system involves the calculation of a numeric interest measure (IM) at each of multiple IP-candidate sites near the SSPs. The IM is indicative of the presence of image features at the IP-candidate site and provides a basis for comparing the IP-candidate sites at each of the SSPs and selecting the IP-candidate site having the most significant IM. In a preferred embodiment, IP-candidate sites having an IM that does not exceed a predetermined minimum threshold are discarded. The method also involves locating a tie point (TP) on an overlapping one of the digital images correlating to the IP. The TP together with the IP comprise a tie point pair (TPP) that can be used to calculate and apply geometric transformations to align the images and thereby form a seamless mosaic. The system and method may also involve radiometric balancing of the images to reduce tonal mismatch.

Claims

1. A computer-implemented method of identifying a tie point pair (TPP) for use in alignment of multiple digital images to form a mosaic image, the digital images including at least one overlapping area, the method comprising:

selecting multiple search site points (SSPs) in the overlapping area of a first one of the digital images;

searching for an interesting point (IP) near each of the SSPs, including;

(a) at multiple IP-candidate sites near each of the SSPs, calculating a numeric interest measure (IM), the IM being indicative of the presence of image features at the IP-candidate site,

(b) selecting one of the IP-candidate sites having the largest IM, and

(c) if the IM exceeds a predetermined threshold, designating the IP-candidate site as an interesting point (IP); and

locating on a second one of the digital images overlapping the first one of the digital images a tie point (TP) correlating to the IP, the TP together with the IP comprising a tie point pair (TPP).

2. The method of claim 1 in which the IM is indicative of the minimum directional variance at the IP-candidate site.

3. The method of claim 1, further comprising aligning the first and second digital images based on the TPP.

4. The method of claim 1, further comprising locating multiple TPPs.

5. The method of claim 4, further comprising aligning the first and second digital images based on the multiple TPPs to thereby form a mosaic image.

6. The method of claim 4, further comprising calculating a set of transformation coefficients for use in geographically transforming the first and second digital images and to thereby align the first and second digital images.

7. The method of claim 1, further comprising calculating a set of transformation coefficients using a least squares fit method for use in geographically transforming the first and second digital images and to thereby align the first and second digital images.

8. The method of claim 7, further comprising geometrically transforming the first and second digital images using the set of transformation coefficients to thereby generate a mosaic image.

9. The method of claim 1, further comprising automatically adjusting the radiometric properties of the first and second digital images.

10. The method of claim 1 in which the digital images include multispectral images.

11. A method for use in generating a mosaic from a set of images including at least a first image and a second image, the first image having an overlapping area that overlaps the second image, the method comprising:

designating an IP-candidate site of the first image in the overlapping area;

calculating a numeric interest measure (IM) indicative of the presence of image features at the IP-candidate site; and

testing the IM to determine whether the IM meets a predetermined threshold; and

if the IM meets the predetermined threshold, locating a tie point (TP) on the second image that correlates to the IP-candidate site.

12. The method of claim 11, further comprising:

calculating the IM at multiple IP-candidate sites on the first image, the IP-candidate sites selected from a region of the first image within a predefined search window;

selecting one of the IP-candidate sites having the maximum IM; and

if the maximum IM exceeds the predetermined threshold, locating the TP correlating to said selected IP-candidate site.

13. The method of claim 12, further comprising, determining a set of transformation coefficients based on the selected IP-candidate site and the TP for use in aligning the first and second images.

14. The method of claim 13, further comprising aligning the first and second images based on the transformation coefficients to thereby form a mosaic image.

15. The method of claim 11, further comprising aligning the first and second images based on the IP-candidate site and the corresponding TP.

16. The method of claim 11, further comprising:

identifying a plurality of SSPs in the overlapping area;

calculating a numeric interest measure (IM) of each of multiple IP-candidate sites near each of the SSPs;

for each of the SSPs, selecting one IP-candidate site having an IM exceeding the predetermined threshold; and

searching in the second image for a TP corresponding to each of the selected IP-candidate sites.

17. The method of claim 16, further comprising:

for each selected IP-candidate site, testing multiple TP-candidates on the second image to determine whether the TP-candidate correlates to the selected IP-candidate site and thereby identify the TP corresponding to the IP-candidate site.

18. The method of claim 17 in which the testing of TP-candidates includes calculating, for each TP-candidate corresponding to the selected IP-candidate sites, a numerical correlation coefficient indicative of similarities between the TP-candidate and the selected IP-candidate site;

and, further comprising;

selecting a predetermined correlation threshold; and

discarding TP-candidates having a numerical correlation coefficient that does not meet the predetermined correlation threshold.

19. A computer software program stored on a computer-usable medium for use in generating a mosaic from a set of images including at least a first image and a second image, the first image having an overlapping area that overlaps the second image, the computer software program comprising:

an initial alignment routine for roughly aligning the set of images;

an interesting point identification routine for identifying at least one interesting points (IP) on at least a first one of the images; and

a tie point matching routine for searching in at least a second one of the images for a TP correlating to the IP.

20. The computer software program of claim 19, further comprising:

a transformation calculation engine for calculating transformation coefficients based on the TP and the IP, the transformation coefficients representing geometric transformations necessary to align the images.

21. The computer software program of claim 20, further comprising:

a mosaic generation routine for applying the transformation coefficients to the images and to thereby generate a mosaic image.

22. The computer software program of claim 19, further comprising:

a radiometric adjustment routine for adjusting the radiometric properties of the images to thereby reduce tonal mismatch between the images.

23. The computer software program of claim 19, in which one or both of the interesting point identification routine and the tie point matching routine is separable from the remainder of the computer program to facilitate operation of the computer software program in an enhanced computing environment selected from the following:

(a) a multithreaded processing environment;

(b) a distributed processing environment;

(c) a multi-processor computer;

(d) a computer having multiple logical partitions; and

(e) a parallel processing computer.