Method and apparatus for performing geo-spatial registration of imagery

• US 6,597,818 B2
• Filed: 03/09/2001
• Issued: 07/22/2003
• Est. Priority Date: 05/09/1997
• Status: Expired due to Term

• Current Assignee: SRI International, Inc.
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Samarasekera, Supun, Hirvonen, David James, Zhao, Wenyi, Klinedinst, Thomas Edward, Levienaise-Obadia, Barbara, Lehman, William Brian, Matei, Bodgan, Kumar, Rakesh, Hanna, Keith, Hsu, Stephen Charles, Wildes, Richard Patrick
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US09/803,700
• Publication Number:
  US 20010038718A1
• Time in Patent Office: 865 Days
• Field of Search: 382/115, 382/284, 382/285, 382/287, 382/289, 382/295, 382/296, 382/190, 382/305, 382/103, 382/106, 382/240, 382/294, 244/3.17, 348/117, 707/102, 701/25, 701/28, 701/207, 701/208, 701/209, 701/213, 702/2, 702/151, 702/5, 702/85, 345/121, 345/126, 345/435, 345/436, 345/437, 345/438
• US Class Current: 382/294
• CPC Class Codes: G06T 3/153 using elastic snapping
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/35 using statistical methods
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A system and method for accurately mapping between image coordinates and geo-coordinates, called geo-spatial registration. The system utilizes the imagery and terrain information contained in the geo-spatial database to precisely align geodetically calibrated reference imagery with an input image, e.g., dynamically generated video images, and thus achieve a high accuracy identification of locations within the scene. When a sensor, such as a video camera, images a scene contained in the geo-spatial database, the system recalls a reference image pertaining to the imaged scene. This reference image is aligned very accurately with the sensor'"'"'s images using a parametric transformation. Thereafter, other information that is associated with the reference image can easily be overlaid upon or otherwise associated with the sensor imagery.

Claims

1. A system for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image;

an preprocessor for processing the input image and said reference imagery using an oriented filter; and

an alignment module for aligning said preprocessed input image and said preprocessed reference imagery;

wherein said input is an image mosaic.

2. A system for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

a preprocessor for processing the input image and said reference imagery using an oriented filter; and

an alignment module for aligning said preprocessed input image and said preprocessed reference imagery by performing oriented matching.

3. The system of claim 2 wherein said input image is an image mosaic.

4. The system of claim 2 further comprising:

an outlier rejection module for removing inaccurate correlations between said preprocessed input image and said preprocessed reference image.

5. The system of claim 2 wherein said alignment module registers said preprocessed input image to said preprocessed reference imagery.

6. The system of claim 2 wherein said alignment module registers said preprocessed reference imagery to said preprocessed input image.

7. A system for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image;

an alignment module for aligning said input image and said reference imagery by performing oriented matching of features in said input image and said reference imagery; and

an outlier rejection module for removing inaccurate feature correlations between said preprocessed input image and said preprocessed reference image.

8. The system of claim 7 wherein said input image is an image mosaic.

9. The system of claim 7 wherein said alignment module registers said input image to said reference imagery.

10. The system of claim 7 wherein said alignment module registers said reference imagery to said input image.

11. A system for performing geo-spatial registration of an input image an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth; and

an alignment module for aligning said input image and said reference imagery by performing oriented matching of features in said input image and said reference imagery.

12. The system of claim 11 wherein said alignment module registers said input image to said reference imagery.

13. The system of claim 11 wherein said alignment module registers said reference imagery to said input image.

14. A system for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth; and

an alignment module for aligning said input image and said reference imagery by performing oriented matching of features in said input image and said reference imagery, wherein said input image is in image mosaic.

15. A system for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input image, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

an alignment module for aligning said input image and said reference imagery by performing oriented matching of features in said input image and said reference imagery; and

an outlier rejection module for removing inaccurate feature correlations between said input image and said reference image.

16. A system for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth; and

an alignment module for simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching.

17. The system of claim 16 wherein said alignment module further comprises:

a coarse alignment module for coarsely aligning said reference imagery and said input images.

18. The system of claim 17 wherein said alignment module further comprises:

a fine alignment module for accurately aligning said coarsely aligned input images and said reference image.

19. A system for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

an alignment module for simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching; and

a preprocessor module for orientation filtering said reference imagery and said input images.

20. The system of claim 19 wherein the preprocessor module further comprises an interframe alignment module.

21. A system for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

an alignment module for simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching, and

a synthetic view generation block for warping said reference imagery to have a viewpoint similar to a viewpoint of a sensor that produced said input images.

22. A system for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

an alignment module for simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching,

a local matching module for aligning a plurality of said input images to one another; and

a global matching module for simultaneously aligning the aligned plurality of input images and the reference image.

23. A system for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

a reference database module containing geodetically calibrated reference imagery for producing reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

an alignment module for simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching, and

an outlier rejection module.

24. A method for performing geo-spatial registration of an input image and geodetically calibrated reference imagery comprising:

producing geodetically calibrated reference imagery relating to imagery in said input image, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth;

processing the input image and said reference imagery using an oriented filter; and

aligning said processed input image and said processed reference imagery by performing oriented matching.

25. The method of claim 24 wherein said input image is an image mosaic.

26. The method of claim 24 further comprising:

correlating said processed input image with said processed reference imagery;

removing inaccurate correlations between said preprocessed input image and said preprocessed reference image.

27. A method for performing geo-spatial registration of a sequence of input images and geodetically calibrated reference imagery comprising:

producing geodetically calibrated reference imagery relating to imagery in said input images, wherein said geodetically calibrated reference imagery comprises position information with respect to the Earth; and

simultaneously aligning a plurality of input images from said sequence of input images to said reference imagery by performing oriented matching.