Processing architecture for automatic image registration

• US 7,751,651 B2
• Filed: 04/02/2004
• Issued: 07/06/2010
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Oldroyd, Lawrence A.
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US10/817,476
• Publication Number:
  US 20050220363A1
• Time in Patent Office: 2,286 Days
• Field of Search: 382/103, 382/293, 382/294
• US Class Current: 382/294
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

An image registration method and apparatus for automatically registering images of different perspectives, and where a sensor image is registered with a more precise reference image such that the geocoding of the reference image can be transferred to the sensor image.

Claims

1. A method for automatically registering a sensed image with a reference image, comprising:

using a sensor mounted on a platform, acquiring an image of a scene and at least one sensor parameter that indicates a perspective of the sensor relative to the scene;

determining a footprint of the sensor relative to a coordinate system associated with a reference image database and a digital elevation database, the footprint including at least a portion of the scene in the sensor image;

accessing the digital elevation database to obtain a reference digital elevation model geocoded in the coordinate system and whose geographic extent encompasses an area that includes the sensor footprint;

accessing the reference image database to obtain a reference image geocoded in the coordinate system and that encompasses an area that includes the sensor footprint;

generating a synthetic 3-D model of an area around a center of the scene by combining the geocoded reference digital elevation model and the geocoded reference image such that each pixel of the geocoded reference image is associated with an elevation from the geocoded reference digital elevation model;

transforming the synthetic 3-D model into a synthetic perspective image of the scene, the transforming based on a type of the sensor and the at least one sensor parameter; and

registering the sensor image of the scene with the synthetic perspective image to geocode the sensor image using a mutual information model;

said method performed by one or more processors.

3. The method of claim 1, wherein the at least one sensor parameter further comprises a plurality of sensor parameters that include at least a field of view of the sensor, size of the image in pixel units, a resolution of the image, a focal length, an elevation angle and an azimuth angle.

4. The method of claim 3, wherein registering the sensor image of the scene with the synthetic perspective image further comprises:

determining a projection for the sensor image based on the type of the sensor and the plurality of sensor parameters; and

determining an inverse projection for the sensor image based on the type of the sensor and the plurality of sensor parameters.

5. The method of claim 1, further comprising:

determining a location of the sensor relative to the scene;

determining projection parameters for the perspective of the sensor relative to the scene based on the location of the sensor relative to the scene and the center of the scene; and

determining an inverse projection for the perspective of the sensor relative to the scene based on the location of the sensor relative to the scene and the center of the scene.

6. The method of claim 4, wherein accessing the reference image database further comprises:

retrieving reference data that includes a perspective of the reference image relative to the scene; and

retrieving a reference projection model for the reference image.

7. The method of claim 6, wherein transforming the synthetic 3-D model into the synthetic perspective image further comprises:

determining a projection of the reference image based on the reference data and the reference projection model; and

determining an inverse projection of the reference image based on the reference data and the reference projection model.

8. The method of claim 7, further comprising:

using the inverse projection of the reference image to remove projective distortion from the reference image.

9. The method of claim 7, further comprising:

transforming the synthetic 3-D model into an orthographic projection using the inverse projection of the reference image.

10. The method of claim 7, wherein the projection parameters for the perspective of the sensor define a sensor projection model, and the sensor projection model and the reference projection model are the same projection model.

11. The method of claim 7, wherein the projection parameters for the perspective of the sensor define a sensor projection model, and the sensor projection model and the reference projection model are different projection models.

12. The method of claim 7, wherein registering the sensor image of the scene with the synthetic perspective image further comprises:

determining a translation offset between the sensor image and the synthetic perspective image; and

matching pixels in the sensor image to pixels in the synthetic perspective image using the translation offset.

13. The method of claim 12, wherein matching pixels in the sensor image to pixels in the synthetic perspective image further comprises:

matching pixels in the sensor image to pixels in the synthetic perspective image using the inverse projection of the perspective of the sensor.

14. The method of claim 1, wherein registering the sensor image further comprises:

associating the geocode of a pixel in the reference image with the geocode of a pixel in the sensor image to geocode the sensor image.

2. The method of 1, wherein a type of the sensor further comprises at least one of the following:

a lens camera, and a synthetic aperture radar sensor.

15. A method for automatically registering a sensed image with a reference image, comprising:

using a sensor mounted on a platform, acquiring an image of a scene and at least one sensor parameter that indicates a perspective of the sensor relative to the scene;

determining a footprint of the sensor relative to a coordinate system associated with a reference image database, the footprint including at least a portion of the scene in the sensor image;

accessing the reference image database to obtain a reference image geocoded in the coordinate system and that includes a portion of a right stereo image and a portion of a left stereo image, which encompasses an area that includes the sensor footprint;

processing the left and right stereo images to derive a reference digital elevation model geocoded in the coordinate system and whose geographic extent encompasses an area that includes the sensor footprint;

generating a synthetic 3-D model of an area around a center of the scene by combining the geocoded reference digital elevation model and the geocoded reference image such that each pixel of the geocoded reference image is associated with an elevation from the geocoded reference digital elevation model;

transforming the synthetic 3-D model into a synthetic perspective image of the scene, the transforming based on a type of the sensor and the at least one sensor parameter; and

registering the sensor image of the scene with the synthetic perspective image to geocode the sensor image using a mutual information model;

said method performed by one or more processors.

16. The method of claim 15, wherein a type of the sensor further comprises one or more of the following:

a lens camera, and a synthetic aperture radar sensor.

17. The method of claim 15, wherein the at least one sensor parameter further comprises a plurality of sensor parameters that include at least a field of view of the sensor, size of the image in pixel units, a resolution of the image, a focal length, an elevation angle and an azimuth angle.

18. The method of claim 17, wherein registering the sensor image of the scene with the synthetic perspective image further comprises:

determining a projection for the sensor image based on the type of sensor and the plurality of sensor parameters; and

determining an inverse projection for the sensor image based on the type of the sensor and the plurality of sensor parameters.

19. The method of claim 15, further comprising:

determining a location of the sensor relative to the scene;

determining projection parameters for the perspective of the sensor relative to the scene based on the location of the sensor relative to the scene and the center of the scene; and

determining an inverse projection for the perspective of the sensor relative to the scene based on the location of the sensor relative to the scene and the center of the scene.

20. The method of claim 19, wherein accessing the reference image database further comprises:

retrieving reference data that includes a perspective of the reference image relative to the scene; and

retrieving a reference projection model for the reference image.

21. The method of claim 20, wherein transforming the synthetic 3-D model into the synthetic perspective image further comprises:

determining a projection of the reference image based on the reference data and the reference projection model; and

determining an inverse projection of the reference image based on the reference data and the reference projection model.

22. The method of claim 21, further comprising:

using the inverse projection of the reference image to remove projective distortion from the reference image; and

transforming the synthetic 3-D model into an orthographic projection using the inverse projection of the reference image.

23. The method of claim 21, wherein registering the sensor image of the scene with the synthetic perspective image further comprises:

determining a translation offset between the sensor image and the synthetic perspective image; and

matching pixels in the sensor image to pixels in the synthetic perspective image using the translation offset.

24. The method of claim 23, wherein matching pixels in the sensor image to pixels in the synthetic perspective image further comprises:

matching pixels in the sensor image to pixels in the synthetic perspective image using the inverse projection of the perspective of the sensor.

25. The method of claim 15, wherein registering the sensor image further comprises:

associating the geocode of a pixel in the reference image with the geocode of a pixel in the sensor image to geocode the sensor image.