System for mosaicing digital ortho-images

• US 20040057633A1
• Filed: 09/19/2002
• Published: 03/25/2004
• Est. Priority Date: 09/19/2002
• Status: Active Grant

• Current Assignee: VI Technologies, LLC (VI GP LLC (f/k/a M7 Visual Intelligence GP LLC))
• Original Assignee: M7 Visual Intelligence, LP
• Inventors: Smitherman, Chester L., Mai, Tuy Vu
• Granted Patent: US 6,928,194 B2
• Time in Patent Office: Days
• US Class Current: 382/284
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images

Abstract

The present invention provides a system for mosaicing multiple input images, captured by one or more remote sensors, into a seamless mosaic of an area of interest. Each set of input images captured by the remote sensors within a capture interval are ortho-rectified and mosaiced together into a composite image. Successive composite images, along a given flight path, are then mosaiced together to form a strip. Adjacent strips are then mosaiced together to form a final image of the area of interest.

Claims

1. A system for rendering multiple input images into a single composite image, comprising:

a first system for performing initial processing on each input image;

a second system for determining geographical boundaries of each input image, cooperatively engaged with the first system;

a third system for mapping each input image into the composite image with accurate geographical position, cooperatively engaged with the first and second systems;

a fourth system for balancing color of the input images mapped into the composite image, cooperatively engaged with the first, second and third systems; and

a fifth system for blending borders between adjacent input images mapped into the composite image, cooperatively engaged with the first, second, third and fourth systems.

2. The system of claim 1, wherein each system is implemented in a separate functional instance.

3. The system of claim 1, wherein a plurality of the systems are implemented in a single functional instance.

4. The system of claim 1, wherein each system is implemented in software.

5. The system of claim 1, wherein each system is implemented in hardware.

6. The system of claim 1, wherein the systems are implemented in a combination of hardware and software.

7. The system of claim 1, wherein the system for mapping each input image further comprises a super-sampling system.

8. The system of claim 1, wherein multiple image transformations are reduced to a single transformation matrix.

9. The system of claim 1, wherein the first system performs a white balancing process on the input images.

10. The system of claim 1, wherein the first system performs an anti-vignetting process on the input images.

11. The system of claim 1, further comprising a sixth system for controlling the exposure time of the input images based on image intensity, cooperatively engaged with the first, second, third, fourth and fifth systems.

12. The system of claim 11, wherein the system for controlling the exposure time of the input images based on image intensity is incorporated within the fourth system for balancing color of the input images.

13. The system of claim 11, wherein the system for controlling the exposure time of the input images based on image intensity focuses on green-dominant pixels.

14. A method of geographically aligning a plurality of input images of a target terrain collected from an imaging sensor, comprising the steps of:

determining the imaging sensor'"'"'s attitude with respect to a target terrain;

providing a primary input image and at least one secondary image;

aligning the primary input image with respect to the target terrain; and

aligning the secondary input image with respect to the primary input image.

15. A method of mosaicing two overlapping digital input images together to form an output image, comprising the steps of:

providing a reference image, comprising a number of pixels having certain intensity;

providing a secondary image, comprising a number of pixels having certain intensity, that overlaps the reference image in an overlap area; and

scaling pixel values of the secondary image within the overlap area such that the intensity of the secondary image pixels within the overlap area is equivalent to the intensity of the reference image pixels within the overlap area.

16. The method of claim 15, wherein the reference and secondary images are correlated to compute a balancing correlation matrix.

17. The method of claim 16, wherein separate balancing correlation matrices are computed for each color channel.

18. The method of claim 15, further comprising a blending process for smoothing image transition within the overlap area.

19. The method of claim 18, wherein the blending process assigns a relative weight, determinative of inclusion in the output image, to the pixels of the secondary image.

20. The method of claim 19, wherein the relative weight ranges between zero, indicating no inclusion, and unity, indicating full inclusion.

21. A method of maintaining a desired image intensity in an imaging system, comprising the steps of:

evaluating a target image captured by the imaging system to identify green-dominant pixels in the target image;

determining an average intensity of the target image based on the green-dominant pixels;

evaluating the difference between the average intensity and the desired image intensity; and

adjusting exposure time of the imaging system responsive to the difference between the average intensity and the desired image intensity.

22. A system for rendering multiple, partially-overlapping input images of a target terrain, taken successively along a flight line over the target terrain, into a seamless image strip of the target terrain, comprising:

a system for providing a reference image and a secondary image, having a partially overlapping area and a boundary area with the reference image;

a system for dividing the secondary image, along the boundary with the reference image, into segments, cooperatively engaged with the system for providing a reference image and a secondary image;

a system for computing a balancing correlation matrix for each such segment, cooperatively engaged with the system for providing a reference and secondary images and the system for dividing the boundary area into segments;

a system for modifying pixel values within each segment, along a gradient that starts at the boundary and terminates at a predetermined transition distance, by the balancing correlation matrix for that segment, cooperatively engaged with the system for providing a reference and secondary images, the system for dividing the boundary area into segments, and the system for computing the balancing correlation matrix; and

a system for feathering pixels at the boundary to eliminate any visible seam, cooperatively engaged with the other systems.

23. The system of claim 22, wherein the reference and secondary images are orthorectified composite images.

24. The system of claim 22 further comprising a system for controlling exposure time of the input images based on relative intensity of the input images, cooperatively engaged with the other systems.

25. The system of claim 22, wherein pixel values at the boundary of two adjacent segments are scaled based on the distance-weighted average of the two corresponding balancing correlation matrix to provide smooth transition from one segment to the next.

26. The system of claim 24, wherein the system for controlling the exposure time analyzes relative intensity using only green-dominant pixels.

27. A method of establishing a seam line between adjacent image strips that minimizes imaging effects of elevated features in the image strips, comprising the steps of:

selecting an initial seam line between the image strips;

dividing the initial seam line into small segments;

determining the position of an elevated feature in a segment; and

altering the route of the seam line in that segment based on the position of the elevated feature.

28. A method for rendering multiple, partially-overlapping input images of a target terrain into a seamless image mosaic of the target terrain, comprising the steps of:

normalizing the intensity of each input image to a desired mean and standard deviation;

determining a reference image and a secondary image, having a partially overlapping area and a boundary area with the reference image;

establishing a segmented seam line between the reference and secondary images that minimizes imaging effects of elevated features in those images;

dividing the secondary images, along the boundary with the reference image, into segments;

computing a balancing correlation matrix for each such segment;

modifying pixel values within each segment, along a gradient that starts at the boundary and terminates at a predetermined transition distance, by the balancing correlation matrix for that segment;

feathering pixels at the boundary to eliminate any visible seam.

29. The method of claim 28, wherein the secondary images are divided into segments along the boundary with the reference image corresponding to the segments of the seam line.

30. The method of claim 28, wherein pixel values at the boundary of two adjacent segments of the secondary image are scaled based on the distance-weighted average of the two corresponding balancing correlation matrices to provide smooth transition from one segment to the next.

31. The method of claim 28, wherein each input image is an image strip corresponding to a flight line.

32. A method of processing color input images to reduce bias caused by man-made structures or water bodies, comprising the steps of:

selecting green-dominant pixels from a first input image and computing an average intensity value thereof;

selecting green-dominant pixels from a second input image and computing an average intensity value thereof;

comparing the average intensity values of the first and second input images; and

processing the first or second input image responsive to the comparison.