Use of Overlap Areas to Optimize Bundle Adjustment

• US 20140375773A1
• Filed: 06/20/2013
• Published: 12/25/2014
• Est. Priority Date: 06/20/2013
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Weisenburger, Shawn, He, Shaojun
• Granted Patent: US 9,247,239 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof

Abstract

Novel tools and techniques for defining a search range for conjugate points in a set of images. In one technique, an intra-overlap area in two images can be used to define a search range in which a conjugate point can be found; in an aspect, this search range might be expressed as a distance range from a station at which one or both of the two images were captured. That distance range can be used to narrow the search range in an image captured from another station, substantially reducing both the computation time to identify a conjugate match and the likelihood of identifying an incorrect match.

Claims

1. A system, comprising:

an image capture system comprising a plurality of image sensors, the image capture system configured to;

when oriented at a first station, capture a first image with a first image sensor having a first center of projection;

when oriented at the first station, capture a second image with a second image sensor having a second center of projection;

when oriented at a second station, capture a third image with one of the plurality of image sensors having a third center of projection; and

a computer, comprising a processor and a non-transitory computer readable medium having a set of instructions executable by the processor to cause the computer to perform one or more operations, the set of instructions comprising;

instructions to determine an intra-overlap area in the first and second images, the intra-overlap area having a common field of view in the first and second images and being defined by an overlap between a field of view of the first sensor and a field of view of the second sensor;

instructions to determine an inter-overlap area in the first, second, and third images, the inter-overlap area having a common field of view in the first, second, third images;

instructions to identify a first coordinate location, in the first image, of a conjugate point captured in the intra-overlap area of the first and second images and in the inter-overlap area of the first, second, and third images;

instructions to identify a second coordinate location, in the second image, of the conjugate point;

instructions to compute an estimated distance range from the first station to the first conjugate point, based at least in part on the first and second coordinate locations, wherein the estimated distance range corresponds to a segment of the third image;

instructions to define a search area in the third image, based at least in part on the computed estimated distance range and relative orientations of the image capture system at the first and second stations;

instructions to analyze the search area to identify a third coordinate location, in the third image, of the conjugate point, wherein analysis of the third image is limited to analysis of the search area, and wherein limitation of the analysis substantially reduces a number of pixels in the third image analyzed by the computer; and

instructions to perform a bundle adjustment calculation to orient the third image relative to at least one of the first and second images, based at least in part on the first, second, and third coordinate locations of the conjugate point.

2. A method, comprising:

identifying, with a computer, a first coordinate location, in a first image, of a conjugate point;

identifying, with the computer, a second coordinate location, in a second image, of the conjugate point;

computing, with the computer, an estimated distance range from a center of projection of at least one of the first and second images to the conjugate point, based at least in part on the first and second coordinate locations;

defining, with the computer, a search area in a third image, based at least in part on the computed estimated distance range and relative orientations of the image capture system at the first and second stations; and

analyzing the search area, with the computer, to identify a third coordinate location, in a third image, of the conjugate point.

3. The method of claim 2, further comprising:

orienting an image capture system at a first station, the image capture system comprising a plurality of image sensors;

capturing the first image at the first station with a first image sensor;

capturing the second image at the first station with a second image sensor;

orienting the image capture system at a second station; and

capturing the third image at the second station with one of the plurality of image sensors.

4. The method of claim 3, further comprising:

identifying, based at least in part on the computed estimated distance range and relative orientations of the image capture system at the first and second stations, the third image from among a plurality of images captured at the second station.

5. The method of claim 3, wherein the plurality of image sensors are affixed radially around a central axis of the image capture system and are configured to capture a plurality of images when oriented at the first station, the plurality of images comprising the first image and the second image and collectively providing a panoramic mosaic of 360 degrees around the image capture system.

6. The method of claim 3, wherein the first image sensor is adjacent the second image sensor within the plurality of image sensors.

7. The method of claim 3, wherein the image capture system is calibrated such that the first image sensor has a known translation and orientation relative to the second image sensor.

8. The method of claim 3, wherein the first sensor has a first field of view when at the first station and the second image sensor has a second field of view when at the first station, and wherein the conjugate point is within an intra-overlap defined by an overlap between the first field of view and the second field of view.

9. The method of claim 8, wherein the overlap between the first field of view and the second field of view is between about 1 percent of each of the fields of view and about 50 percent of each of the fields of view.

10. The method of claim 8, wherein the overlap between the first field of view and the second field of view is between about 5 percent of each of the fields of view and about 20 percent of each of the fields of view.

11. The method of claim 8, wherein the overlap between the first field of view and the second field of view is between about 8 percent of each of the fields of view and about 12 percent of each of the fields of view.

12. The method of claim 3, wherein the computer is integrated with the image capture system.

13. The method of claim 3, wherein the computer is separate from the image capture system.

14. The method of claim 3, wherein the computer further comprises a communication interface configured to communicate with the image capture system.

15. The method of claim 3, wherein the communication interface provides wireless communication between the image capture system and the computer.

16. The method of claim 2, wherein the computed estimated distance range corresponds to a segment of the third image.

17. The method of claim 2, further comprising limiting analysis of the third image to analysis of the search area.

18. The method of claim 2, further comprising:

performing, with the computer, a bundle adjustment calculation to orient the third image relative to at least one of the first and second images, based at least in part on the first, second, and third coordinate locations of the conjugate point.

19. The method of claim 2, further comprising:

identifying, with the computer, an intra-overlap area in the first and second images, the intra-overlap area having a common field of view in the first and second images.

20. The method of claim 2, further comprising:

identifying, with the computer, an inter-overlap area in the first, second, and third images, the inter-overlap area having a common field of view in the first, second, third images.

21. The method of claim 2, further comprising:

orienting at least one of the images relative to a local or global coordinate system.

22. The method of claim 21, further comprising:

calculating a position, in the local or global coordinate system, of the conjugate point.

23. The method of claim 2, further comprising:

orienting at least one of the first and second images; and

defining a vertical dimension of the search area based at least in part on the computed distance range and an orientation of at least one of the first and second images.

24. The method of claim 2, wherein at least one of the images is a panoramic image taken from an image capture system at ground level.

25. The method of claim 2, wherein at least one of the images is an aerial image.

26. An apparatus, comprising:

a non-transitory computer readable medium having encoded thereon a set of instructions executable by one or more computers to perform one or more operations, the set of instructions comprising;

instructions to identify a first coordinate location, in a first image, of a conjugate point;

instructions to identify a second coordinate location, in a second image, of the conjugate point;

instructions to compute an estimated distance range from a center of projection of at least one of the first and second images to the first conjugate point, based at least in part on the first and second coordinate locations;

instructions to define a search area in a third image, based at least in part on the computed estimated distance range and relative orientations of the image capture system at the first and second stations; and

instructions to analyze the search area to identify a third coordinate location, in a third image, of the conjugate point.

27. A computer system, comprising:

one or more processors; and

a computer readable medium in communication with the one or more processors, the computer readable medium having encoded thereon a set of instructions executable by the computer system to perform one or more operations, the set of instructions comprising;

instructions to identify a first coordinate location, in a first image, of a conjugate point;

instructions to identify a second coordinate location, in a second image, of the conjugate point;

instructions to compute an estimated distance range from a center of projection of at least one of the first and second images to the first conjugate point, based at least in part on the first and second coordinate locations;

instructions to define a search area in a third image, based at least in part on the computed estimated distance range and relative orientations of the image capture system at the first and second stations; and

instructions to analyze the search area to identify a third coordinate location, in a third image, of the conjugate point.