SYSTEM AND METHOD FOR AUTOMATIC STEREO MEASUREMENT OF A POINT OF INTEREST IN A SCENE

• US 20110007948A1
• Filed: 08/09/2010
• Published: 01/13/2011
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Oldroyd, Lawrence A.
• Granted Patent: US 8,107,722 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A system for performing a three dimensional stereo measurement that uses a sensor for obtaining a sensor image of a scene, and a database for providing first and second reference images of the scene that are a stereo pair of images. At least one processing system is responsive to an output of the sensor and in communication with the database. The processing system registers the sensor image with the first reference image, and also selects a point of interest from one of the sensor image and the first reference image. The processing system performs a stereo point measurement from the selected point of interest and the first reference image to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image.

Claims

1. A system for performing three dimensional, stereo measurement, the system comprising:

a sensor for obtaining a sensor image of a scene;

a database for providing first and second reference images of the scene that are a stereo pair of images;

at least one processing system responsive to an output of the sensor and in communication with the database, and operable to;

register the sensor image with the first reference image;

select a point of interest from the sensor image;

derive a point of interest in the first reference image from a selected point of interest in the sensor image and a registration between the sensor image and the first reference image;

perform a stereo point measurement from the selected point of interest and the first reference image to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image; and

modify the stereo point measurement using stereo evaluation to select an adjacent three dimensional point which more accurately corresponds with the selected point in the sensor image;

wherein said three dimensional stereo measurement of the selected point of interest in the sensor image is in scene coordinates.

2. The system of claim 1, further comprising:

a display for displaying both of said first and second reference images with said points marked on each of said reference images to enable an individual to visually confirm an accuracy of said stereo point measurement operation.

3. The system of claim 1, wherein:

the first and second reference images are each comprised of a plurality of pixels; and

wherein the selected point of interest in the first reference image is defined by a first pixel, and the point in the second reference image comprises a second pixel that is the stereo mate in the second reference image of the first pixel.

4. The system of claim 1, wherein:

the first reference image comprises a plurality of pixels;

the second reference image comprises a plurality of pixels;

the first reference image is provided with a corresponding elevation model; and

coordinates are associated with each said pixel in the first reference image.

5. The system of claim 3, wherein:

the selected point of interest of the first reference image is used by the processing system to obtain coordinate data that corresponds to the pixel of the selected point in the first reference image; and

the point in the second reference image is used by the processing system to obtain coordinate data therefor.

6. The system of claim 3, wherein the processing system is configured to perform a stereo point measurement.

7. The system of claim 6, wherein the stereo point measurement is accomplished by:

obtaining a first sub-image of the first reference image in an area circumscribing the selected point in the first reference image;

obtaining a second sub-image of the second reference image in an area circumscribing the selected point in the second reference image, and where the second sub-image is larger than the first sub-image;

overlaying at least a portion of the first sub-image over the second sub-image;

matching the two sub-images to determine a first offset of the first sub-image within the second image, and a second offset of the second sub-image relative to the second reference image;

using the first and second offsets to translate coordinates of the first pixel to stereo matching coordinates of the second pixel, and

wherein the stereo matching coordinates of the second pixel represent a specific point on the second reference image that matches corresponds to the location of the first pixel on the first reference image.

8. The system of claim 7, wherein the processing system is configured to obtain the stereo matching coordinates of the second pixel by:

determining column and row offset values that represent the first offset of the first sub-image within the second sub-image;

determining column and row offset values that represent the second offset of the second sub-image relative to the second reference image;

adding column and row coordinates of the first pixel in the first sub-image to the column and row offset values of the first sub-image offset to produce a modified first sub-image offset having associated column and row offset values; and

adding the modified first sub-image column and row offset values to the column and row offset values of the second sub-image, to produce the coordinates of the second pixel.

9. The system of claim 3, wherein the processing system is configured to:

define a three dimensional grid of points with the three dimensional measurement of the selected point of interest at a volumetric center of the three dimensional grid of points;

process each one of said three dimensional grid of points using the parametric data for said first and second reference images to identify a subset of the grid points that process to derive the location of the first pixel within the first reference image;

use the subset of said grid points to derive a location of a corresponding, stereo matching pixel in the second reference image;

determine a distance between the derived location of said corresponding, stereo matching pixel and the second pixel, within the second reference image; and

repeat the processing of each one of the three dimensional grid of points and using the subset of the grid points to derive a location of a corresponding, stereo matching pair, to determine a specific three dimensional grid point that, when processed using the parametric data, yields a location for a specific matching pixel that has a minimum distance from the second pixel, and denoting a three dimensional grid point of the specific matching pixel as an optimum three dimensional stereo coordinate location for the selected point of interest in the sensor image.

21. The system of claim 1, wherein:

the selected point of interest may be chosen from one of the sensor image and the first reference image;

the selected point of interest in the first reference image is derived from the selected point of interest in the sensor image and the registration between the sensor image and the first reference image; and

the selected point of interest in the sensor image is derived from the selected point of interest in the first reference image and the registration between the sensor image and the first reference image.

10. A system for performing three dimensional, stereo measurement, comprising:

a sensor that obtains a sensor image of a scene;

at least one database for providing first and second reference images of the scene that are a stereo pair of images;

a processing system in communication with the at least one database configured to;

register the sensor image with the first reference image;

select a point of interest from one of the sensor image and the first reference image;

perform a stereo point measurement from the selected point of interest and the first reference image to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image;

define a three dimensional grid;

analyze parametric data from the first and second reference images, the selected points of interest in the first and second reference images, and each three dimensional point within the grid, to determine three dimensional coordinates of a calculated point within the second reference image that is a minimum distance from the stereo mate point in the second reference image; and

denote the calculated point as an optimum stereo coordinate measurement of the point in the second reference image.

11. The system of claim 10, further comprising a display device.

12. The system of claim 11, wherein the display device displays both of the first and second reference images with the points marked on each of the reference images to enable an individual to visually confirm an accuracy of the stereo point measurement operation.

13. The system of claim 10, wherein the first and second reference images are each comprised of a plurality of pixels.

15. The system of claim 10, wherein:

the first reference image includes a plurality of pixels;

the second reference image includes a plurality of pixels;

the first reference image includes a corresponding elevation model; and

the processing system associates coordinates with each said pixel in the first reference image.

16. The system of claim 12, further comprising:

using the selected point of interest of the first reference image to obtain a coordinate data that corresponds to the pixel of the selected point in the first reference image; and

using the point in the second reference image to obtain coordinate data therefor.

17. The system of claim 10, wherein the processing system performs a stereo point measurement by:

obtaining a first sub-image of the first reference image in an area circumscribing the selected point in the first reference image;

obtaining a second sub-image of the second reference image in an area circumscribing the selected point in the second reference image, and where the second sub-image is larger than the first sub-image;

overlaying at least a portion of the first sub-image over the second sub-image;

analyzing the two sub-images to determine a first offset of the first sub-image within the second image, and a second offset of the second sub-image relative to the second reference image; and

using the first and second offsets to translate the coordinates of the first pixel to stereo matching coordinates of the second pixel.

18. The system of claim 17, wherein the stereo matching coordinates of the second pixel represent a specific point on the second reference image that matches the location of the first pixel on the first reference image.

14. The system of claim 14, wherein the processing system determines that the selected point of interest in the first reference image is defined by a first pixel, and the point in the second reference image comprises a second pixel that is a stereo mate in the second reference image of the first pixel.

19. A system for performing three dimensional, stereo measurement, comprising:

a sensor carried on an airborne mobile platform for generating an image of a scene;

at least one database for providing;

a first reference image of the scene, with the scene being comprised of a plurality of pixels; and

a second reference image of the scene that is comprised of a plurality of pixels, the first and second reference images of the scene comprising a stereo pair of images;

a processing system operable to;

register the sensor image with the first reference image;

select a first pixel representing a point of interest from one of the sensor image and the first reference image;

perform a stereo point measurement using the first pixel to determine a second pixel in the second reference image that represents a stereo mate of the first pixel in the first reference image;

define a three dimensional grid around a portion of the first reference image;

use said three dimensional grid and the first and second pixels to determine three dimensional coordinates of a calculated point in the second reference image that is closest to the stereo mate point in the second reference image; and

denote the calculated point as an optimum stereo coordinate measurement of the point in the second reference image.

20. The system of claim 19, wherein the processing system using the three dimensional grid comprises the processing system:

using an iterative examination of partial derivatives of parametric equations used to calculate pixel locations in the first and second reference images from each of the three dimensional coordinates; and

analyzing individual coordinate axes of the three dimensional grid, using information from said partial derivatives, to determine a specific location of a particular point within the three dimensional grid that is closest to the stereo mate point in the second reference image.