METHOD FOR AUTOMATIC STEREO MEASUREMENT OF A POINT OF INTEREST IN A SCENE

• US 20070127101A1
• Filed: 02/06/2007
• Published: 06/07/2007
• Est. Priority Date: 04/02/2004
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Oldroyd, Lawrence
• Granted Patent: US 7,773,799 B2
• Time in Patent Office: Days
• US Class Current: 359/24
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

A method for performing automatic stereo measure of a selected point in a scene. A sensor image is initially obtained from a sensor providing an image of a scene. First and second reference images of the scene that are a stereo pair of images are also provided. The sensor image is registered with the first reference image and a point of interest is selected from one of the sensor or first reference images. A stereo point measurement using the selected point and the two reference images is performed to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image. An analysis of a three dimensional grid volume centered about the selected point in the first reference image is then performed. The analysis uses both reference images, the selected point and its stereo mate to determine the three dimensional coordinates of the point in the grid that best matches the location of the stereo mate point in the second reference image.

Claims

1. A method for performing three dimensional, stereo measurement, comprising:

obtaining a sensor image from a sensor providing an image of a scene;

providing first and second reference images of the scene that are a stereo pair of images;

registering the sensor image with the first reference image;

selecting a point of interest from one of the sensor image and the first reference image; and

performing a stereo point measurement from the selected point of interest and the first reference image to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image.

2. The method of claim 1, further comprising:

displaying both of said first and second reference images on a display with said points marked on each of said reference images to enable an individual to visually confirm an accuracy of said stereo point measurement operation.

3. The method of claim 1, wherein:

the operation of providing first and second reference images comprises providing first and second reference images that are each comprised of a plurality of pixels; and

where the selected point of interest in the first reference image is defined by a first pixel, and the point in the second reference image comprises a second pixel that is the stereo mate in the second reference image of the first pixel.

4. The method of claim 1, wherein the operation of providing first and second reference images comprises:

providing a first reference image having a plurality of pixels;

providing a second reference image having a plurality of pixels;

providing said first reference image with a corresponding elevation model; and

associating coordinates with each said pixel in the first reference image.

5. The method of claim 3, further comprising:

using the selected point of interest of the first reference image to obtain a coordinate data that corresponds to the pixel of the selected point in the first reference image; and

using the point in the second reference image to obtain coordinate data therefor.

6. The method of claim 3, wherein the operation of performing a stereo point measurement comprises:

a) obtaining a first sub-image of the first reference image in an area circumscribing the selected point in the first reference image; and

b) obtaining a second sub-image of the second reference image in an area circumscribing the selected point in the second reference image, and where the second sub-image is larger than the first sub-image;

c) overlaying at least a portion of the first sub-image over the second sub-image;

d) matching the two sub-images using an image matching process to determine a first offset of the first sub-image within the second image, and a second offset of the second sub-image relative to the second reference image;

e) using the first and second offsets to translate the coordinates of the first pixel to the stereo matching coordinates of the second pixel, and

f) wherein the stereo matching coordinates of the second pixel represent a specific point on the second reference image that matches the location of the first pixel on the first reference image.

7. The method of claim 6, wherein operation f) comprises:

determining column and row offset values that represent the first offset of the first sub-image within the second sub-image;

determining column and row offset values that represent the second offset of the second sub-image relative to the second reference image;

adding column and row coordinates of the first pixel in the first sub-image to said column and row offset values of the first sub-image offset to produce a modified first sub-image offset having associated column and row offset values; and

adding the modified first sub-image column and row offset values to said column and row offset values of the second sub-image, to produce the coordinates of the second pixel.

8. The method of claim 3, further comprising:

a) defining a three dimensional grid of points with said selected point at a volumetric center of the three dimensional grid of points;

b) processing each one of said three dimensional grid of points using said parametric data for said first and second reference images to identify a subset of said grid points that process to derive the location of said first pixel within said first reference image;

c) using said subset of said grid points to derive a location of a corresponding, stereo matching pixel in said second reference image;

d) determining a distance between said derived location of said corresponding, stereo matching pixel and said second pixel, within said second reference image; and

e) repeating operations c) and d) to determine a specific three dimensional grid location that, when processed using said parametric data, yields a location for a specific matching pixel that has a minimum distance from said second pixel, and denoting a location of said specific matching pixel as an optimum three dimensional stereo coordinate location within said second reference image that corresponds to said first pixel in said first reference image.

9. A method for performing three dimensional, stereo measurement, comprising:

obtaining a sensor image from a sensor providing an image of a scene;

providing first and second reference images of the scene that are a stereo pair of images;

registering the sensor image with the first reference image;

selecting a point of interest from one of the sensor image and the first reference image;

performing a stereo point measurement from the selected point of interest and the first reference image to determine a point in the second reference image that represents a stereo mate of the selected point in the first reference image;

defining a three dimensional grid; and

using parametric data from the selected points of interest in the first and second reference images, and each three dimensional point within said grid, to determine three dimensional coordinates of a calculated point within said second reference image that is a minimum distance from said stereo mate point in the second reference image; and

denoting said calculated point as the optimum stereo coordinate measurement of said point in said second reference image.

10. The method of claim 9, further comprising displaying both of said first and second reference images on a display with said points marked on each of said reference images to enable an individual to visually confirm an accuracy of said stereo point measurement operation.

11. The method of claim 9, wherein:

the operation of providing first and second reference images comprises providing first and second reference images that are each comprised of a plurality of pixels; and

where the selected point of interest in the first reference image is defined by a first pixel, and the point in the second reference image comprises a second pixel that is the stereo mate in the second reference image of the first pixel.

12. The method of claim 9, wherein the operation of providing first and second reference images comprises:

providing a first reference image having a plurality of pixels;

providing a second reference image having a plurality of pixels;

providing said first reference image with a corresponding elevation model; and

associating coordinates with each said pixel in the first reference image.

13. The method of claim 11, further comprising:

using the selected point of interest of the first reference image to obtain a coordinate data that corresponds to the pixel of the selected point in the first reference image; and

using the point in the second reference image to obtain coordinate data therefor.

14. The method of claim 11, wherein the operation of performing a stereo point measurement comprises:

a) obtaining a first sub-image of the first reference image in an area circumscribing the selected point in the first reference image; and

b) obtaining a second sub-image of the second reference image in an area circumscribing the selected point in the second reference image, and where the second sub-image is larger than the first sub-image;

c) overlaying at least a portion of the first sub-image over the second sub-image;

d) matching the two sub-images using an image matching process to determine a first offset of the first sub-image within the second image, and a second offset of the second sub-image relative to the second reference image; and

e) using the first and second offsets to translate the coordinates of the first pixel to the stereo matching coordinates of the second pixel, and

f) wherein the stereo matching coordinates of the second pixel represent a specific point on the second reference image that matches the location of the first pixel on the first reference image.

15. The method of claim 14, wherein operation f) comprises:

determining column and row offset values that represent the first offset of the first sub-image within the second sub-image;

determining column and row offset values that represent the second offset of the second sub-image relative to the second reference image;

adding column and row coordinates of the first pixel in the first sub-image to said column and row offset values of the first sub-image offset to produce a modified first sub-image offset having associated column and row offset values; and

adding the modified first sub-image column and row offset values to said column and row offset values of the second sub-image, to produce the coordinates of the second pixel.

16. The method of claim 11, wherein using said parametric data further comprises:

defining said three dimensional grid of points with said selected point of said first reference image placed at an approximate volumetric center of the three dimensional grid of points.

17. The method of claim 16, further comprising:

a) processing each one of said three dimensional grid of points using said parametric data for said first and second reference images to identify a subset of said grid points that process to derive the location of said first pixel within said first reference image; and

b) using said subset of said grid points to derive a location of a corresponding, stereo matching pixel in said second reference image.

18. The method of claim 17, further comprising:

c) determining a distance between said derived location of said corresponding, stereo matching pixel and said second pixel, within said second reference image; and

d) repeating operations b) and c) to determine a specific three dimensional grid location that, when processed using said parametric data, yields a location for a specific matching pixel that has a minimum distance from said second pixel; and

e) denoting a location of said specific matching pixel as an optimum three dimensional stereo coordinate location within said second reference image that corresponds to said first pixel in said first reference image.

19. A method for performing three dimensional, stereo measurement, comprising:

obtaining a sensor image from a sensor providing an image of a scene;

providing a first reference image of said scene that is comprised of a plurality of pixels;

providing a second reference image of said scene that is comprised of a plurality of pixels, the first and second reference images of the scene comprising a stereo pair of images;

registering the sensor image with the first reference image;

selecting a first pixel representing a point of interest from one of the sensor image and the first reference image;

performing a stereo point measurement using the first pixel to determine a second pixel in the second reference image that represents a stereo mate of the first pixel in the first reference image;

defining a three dimensional grid around a portion of said first reference image;

using said three dimensional grid and said first and second pixels to determine three dimensional coordinates of a point in said second reference image that is closest to said stereo mate point in the second reference image; and

denoting said calculated point as the optimum stereo coordinate measurement of said point in said second reference image.

20. The method of claim 19, wherein using said three dimensional grid comprises:

using an iterative examination of partial derivatives of parametric equations used to calculate pixel locations in said first and second reference images from each of said three dimensional coordinates;

analyzing individual coordinate axes of said three dimensional grid, using information from said partial derivatives, to determine a specific location of a particular point within said three dimensional grid that is closest to said stereo mate point in the second reference image.