Image processing device and method

• US 20070263924A1
• Filed: 05/10/2007
• Published: 11/15/2007
• Est. Priority Date: 05/10/2006
• Status: Active Grant

• Current Assignee: Topcon Corporation
• Original Assignee: Topcon Corporation
• Inventors: Kochi, Nobuo, Anai, Tetsuji, Otani, Hitoshi
• Granted Patent: US 7,747,151 B2
• Time in Patent Office: Days
• US Class Current: 382/154
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

An image processing device is provided that can precisely measure the photographing position or posture of a photographing device or the coordinates of an object based on sequentially changing photographed images. A series of sequentially photographed images are acquired, from which feature points are extracted. The feature points are tracked and correlated to each other. Stereo images are selected from the series of photographed images correlated, and subjected to an orientation and a 3D measurement. The error range of corresponding points obtained by the 3D measurement is calculated. Based on the calculated error range, it is determined whether or not the corresponding points are appropriate for 3D measurement. Those corresponding points determined as inappropriate are deleted. Another orientation and another 3D measurement are performed using those corresponding points excluding the deleted ones, thus improving the measurement precision.

Claims

1. An image processing device comprising:

a photographed image acquisition section for acquiring a series of sequentially photographed images of a relatively moving object where three or more adjacent images have an overlapping area with each other;

a feature extraction section for extracting feature points from any of the sequentially photographed images;

a feature point tracking section for tracking and correlating the feature points for the series of photographed images;

a stereo image selection section for selecting stereo images from the series of photographed images, the stereo images being a pair of images;

an orientation process/three-dimensional (3D) measurement section for performing an orientation and a 3D measurement using corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected by the stereo image selection section;

an error range calculation section for calculating an error range of 3D coordinates of the corresponding feature points obtained from the stereo images in the 3D measurement;

a corresponding point appropriateness determination section for determining whether or not the corresponding feature points are appropriate for 3D measurement based on the error range of the 3D coordinates of the corresponding feature points calculated by the error range calculation section; and

an erroneous corresponding point deletion section for deleting corresponding feature points determined as inappropriate by the corresponding point appropriateness determination section.

2. The image processing device as recited in claim 1,

wherein the orientation process/3D measurement section performs another orientation and another 3D measurement using those corresponding feature points excluding the corresponding feature points deleted by the erroneous corresponding point deletion section.

3. The image processing device as recited in claim 1,

wherein the corresponding point appropriateness determination section determines as inappropriate those corresponding feature points in the stereo images with the error range exceeding a predetermined threshold.

4. The image processing device as recited in claim 1, further comprising:

an allowable range setting section for setting an overlapping area of the error ranges of a plurality of different stereo images as an allowable range of the corresponding feature points,

wherein the corresponding point appropriateness determination section determines those corresponding feature points in the stereo images with an overlapping area between the error range and the allowable range as appropriate, and those without an overlapping area therebetween as inappropriate.

5. The image processing device as recited in claim 1, further comprising:

an allowable range setting section for setting an overlapping area of the error ranges of a plurality of different stereo images as an allowable range of the corresponding feature points,

wherein the corresponding point appropriateness determination section determines those corresponding feature points existing inside the allowable range as appropriate, and those existing outside the allowable range as inappropriate.

6. The image processing device as recited in claim 4,

wherein the allowable range setting section sets, as an allowable range of the corresponding feature points, an overlapping area of the error ranges in most of the stereo images or an overlapping area of the error ranges in a predetermined number or more of the stereo images.

7. The image processing device as recited in claim 6,

wherein a center of gravity of those corresponding feature points in the stereo images with an overlapping area with the allowable range is obtained using a weighting factor that is 1 or a decreasing function of the error range, and the center of gravity is determined as a positional coordinate of the corresponding feature points.

8. The image processing device as recited in claim 1,

wherein the orientation process/3D measurement section forms rectified images by rectifying the stereo images, and performs a 3D measurement based on the rectified images.

9. The image processing device as recited in claim 1,

wherein the error range is calculated as a function of a baseline length, a photographing distance, a focal length and a pixel size.

10. The image processing device as recited in claim 3, further comprising:

a storage section for storing data indicating an assumed error range in a measurement based on correspondence between the stereo images and the error range,

wherein the stereo image selection section selects stereo images with an error range not more than a predetermined threshold based on the error range data.

11. The image processing device as recited in claim 3, further comprising:

a storage section for storing data indicating an assumed error range in a measurement based on correspondence between the stereo images and the error range,

wherein the stereo image selection section selects stereo images with an error range within a predetermined range based on the error range data.

12. The image processing device as recited in claim 4, further comprising:

a storage section for storing data indicating an assumed error ranges in a measurement based on correspondence between the corresponding feature points and the allowable range,

wherein the stereo image selection section selects stereo images with an error range overlapping the allowable range based on the error range data.

13. The image processing device as recited in claim 1, further comprising:

a storage section for storing data of a baseline length satisfying required precision obtained using necessary measurement elements and measurement precision obtained by measurement, utilizing a relation between the baseline length of the stereo images and the measurement precision of the 3D coordinates,

wherein the stereo image selection section selects stereo images with a baseline length satisfying the required precision based on the baseline data.

14. The image processing device as recited in claim 1, further comprising:

a photographing position measurement section that can measure a photographing position without based on the photographed images.

15. The image processing device as recited in claim 1, further comprising:

a display device for presenting a stereoscopic two-dimensional display of the error range or the allowable range,

wherein the photographed image acquisition section includes a photographing device for photographing a series of photographed images.

16. A method for processing an image, comprising:

a photographed image acquisition step of acquiring a series of sequentially photographed images of a relatively moving object where three or more adjacent images have an overlapping area with each other;

a feature extraction step of extracting feature points from any of the sequentially photographed images;

a feature point tracking step of tracking and correlating the feature points for the series of photographed images;

a stereo image selection step of selecting stereo images from the series of photographed images, the stereo images being a pair of images;

a first orientation step of performing an orientation using corresponding feature points, the corresponding feature points being the correlated feature points in the stereo images selected in the stereo image selection step;

a first 3D measurement step of performing a 3D measurement using orientation results in the first orientation step;

an error range calculation step of calculating an error range of 3D coordinates of the corresponding feature points obtained from the stereo images in the 3D measurement;

a corresponding point appropriateness determination step of determining whether or not the corresponding feature points are appropriate for 3D measurement based on the error range of the 3D coordinates of the corresponding feature points calculated in the error range calculation step; and

an erroneous corresponding point deletion step of deleting those corresponding feature points determined as inappropriate in the corresponding point appropriateness determination step.

17. The method for processing an image as recited in claim 16, further comprising:

a second orientation step of performing another orientation using those corresponding feature points excluding the corresponding feature points deleted in the erroneous corresponding point deletion step; and

a second 3D measurement step of performing a 3D measurement using orientation results in the second orientation step.

18. A computer readable program for causing a computer to perform the method for processing an image as recited in claim 16.