Image processing device and method

• US 7,747,151 B2
• Filed: 05/10/2007
• Issued: 06/29/2010
• Est. Priority Date: 05/10/2006
• Status: Active Grant

• Current Assignee: Topcon Corporation
• Original Assignee: Topcon Corporation
• Inventors: Kochi, Nobuo, Anai, Tetsuji, Otani, Hitoshi
• Primary Examiner(s): Mahoney; Christopher E
• Assistant Examiner(s): PARKER, AUTUMN H
• Application Number:
  US11/798,190
• Publication Number:
  US 20070263924A1
• Time in Patent Office: 1,146 Days
• Field of Search: 396/55, 396/222, 396/324, 396/335, 382/103, 382/106, 382/154, 348/46, 348/50, 348/169, 348/172
• US Class Current: 396/55
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

An image processing device is provided that can precisely measure the photographing position or posture of a photographing device or the coordinates of an object based on sequentially changing photographed images. A series of sequentially photographed images are acquired, from which feature points are extracted. The feature points are tracked and correlated to each other. Stereo images are selected from the series of photographed images correlated, and subjected to an orientation and a 3D measurement. The error range of corresponding points obtained by the 3D measurement is calculated. Based on the calculated error range, it is determined whether or not the corresponding points are appropriate for 3D measurement. Those corresponding points determined as inappropriate are deleted. Another orientation and another 3D measurement are performed using those corresponding points excluding the deleted ones, thus improving the measurement precision.

Claims

1. An image processing device comprising:

a photographed image acquisition section for acquiring a series of sequentially photographed images of a relatively moving object where three or more adjacent images have an overlapping area with each other, the photographed image acquisition section including or communicating with a photographing device for photographing the series of photographed images;

a feature extraction section for extracting feature points from any of the sequentially photographed images;

a feature point tracking section for tracking and correlating the feature points for the series of photographed images;

a stereo image selection section for selecting stereo images from the series of photographed images, the stereo images being a pair of images;

an orientation process/three-dimensional (3D) measurement section for determining orientation of the photographing device using a corresponding feature points and for performing a 3D measurement to measure 3D coordinates of the corresponding feature points using the corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected by the stereo image selection section;

an error range calculation section for calculating an error range of 3D coordinates;

a corresponding point appropriateness determination section for determining whether or not the corresponding feature points are appropriate for 3D measurement based on the error range of the 3D coordinates of the corresponding feature points calculated by the error range calculation section;

an erroneous corresponding point deletion section for deleting corresponding feature points determined as inappropriate by the corresponding point appropriateness determination section; and

an allowable range setting section for setting an overlapping area of the three or more error ranges of the 3D coordinates, for an identical point on the object, obtained from different stereo images as an allowable range of the 3D coordinates;

wherein the corresponding point appropriateness determination section determines those corresponding feature points in the stereo images used to measure the 3D coordinates without an overlapping area between the error range and the allowable range as inappropriate,

wherein the error range is calculated as a function of a baseline length, a photographing distance, a focal length and a pixel size.

2. The image processing device as recited in claim 1,

wherein the orientation process/3D measurement section performs another orientation and another 3D measurement using those corresponding feature points excluding the corresponding feature points deleted by the erroneous corresponding point deletion section.

3. The image processing device as recited in claim 1,

wherein the corresponding point appropriateness determination section determines as inappropriate those corresponding feature points in the stereo images used to measure the 3D coordinates with the error range exceeding a predetermined threshold.

4. The image processing device as recited in claim 3, further comprising:

a storage section for storing data indicating an assumed error range in a measurement based on correspondence between the stereo images and the error range,

wherein the stereo image selection section selects stereo images with an error range not more than a predetermined threshold based on the error range data.

5. The image processing device as recited in claim 3, further comprising:

a storage section for storing data indicating an assumed error range in a measurement based on correspondence between the stereo images and the error range,

wherein the stereo image selection section selects stereo images with an error range within a predetermined range based on the error range data.

6. The image processing device as recited in claim 1,

wherein the corresponding point appropriateness determination section determines those corresponding feature points used to measure the 3D coordinates existing inside the allowable range as appropriate, and those used to measure the 3D coordinates existing outside the allowable range as inappropriate.

7. The image processing device as recited in claim 1,

wherein the allowable range setting section sets, as an allowable range of the 3D coordinates, an overlapping area of the error ranges in most of the 3D coordinates obtained from the stereo images or an overlapping area of the error ranges in a predetermined number or more of the 3D coordinates obtained from the stereo images.

8. The image processing device as recited in claim 7,

wherein a center of gravity of the 3D coordinates with an overlapping area between the allowable range and the error range is obtained using a weighting factor that is 1 or a decreasing function of the error range, and the center of gravity is determined as a positional coordinate of the 3D coordinates.

9. The image processing device as recited in claim 1,

wherein the orientation process/3D measurement section forms rectified images by rectifying the stereo images, and performs a 3D measurement based on the rectified images.

10. The image processing device as recited in claim 1, further comprising:

a storage section for storing data indicating an assumed error ranges in a measurement based on correspondence between the corresponding feature points and the allowable range,

wherein the stereo image selection section selects stereo images with an error range overlapping the allowable range based on the error range data.

11. The image processing device as recited in claim 1, further comprising:

a storage section for storing data of a baseline length satisfying required precision obtained using necessary measurement elements and measurement precision obtained by measurement, utilizing a relation between the baseline length of the stereo images and the measurement precision of the 3D coordinates,

wherein the stereo image selection section selects stereo images with a baseline length satisfying the required precision based on the baseline data.

12. The image processing device as recited in claim 1, further comprising:

a photographing position measurement section that can measure a photographing position without being based on the photographed images.

13. The image processing device as recited in claim 1, further comprising:

a display device for presenting a stereoscopic two-dimensional display of the error range or the allowable range.

14. A method for processing an image, comprising:

a photographed image acquisition step of acquiring, by means of a photographing device, a series of sequentially photographed images of a relatively moving object where three or more adjacent images have an overlapping area with each other;

a feature extraction step of extracting feature points from any of the sequentially photographed images;

a feature point tracking step of tracking and correlating the feature points for the series of photographed images;

a stereo image selection step of selecting stereo images from the series of photographed images, the stereo images being a pair of images;

a first orientation step of an determining orientation of the photographing device using corresponding feature points, the corresponding feature points being the correlated feature points in the stereo images selected in the stereo image selection step;

a first 3D measurement step of performing a 3D measurement to measure 3D coordinates of the corresponding feature points using orientation results in the first orientation step;

an error range calculation step of calculating an error range of 3D coordinates;

a corresponding point appropriateness determination step of determining whether or not the corresponding feature points are appropriate for 3D measurement based on the error range of the 3D coordinates of the corresponding feature points calculated in the error range calculation step;

an erroneous corresponding point deletion step of deleting those corresponding feature points determined as inappropriate in the corresponding point appropriateness determination step; and

an allowable range setting step of setting an overlapping area of the three or more error ranges of the 3D coordinates, for an identical point on the object, obtained from different stereo images as an allowable range of the 3D coordinates;

wherein the corresponding point appropriateness determination step determines those corresponding feature points in the stereo images used to measure the 3D coordinates without an overlapping area between the error range and the allowable range as inappropriate; and

wherein the error range is calculated as a function of a baseline length, a photographing distance, a focal length and a pixel size.

15. The method for processing an image as recited in claim 14, further comprising:

a second orientation step of performing another orientation using those corresponding feature points excluding the corresponding feature points deleted in the erroneous corresponding point deletion step; and

a second 3D measurement step of performing a 3D measurement using orientation results in the second orientation step.