Image processing device and method

• US 7,747,150 B2
• Filed: 04/03/2007
• Issued: 06/29/2010
• Est. Priority Date: 04/06/2006
• Status: Active Grant

• Current Assignee: Topcon Corporation
• Original Assignee: Topcon Corporation
• Inventors: Kochi, Nobuo, Anai, Tetsuji, Otani, Hitoshi
• Primary Examiner(s): Mahoney; Christopher E
• Assistant Examiner(s): PARKER, AUTUMN H
• Application Number:
  US11/730,747
• Publication Number:
  US 20070236561A1
• Time in Patent Office: 1,183 Days
• Field of Search: 396/55, 396/222, 396/324, 396/335, 382/103, 382/106, 382/154, 348/46, 348/50, 348/169, 348/172
• US Class Current: 396/55
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

An image processing device is provided that can precisely measure the photographing position or posture of a photographing device or the coordinates of an object based on sequentially changing photographed images. A series of sequentially photographed images are acquired, from which feature points are extracted. The feature points are tracked and correlated. Stereo images are selected from the series of photographed images correlated. Images selectable as stereo images are grouped into a group of images, from which stereo images are selected, before an orientation and a 3D measurement are performed. The images are grouped according to the distance from the object to the photographing position or the photographing magnification. Images with abrupt changes are removed so as to improve the measurement precision.

Claims

1. An image processing device comprising:

a photographed image acquisition section for acquiring a series of sequentially photographed images, extracted from frames of a moving image of a relatively moving object captured continuously, where four or more adjacent images have an overlapping area with each other;

a feature extraction section for extracting feature points from any of the sequentially photographed images;

a feature point tracking section for tracking and correlating the feature points for the series of photographed images;

a stereo image selection section for selecting stereo images from the series of photographed images, the stereo images being a pair of images, and

an orientation process/3D (three dimensional) measurement section for performing an orientation and a 3D measurement using corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected by the stereo image selection section,

wherein the stereo image selection section has an image group setting section for grouping images selectable as the stereo images into a group of images, and selects stereo images from the group of images determined as belonging to one group by the image group setting section,

wherein the image group setting section groups the series of photographed images into a plurality of groups of images according to a distance from the object to a photographing position as a group of images with the distance being approximately constant or according to a photographing magnification as a group of images with the photographing magnification being approximately constant, to prevent unstable precision,

wherein the stereo image selection section specifies, as inappropriate images, photographed images photographed while the distance from the object to the photographing position, the photographing magnification or a photographing direction is changing greatly, and excludes the specified inappropriate images when grouping the series of photographed images.

2. The image processing device as recited in claim 1, further comprising:

a photographing position measurement section for finding a position where the series of photographed images have been photographed,

wherein the stereo image selection section uses, as the photographing position, the position found by the photographing position measurement section.

3. The image processing device as recited in claim 1,

wherein the stereo image selection section has a correlation table defining a relationship between a baseline length of the stereo images and a measurement precision of 3D coordinates, and selects the stereo images having a baseline length satisfying a required precision based on the correlation table.

4. The image processing device as recited in claim 1, further comprising:

a sensor section for measuring a position, speed, acceleration, angle or angular speed,

wherein, when a change in distance from the object to a photographing position, a change in direction of an optical axis of a photographing device or a change in tilt of the photographing device are detected based on a change over time in position, speed, acceleration, angle or angular speed output from the sensor section, the stereo image selection section specifies, as inappropriate images, photographed images photographed while such changes are drastic, and selects the stereo images from photographed images excluding the specified inappropriate images.

5. The image processing device as recited in claim 1,

wherein the image group setting section groups the series of photographed images according to the distance from the object to a photographing position and the photographing magnification.

6. An image processing device comprising:

a photographed image acquisition section for acquiring a series of sequentially photographed images, extracted from frames of a moving image of a relatively moving object captured continuously, where four or more adjacent images have an overlapping area with each other;

a feature extraction section for extracting feature points from any of the sequentially photographed images;

a feature point tracking section for tracking and correlating the feature points for the series of photographed images;

a stereo image selection section for selecting stereo images from the series of photographed images; and

an orientation process/3D measurement section for performing an orientation and a 3D measurement using corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected by the stereo image selection section,

wherein the stereo image selection section specifies, as inappropriate images, photographed images photographed while a distance from the object to a photographing position, a photographing magnification or a photographing direction is changing greatly, and selects the stereo images from photographed images excluding the specified inappropriate images,

wherein the stereo image selection section has a correlation table defining a relationship between a baseline length of the stereo images and a measurement precision of 3D coordinates, and selects the stereo images having a baseline length satisfying a required precision based on the correlation table.

7. The image processing device as recited in claim 6, further comprising:

a sensor section for measuring a position, speed, acceleration, angle or angular speed,

wherein, when a change in distance from the object to a photographing position, a change in direction of an optical axis of a photographing device or a change in tilt of the photographing device are detected based on a change over time in position, speed, acceleration, angle or angular speed output from the sensor section, the stereo image selection section specifies, as inappropriate images, photographed images photographed while such changes are drastic, and selects the stereo images from photographed images excluding the specified inappropriate images.

8. An image processing method comprising:

a photographed image acquisition step of acquiring a series of sequentially photographed images, extracted from frames of a moving image of a relatively moving object captured continuously, where four or more adjacent images have an overlapping area with each other;

a feature extraction step of extracting feature points from any of the sequentially photographed images;

a feature point tracking step of tracking and correlating the feature points for the series of photographed images;

an image group setting step of grouping images from the series of photographed images into a group of images, the images to be grouped being selectable as stereo images;

a stereo image selection step of selecting the stereo images from the group of images determined as belonging to one group in the image group setting step;

an orientation step of performing an orientation using the corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected in the stereo image selection step; and

a 3D measurement step of performing a 3D measurement using orientation results in the orientation step,

wherein the image group setting step groups the series of photographed images into a plurality of groups of images according to a distance from the object to a photographing position as a group of images with the distance being approximately constant or according to a photographing magnification as a group of images with the photographing magnification being approximately constant, to prevent unstable precision,

wherein the stereo image selection step specifies, as inappropriate images, photographed images photographed while the distance from the object to the photographing position, the photographing magnification or a photographing direction is changing greatly, and excludes the specified inappropriate images when grouping the series of photographed images.

9. The image processing method as recited in claim 8,

wherein the image group setting step groups the series of photographed images according to the distance from the object to a photographing position and the photographing magnification.

10. An image processing method comprising:

a photographed image acquisition step of acquiring a series of sequentially photographed images, extracted from frames of a moving image of a relatively moving object captured continuously, where four or more adjacent images have an overlapping area with each other;

a feature extraction step of extracting feature points from any of the sequentially photographed images;

a feature point tracking step of tracking and correlating the feature points for the series of photographed images;

a stereo image selection step of selecting stereo images from the series of photographed images, wherein photographed images photographed while a distance from the object to a photographing position, a photographing magnification or a photographing direction is changing greatly are specified as inappropriate images, and stereo images are selected from photographed images excluding the specified inappropriate images;

an orientation step of performing an orientation using corresponding feature points, the corresponding feature points being the feature points correlated in the stereo images selected in the stereo image selection step; and

a 3D measurement step of performing a 3D measurement using orientation results in the orientation step,

wherein the stereo image selection step has a correlation table defining a relationship between a baseline length of the stereo images and a measurement precision of 3D coordinates, and selects the stereo images having a baseline length satisfying a required precision based on the correlation table.