Image processing device and method
First Claim
1. An image processing device comprising:
- a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually;
a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section;
a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and
a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points,wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors,wherein the characteristic point tracking section uses the photographed image from which the characteristic points have been extracted and a plurality of the photographed images sequentially acquired prior or subsequent thereto to search for candidate corresponding points corresponding to the characteristic points;
forms the movement vectors based on positional relationship between candidate corresponding points found and the characteristic points;
selects candidate corresponding points judged to probably be corresponding points corresponding to the characteristic points based on the movement vectors and the screen relative movement amount;
searches a plurality of the photographed images sequentially acquired prior or subsequent thereto, for candidate corresponding points corresponding to the characteristic points and the corresponding points; and
sequentially repeats forming movement vectors, selecting candidate corresponding points and searching for candidate corresponding points.
1 Assignment
0 Petitions
Accused Products
Abstract
A technique is provided that can precisely measure the photographing position and posture of a photographing device and the coordinates of an object from moving images or photographed images that sequentially change gradually, even in the case with the need to process the sway of the photographing device and the disappearance and reappearance of characteristic points. An image processing device 100 according to the present invention includes: a photographed image acquisition 2 section for sequentially acquiring photographed images, that change gradually, of a relatively moving object; a characteristic extraction section 3 for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section 4 for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section 5 for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points.
-
Citations
13 Claims
-
1. An image processing device comprising:
-
a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually; a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the characteristic point tracking section uses the photographed image from which the characteristic points have been extracted and a plurality of the photographed images sequentially acquired prior or subsequent thereto to search for candidate corresponding points corresponding to the characteristic points;
forms the movement vectors based on positional relationship between candidate corresponding points found and the characteristic points;
selects candidate corresponding points judged to probably be corresponding points corresponding to the characteristic points based on the movement vectors and the screen relative movement amount;
searches a plurality of the photographed images sequentially acquired prior or subsequent thereto, for candidate corresponding points corresponding to the characteristic points and the corresponding points; and
sequentially repeats forming movement vectors, selecting candidate corresponding points and searching for candidate corresponding points.
-
-
2. An image processing device comprising:
-
a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually; a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the characteristic point tracking section uses tracking results of respective characteristic points between successive photographed images to obtain movement vectors of the respective characteristic points and a screen relative movement amount;
compares the movement vectors of the respective characteristic points with estimated positions of the respective characteristic points estimated from the screen relative movement amount; and
eliminates those with large differences as failing points. - View Dependent Claims (3, 4)
-
-
5. An image processing device comprising:
-
a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually; a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the processing section detects disappearance of the characteristic points or the corresponding points; and
when a predetermined number of points have disappeared or a predetermined range of area or more has become vacant, commands the characteristic extraction section to create new characteristic points.
-
-
6. An image processing device comprising:
-
a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually; a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the processing section uses movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors of respective characteristic points of a first photographed image to create an estimated image related to a second photographed image acquired with a predetermined amount distanced therefrom (in time or space);
compares the second photographed image with the estimated image related thereto;
determines corresponding points from the candidate corresponding points; and
calculates coordinates of a photographing position or the object using the determined corresponding points. - View Dependent Claims (7, 8, 9)
-
-
10. An image processing method comprising:
-
a photographed image acquisition step of sequentially acquiring photographed images of a relatively moving object that change gradually; a characteristic extraction step of extracting characteristic points from a photographed image acquired in the photographed image acquisition step; a characteristic point tracking step of searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing step of determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the characteristic point tracking step comprises using the photographed image from which the characteristic points have been extracted and a plurality of the photographed images sequentially acquired prior or subsequent thereto to search for candidate corresponding points corresponding to the characteristic points;
forming the movement vectors based on positional relationship between candidate corresponding points found and the characteristic points;
selecting candidate corresponding points judged to probably be corresponding points corresponding to the characteristic points based on the movement vectors and the screen relative movement amount;
searching a plurality of the photographed images sequentially acquired prior or subsequent thereto, for candidate corresponding points corresponding to the characteristic points and the corresponding points; and
sequentially repeating forming movement vectors, selecting candidate corresponding points and searching for candidate corresponding points.
-
-
11. An image processing method comprising:
-
a photographed image acquisition step of sequentially acquiring photographed images of a relatively moving object that change gradually; a characteristic extraction step of extracting characteristic points from a photographed image acquired in the photographed image acquisition step; a characteristic point tracking step of searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing step of determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the characteristic point tracking step comprises using tracking results of respective characteristic points between successive photographed images to obtain movement vectors of the respective characteristic points and a screen relative movement amount;
comparing the movement vectors of the respective characteristic points with estimated positions of the respective characteristic points estimated from the screen relative movement amount; and
eliminating those with large differences as failing points.
-
-
12. An image processing method comprising:
-
a photographed image acquisition step of sequentially acquiring photographed images of a relatively moving object that change gradually; a characteristic extraction step of extracting characteristic points from a photographed image acquired in the photographed image acquisition step; a characteristic point tracking step of searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing step of determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the processing step comprises detecting disappearance of the characteristic points or the corresponding points; and
when a predetermined number of points have disappeared or a predetermined range of area or more has become vacant, creating new characteristic points.
-
-
13. An image processing method comprising:
-
a photographed image acquisition step of sequentially acquiring photographed images of a relatively moving object that change gradually; a characteristic extraction step of extracting characteristic points from a photographed image acquired in the photographed image acquisition step; a characteristic point tracking step of searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing step of determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the processing step comprises using movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors of respective characteristic points of a first photographed image to create an estimated image related to a second photographed image acquired with a predetermined amount distanced therefrom (in time or space);
comparing the second photographed image with the estimated image related thereto;
determining corresponding points from the candidate corresponding points; and
calculating coordinates of a photographing position or the object using the determined corresponding points.
-
Specification