Estimating device and estimation method

• US 9,652,861 B2
• Filed: 09/10/2015
• Issued: 05/16/2017
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Yamaguchi, Nobuyasu
• Primary Examiner(s): Strege, John
• Application Number:
  US14/850,298
• Publication Number:
  US 20160125617A1
• Time in Patent Office: 614 Days
• Field of Search: 382209, 382218, 382291, 382294
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

An estimation method executed by a computer includes: extracting, from an image, a plurality of characteristic points satisfying a certain requirement regarding changes in gray levels between the plurality of characteristic points and surrounding points; identifying, as map points from the plurality of characteristic points, characteristic points existing on planes, by excluding corners of an object depicted on the image; extracting, from another image, another plurality of characteristic points satisfying the certain requirement; executing matching of the another plurality of characteristic points with the map points based on a region including the map points; and estimating, based on results of the matching, a position and an orientation of an imaging device while the another image is captured.

Claims

1. An estimation method executed by a computer, the estimation method comprising:

extracting, from an image, a plurality of characteristic points satisfying a certain requirement regarding changes in gray levels between the plurality of characteristic points and surrounding points;

identifying, as map points from the plurality of characteristic points, characteristic points existing on planes, by excluding corners of an object depicted on the image;

extracting, from another image, another plurality of characteristic points satisfying the certain requirement;

executing matching of the another plurality of characteristic points with the map points based on a region including the map points; and

estimating, based on results of the matching, a position and an orientation of an imaging device while the another image is captured.

2. The estimation method according to claim 1, wherein

the identifying includes:

determining, for each of the plurality of characteristic points, whether or not a target characteristic point from among the plurality of characteristic points and adjacent characteristic points are distributed on a same plane in the image, and

identifying, for each of the plurality of characteristic points, the target characteristic point as a map point from among the map points when the target characteristic point and the adjacent characteristic points are distributed on the same plane, and

the adjacent characteristic points are located adjacent to the target characteristic point.

3. The estimation method according to claim 1, further comprising:

storing image data of the region including the map points into a storage unit, and

wherein the matching is executed using the image data stored in the storage unit when the another image is acquired.

4. The estimation method according to claim 1, further comprising:

identifying, as another map point, a characteristic point from among the plurality of characteristic points, the characteristic point existing at a corner of the object; and

storing the map points and the another map point into the storage unit while distinguishing the map points from the another map point.

5. The estimation method according to claim 4,

wherein the executing of the matching includes:

executing matching of the map points with the another plurality of characteristic points, and

executing another matching using the another map point when the number of successfully matched characteristic points that have been subjected to the matching is smaller than a certain value.

6. The estimation method according to claim 5, wherein

the another matching is executed based on priorities of another map points including the another map point, and

the priorities are calculated for the another map points based on a positional relationship between a first map point from among the another map points and a second map point which is separated by a certain distance or less from the first map point.

7. The estimation method according to claim 5, wherein the estimating is executed using a successfully matched characteristic point, from among the another plurality of characteristic points, which has been subjected to the matching or the another matching.

8. The estimation method according to claim 7, wherein the estimating is executed using a first group which succeeds in the matching and a second group which succeeds in the another matching, while a weight added to the first group is larger than another weight added to the second group.

9. The estimation method according to claim 3, wherein three-dimensional positions of the map points are acquired from the imaging device provided with a three-dimensional distance sensor.

10. The estimation method according to claim 1, wherein

the another image is captured following the image, and

the image is captured by the imaging device.

11. An estimating device comprising:

a memory; and

a processor coupled to the memory and configured to;

extract, from an image, a plurality of characteristic points satisfying a certain requirement regarding changes in gray levels between the plurality of characteristic points and surrounding points,

identifying, as map points from the plurality of characteristic points, characteristic points existing on planes, by excluding corners of an object depicted on the image,

extract, from another image, another plurality of characteristic points satisfying the certain requirement,

execute matching of the another plurality of characteristic points with the map points based on a region including the map points, and

estimate, based on results of the matching, a position and an orientation of an imaging device while the another image is captured.

12. The estimating device according to claim 11, wherein

the processor is configured to:

determine, for each of the plurality of characteristic points, whether or not a target characteristic point from among the plurality of characteristic points and adjacent characteristic points are distributed on a same plane in the image, and

identify, for each of the plurality of characteristic points, the target characteristic point as a map point from among the map points when the target characteristic point and the adjacent characteristic points are distributed on the same plane, and

the adjacent characteristic points are located adjacent to the target characteristic point.

13. The estimating device according to claim 11, wherein

the processor is configured to store image data of the region including the map points into a storage unit, and

the matching is executed using the image data stored in the storage unit when the another image is acquired.

14. The estimating device according to claim 11, wherein the processor is configured to:

identify, as another map point, a characteristic point from among the plurality of characteristic points, the characteristic point existing at a corner of the object, and

store the map points and the another map point into the storage unit while distinguishing the map points from the another map point.

15. The estimating device according to claim 14, wherein the processor is configured to:

execute matching of the map points with the another plurality of characteristic points, and

execute another matching using the another map point when the number of successfully matched characteristic points that have been subjected to the matching is smaller than a certain value.

16. The estimating device according to claim 15, wherein

the another matching is executed based on priorities of another map points including the another map point, and

the priorities are calculated for the another map points based on a positional relationship between a first map point from among the another map points and a second map point which is separated by a certain distance or less from the first map point.

17. The estimating device according to claim 15, wherein the processor is configured to estimate the position and the orientation of the imaging device using a successfully matched characteristic point, from among the another plurality of characteristic points, which has been subjected to the matching or the another matching.

18. The estimating device according to claim 17, wherein the processor is configured to estimate the position and the orientation of the imaging device using a first group which succeeds in the matching and a second group which succeeds in the another matching, while a weight added to the first group is larger than another weight added to the second group.

19. The estimating device according to claim 13, wherein three-dimensional positions of the map points are acquired from the imaging device provided with a three-dimensional distance sensor.

20. An estimating device configured to execute matching of a plurality of characteristic points extracted from an image with a plurality of map points and estimate a position and an orientation of a camera capturing the image, based on deviations of successfully matched characteristic points from among the characteristic points subjected to the matching from points obtained by projecting map points corresponding to the successfully matched characteristic points on the image, the estimating device comprising:

a memory; and

a processor coupled to the memory and configured to;

identify shapes of regions each having a certain range and including a certain map point from among the plurality of map points,

determine whether or not the shapes are planes,

adjust rates of causing the plurality of map points to contribute to an estimation of the position and the orientation of the camera, and

wherein the adjust rates of causing a map point included in a planar region to contribute to the estimation of the position and the orientation of the camera is larger than the adjust rate of causing another map point included in a non-planar region to contribute to the estimation of the position and the orientation of the camera.