Position/orientation measurement method, and position/orientation measurement apparatus

• US 7,092,109 B2
• Filed: 01/08/2004
• Issued: 08/15/2006
• Est. Priority Date: 01/10/2003
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Satoh, Kiyohide, Anabuki, Mahoro, Uchiyama, Shinji
• Primary Examiner(s): Rosenberger, Richard A.
• Application Number:
  US10/752,487
• Publication Number:
  US 20040176925A1
• Time in Patent Office: 950 Days
• Field of Search: 356/601, 356/611, 356/620, 348/139, 348/169, 702/153
• US Class Current: 356/620
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

A first error coordinate between the image coordinate of a first indicator, which is arranged on the real space and detected on a first image captured by a first image sensing unit, and the estimated image coordinate of the first indicator, which is estimated to be located on the first image in accordance with the position/orientation relationship between the first image sensing unit (with the position and orientation according to a previously calculated position/orientation parameter) and the first indicator, is calculated. On the other-hand, a second error coordinate between the image coordinate of a second indicator, which is arranged on the first image sensing unit and detected on a second image that includes the first image sensing unit, and the estimated image coordinate of the second indicator, which is estimated to be located on the second image in accordance with the position/orientation relationship between the first image sensing unit (with the position and orientation according to the position/orientation parameter), and the second indicator, is calculated. Using the first and second error coordinates, the position/orientation parameter is corrected.

Claims

1. A position/orientation measurement method for calculating a position and orientation of an object, comprising:

a subjective viewpoint image acquisition step of acquiring a first image obtained by capturing a scene by a subjective viewpoint image sensing unit attached to the object;

an objective viewpoint image acquisition step of acquiring a second image obtained by capturing an image of the object by an objective viewpoint image sensing unit;

a first detection step of detecting a feature value associated with an image coordinate of a first indicator arranged in the scene from the first image;

a second detection step of detecting a feature value associated with an image coordinate of a second indicator arranged on the object or the objective viewpoint image sensing unit from the second image; and

a position/orientation calculation step of calculating the position and orientation of the object using the feature value associated with the image coordinate of the first indicator detected in the first detection step and the feature value associated with the image coordinate of the second indicator detected in the second detection step.

2. The method according to claim 1, wherein the object is an image sensing device, which has the subjective viewpoint image sensing unit.

3. The method according to claim 1, wherein the objective viewpoint image sensing unit is fixed in the scene.

4. The method according to claim 1, wherein the objective viewpoint image sensing unit comprises a plurality of cameras.

5. The method according to claim 1, wherein the position/orientation calculation step comprises:

an estimation step of estimating a feature value associated with the image coordinate of the first indicator in the first image, and a feature value associated with the image coordinate of the second indicator in the second image, on the basis of estimated values of the position and orientation of the object; and

a correction step of correcting the estimated values of the position and orientation of the object to reduce errors between the feature values associated with the image coordinates of the first and second indicators detected in the first and second detection steps, and the feature values associated with the image coordinates of the first and second indicators estimated in the estimation step.

6. The method according to claim 1, wherein the feature value associated with the image coordinate of the first indicator includes an image coordinate or image coordinates of a plurality of points specified by the first indicator, and the feature value associated with the image coordinate of the second indicator includes an image coordinate or image coordinates of a plurality of points specified by the second indicator.

7. The method according to claim 6, wherein at least one indicator that allows to detect an image coordinate is set as each of the first and second indicators, and the total number of image coordinates detected in the first and second detection steps is not less than 3.

8. The method according to claim 6, wherein the position/orientation calculation step comprises:

a position calculation step of calculating information associated with the position of the object using the image coordinate of the second indicator detected in the second detection step; and

an orientation calculation step of calculating information associated with the orientation of the object using the image coordinate of the first indicator detected in the first detection step.

9. The method according to claim 8, wherein one indicator is set as the second indicator, and indicators which allow to detect not less than two image coordinates are set as the first indicator.

10. The method according to claim 1, wherein the feature value associated with the image coordinate of the second indicator detected in the second detection step is an image coordinate of the second indicator, and

the position/orientation calculation step includes a step of calculating a line that constrains a position of the second indicator on a three-dimensional space on the basis of the image coordinate of the second indicator detected in the second detection step and a position and orientation of the objective viewpoint image sensing unit, and calculating the position and orientation of the object on the basis of the feature value associated with the image coordinate of the first indicator detected in the first detection step under a constraint that the second indicator is present on the line.

11. The method according to claim 10, wherein the position/orientation calculation step includes a step of calculating the position and orientation of the object by calculating unknown values, which include a parameter that represents the position of the second indicator on the line and a parameter that represents the orientation of the object, on the basis of the feature value associated with the image coordinate of the first indicator detected in the first detection step.

12. A program for making a computer execute a position/orientation measurement method of claim 1.

13. A computer readable storage medium storing a program of claim 10.

14. A position/orientation measurement method of calculating a first position/orientation parameter of an object which is attached to a first image sensing unit used to capture an image of a real space, and has a known position/orientation relationship with the first image sensing unit, comprising:

a first image coordinate detection step of detecting an image coordinate of a first indicator arranged on the real space from a first image captured by the first image sensing unit;

a first estimated image coordinate detection step of calculating an estimated image coordinate of the first indicator in accordance with the position/orientation relationship between the first image sensing unit and the first indicator under an assumption that the position and orientation of the first image sensing unit follow a previously calculated second position/orientation parameter;

a first error coordinate calculation step of calculating a first error coordinate between the image coordinate detected in the first image coordinate detection step and the estimated image coordinate calculated in the first estimated image coordinate detection step on the first image;

a second image coordinate detection step of detecting an image coordinate of a second indicator arranged on the first image sensing unit or the object from a second image which is captured by a second image sensing unit and includes the object;

a second estimated image coordinate detection step of calculating an estimated image coordinate of the second indicator in accordance with the position/orientation relationship between the first image sensing unit and the second indicator under an assumption that the position and orientation of the first image sensing unit follow the previously calculated second position/orientation parameter;

a second error coordinate calculation step of calculating a second error coordinate between the image coordinate detected in the second image coordinate detection step and the estimated image coordinate calculated in the second estimated image coordinate detection step on the second image;

a correction value calculation step of calculating a correction value used to correct the previously calculated second position/orientation parameter using the first and second error coordinates;

a correction step of correcting the previously calculated second position/orientation parameter using the correction value; and

a parameter calculation step of calculating the first position/orientation parameter of the object, which has the known position/orientation relationship with the first image sensing unit, using the second position/orientation parameter corrected in the correction step.

15. The method according to claim 14, wherein the object is an image sensing device, which has the first image sensing unit.

16. A position/orientation measurement method of calculating a position and orientation of an image sensing device as an object to be measured, which captures an image of a scene, comprising:

a detection step of detecting a feature value associated with an image coordinate of a first indicator arranged in the scene from the image captured by the image sensing device;

a measurement step of measuring a three dimensional position of a second indicator arranged on the image sensing device; and

a position/orientation calculation step of calculating the position and orientation of the image sensing device using the feature value associated with the image coordinate of the first indicator detected in the detection step, and the three dimensional position of the second indicator measured in the measurement step.

17. A position/orientation measurement apparatus for calculating a position and orientation of an object, comprising:

subjective image sensing unit, attached to the object, adapted to capture an image of a scene;

objective viewpoint image sensing unit adapted to capture an image of the object;

first detection unit adapted to detect a feature value associated with an image coordinate of a first indicator arranged in the scene from a first image captured by said subjective viewpoint image sensing unit;

second detection unit adapted to detect a feature value associated with an image coordinate of a second indicator arranged on the object or said subjective viewpoint image sensing unit from a second image captured by said objective viewpoint image sensing unit; and

position/orientation calculation unit adapted to calculate the position and orientation of the object using the feature value associated with the image coordinate of the first indicator detected by said first detection unit, and the feature value associated with the image coordinate of the second indicator detected by said second detection unit.