Position/orientation measurement method and apparatus

• US 8,073,201 B2
• Filed: 02/03/2006
• Issued: 12/06/2011
• Est. Priority Date: 02/04/2005
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Satoh, Kiyohide, Uchiyama, Shinji, Endo, Takaaki, Morita, Kenji
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US11/815,540
• Publication Number:
  US 20090022369A1
• Time in Patent Office: 2,132 Days
• Field of Search: 382/100, 382/106, 382/107, 382/108, 382/123, 382/168, 382/180, 382/181, 382191-203, 382/206, 382/232, 382/254, 382/274, 382/275, 382/276, 382286-297, 382/305, 382/312, 386/343, 386/220, 715/863, 345/1.3
• US Class Current: 382/106
• CPC Class Codes: G01B 21/04 by measuring coordinates of points
• CPC Class Codes: G01B 21/22 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

This invention relates to a position/orientation measurement apparatus which can measure a position and orientation while achieving both high stability and precision. An image including indices laid out on a space is captured, and the indices are detected from the captured image. When a plurality of indices are detected, their distribution range is calculated, and an algorithm to be applied in position/orientation calculations is selected according to the size of the range (S3033, S3050, S3060). For example, when the indices are distributed over a sufficiently broad range, six parameters of the position and orientation of an image capture device are calculated as unknowns (S3070). As the distribution range of the indices becomes smaller, the degrees of freedom of unknown parameters to be calculated are reduced (S3035, S3025).

Claims

1. A position/orientation measurement method for using at least one processor to measure a position and an orientation of a measurement target object, characterized by comprising:

a captured image acquisition step of acquiring a captured image;

an index detection step of detecting indices included in the captured image;

a distribution range calculation step of calculating an evaluation value associated with a distribution range of the indices detected in the index detection step; and

a position/orientation calculation step of calculating the position and the orientation of the measurement target object using information associated with image coordinates of the detected indices, and the evaluation value associated with the distribution range,

wherein in the position/orientation calculation step, a first position/orientation and a second position/orientation of the measurement target

object are calculated using different methods by exploiting the information associated with the image coordinates of the indices detected in the index detection step.

2. The position/orientation measurement method according to claim 1, characterized in that the measurement target object is an image capture device, and

the captured image is an image captured by the image capture device.

3. The position/orientation measurement method according to claim 1, characterized in that the captured image is an image captured by an image capture device, which has a known relative positional relationship with the measurement target object.

4. The position/orientation measurement method according to claim 1, characterized in that the captured image is an image captured by an image capture device which captures the measurement target object.

5. The position/orientation measurement method according to claim 1, characterized in that in the position/orientation calculation step, one of a plurality of calculation methods is selected in accordance with the evaluation value of the distribution range.

6. The position/orientation measurement method according to claim 1, characterized in that the evaluation value associated with the distribution range is an evaluation value associated with a three-dimensional distribution range of the detected indices.

7. The position/orientation measurement method according to claim 1, characterized in that the evaluation value associated with the distribution range is an evaluation value associated with a distribution range of the detected indices on the captured image.

8. The position/orientation measurement method according to claim 1, characterized in that in the position/orientation calculation step, an influence of information associated with the image coordinates of the detected indices on calculations of the position and the orientation of the measurement target object is controlled in accordance with the evaluation value associated with the distribution range of the indices.

9. The position/orientation measurement method according to claim 1, wherein in the position/orientation calculation step,

the position and the orientation of the measurement target object are calculated from the first position/orientation and the second position/orientation based on the evaluation value associated with the distribution range of the indices calculated in the distribution range calculation step.

10. The position/orientation measurement method according to claim 1, characterized by further comprising a measurement result acquisition step of acquiring a measurement result of at least one of parameters which represent the position and the orientation of the measurement target object from a measurement unit,

wherein in the position/orientation calculation step, the position and the orientation of the measurement target object are calculated by controlling an influence of the information associated with the image coordinates of the indices and an influence of the measurement result based on the evaluation value associated with the distribution range of the indices.

11. The position/orientation measurement method according to claim 10, characterized in that the measurement unit is a position sensor and/or an orientation sensor attached to the measurement target object.

12. The position/orientation measurement method according to claim 1, characterized by further comprising a holding step of holding at least one of parameters which represent the position and the orientation of the measurement target object, and are calculated from a captured image of a previous frame,

wherein in the position/orientation calculation step,

the position and the orientation of the measurement target object are calculated by controlling an influence of the information associated with the image coordinates of the indices and an influence of the parameters which represent the position and the orientation of the measurement target object, and are calculated from the captured image of the previous frame, based on the evaluation value associated with the distribution range of the indices.

13. The position/orientation measurement method according to claim 1, characterized by further comprising an orientation measurement value acquisition step of acquiring an orientation measurement value from an orientation sensor which measures the orientation of the measurement target object,

wherein in the position/orientation calculation step, an influence of the information associated with the image coordinates of the indices and an influence of the orientation measurement value are set according to a size of an information amount of the information associated with the image coordinates of the indices, and a position and an orientation of the measurement target object are calculated.

14. The position/orientation measurement method according to claim 1, characterized by further comprising an orientation measurement value acquisition step of acquiring an orientation measurement value from an orientation sensor which measures the orientation of the measurement target object,

wherein in the position/orientation calculation step, the position and the orientation of the measurement target object are calculated using, as evaluation criteria, both a difference between actually measured values of the information associated with the image coordinates of the indices detected in the index detection step and theoretical values of the information associated with the image coordinates, which are calculated based on estimated values of the position and the orientation of the measurement target object, and a difference between the estimated values and at least some parameters of the orientation measurement value.

15. A position/orientation measurement method for measuring a position and an orientation of a measurement target object, characterized by comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the measurement target object;

a captured image acquisition step of acquiring a captured image;

an index detection step of detecting indices included in the captured image; and

a position/orientation calculation step of calculating the position and the orientation of the measurement target using information associated with image coordinates of the detected indices, and the orientation measurement value,

wherein in the position/orientation calculation step,

the position and the orientation of the image capture device are calculated using, as evaluation criteria, a difference between actually measured values of the information associated with the image coordinates of the detected indices and theoretical values of the information associated with the image coordinates of the detected indices, which are calculated based on estimated values of the position and the orientation of the image capture device, and

a difference between the estimated values and at least some parameters of the orientation measurement value.

16. The position/orientation measurement method according to claim 15, characterized in that the measurement target object is an image capture device, and the captured image is an image captured by the image capture device.

17. The position/orientation measurement method according to claim 15, characterized in that the captured image is an image captured by an image capture device, which has a known relative positional relationship with the measurement target object.

18. The position/orientation measurement method according to claim 15, characterized in that the captured image is an image captured by an image capture device which captures the measurement target object.

19. The position/orientation measurement method according to claim 15, characterized by further comprising a holding step of holding at least one of parameters which represent the position and the orientation of the measurement target object, and are calculated from a captured image of a previous frame,

wherein in the position/orientation calculation step,

the position and the orientation of the measurement target object are calculated using, as evaluation criteria, the difference between the actually measured values of the information associated with the image coordinates of the indices and the theoretical values of the information associated with the image coordinates of the detected indices, which are calculated based on the actually measured values of the information associated with the image coordinates of the detected indices and the estimated values of the position and the orientation of the measurement target object,

a difference between the estimated values and at last one of parameters which represent a previous position and orientation held by the holding means;

and

a difference between the estimated values and at least one of parameters which represent a position and an orientation and are calculated from the captured image of the previous frame.

20. A position/orientation measurement method for measuring a position and an orientation of an image capture device as a measurement target object, characterized by comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the image capture device;

an image input step of inputting an image captured by the image capture device;

an index detection step of detecting indices included in the captured image;

a weight setting step of setting a weight input by an operator; and

a position/orientation calculation step of calculating the position and the orientation of the image capture device using information associated with image coordinates of the indices detected in the index detection step, and the orientation measurement value,

wherein in the position/orientation calculation step, the position and the orientation of the image capture device are calculated by changing influences of the information associated with the image coordinates of the indices and the orientation measurement value in accordance with the weight set in the weight setting step.

21. The position/orientation measurement method according to claim 20, characterized in that in the position/orientation calculation step, at least one of a difference between actually measured values of information associated with the image coordinates of the indices and theoretical values of the information associated with the image coordinates which are calculated based on the estimated values of the position and the orientation of the image capture device, and a difference between the estimated values and at least some parameters of the orientation measurement value is weighted according to the weight set in the weight setting step, and the position and the orientation of the image capture device are calculated using the difference as an evaluation criterion.

22. A position/orientation measurement method for measuring a position and an orientation of an image capture device as a measurement target object, characterized by comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the image capture device;

an image input step of inputting an image captured by the image capture device;

an index detection step of detecting indices included in the captured image;

a mode selection step of selecting one of a first mode focused on the information associated with the image coordinates of the indices, a second mode focused on the orientation measurement value, and a third mode as an intermediate mode between the first mode and the second mode; and

a position/orientation calculation step of calculating the position and the orientation of the image capture device by applying a method according to the mode selected in the mode selection step using information associated with image coordinates of the indices detected in the index detection step, and the orientation measurement value.

23. The position/orientation measurement method according to claim 22, characterized in that in the position/orientation calculation step,

when the first mode is selected in the mode selection step, the position and the orientation of the image capture device are calculated based on the information associated with the image coordinates of the indices detected in the index detection step without using the orientation measurement value as a prescribed value or an evaluation criterion,

when the third mode is selected in the mode selection step, the position and the orientation of the image capture device are calculated using, as evaluation criteria, both a difference between actually measured values of the information associated with the image coordinates of the indices detected in the index detection step and theoretical values of the information associated with the image coordinates which are calculated based on estimated values of the position and the orientation of the image capture device, and a difference between the estimated values and at least some parameters of the orientation measurement value, and

when the second mode is selected in the mode selection step, unknown parameters of the position and the orientation of the image capture device are calculated based on the information associated with the image coordinates of the indices detected in the index detection step while setting at least some parameters of the orientation measurement value as prescribed values.

24. A position/orientation measurement apparatus for measuring a position and an orientation of a measurement target object, characterized by comprising:

captured image acquisition means for acquiring a captured image;

index detection means for detecting indices included in the captured image;

distribution range calculation means for calculating an evaluation value associated with a distribution range of the indices detected by said index detection means; and

position/orientation calculation means for calculating the position and the orientation of the measurement target object using information associated with image coordinates of the detected indices, and the evaluation value associated with the distribution range,

wherein the position/orientation calculation means calculates a first position/orientation and a second position/orientation of the measurement target object by using different methods by exploiting the information associated with the image coordinates of the indices detected by the index detection means.

25. A position/orientation measurement apparatus for measuring a position and an orientation of a measurement target object, characterized by comprising:

orientation measurement value input means for inputting an orientation measurement value from an orientation sensor attached to the measurement target object;

captured image acquisition means for acquiring a captured image;

index detection means for detecting indices included in the captured image; and

position/orientation calculation means for calculating the position and the orientation of the measurement target object using information associated with image coordinates of the detected indices, and the orientation measurement value,

wherein said position/orientation calculation means calculates the position and the orientation of the image capture device using, as evaluation criteria,

a difference between actually measured values of the information associated with the image coordinates of the detected indices and theoretical values of the information associated with the image coordinates of the detected indices, which are calculated based on estimated values of the position and the orientation of the image capture device, and

a difference between the estimated values and at least some parameters of the orientation measurement value.

26. A computer-readable medium storing commands for causing a computer to perform a position/orientation measurement

method for measuring a position and an orientation of a measurement target object, the method comprising:

a captured image acquisition step of acquiring a captured image;

an index detection step of detecting indices included in the captured image;

a distribution range calculation step of calculating an evaluation value associated with a distribution range of the indices detected in the index detection step; and

a position/orientation calculation step of calculating the position and the orientation of the measurement target object using information associated with image coordinates of the detected indices, and the evaluation value associated with the distribution range,

wherein in the position/orientation calculation step, a first position/orientation and a second position/orientation of the measurement target object are calculated using different methods by exploiting the information associated with the image coordinates of the indices detected in the index detection step.

27. A computer-readable medium storing commands for causing a computer to perform a position/orientation measurement method for measuring a position and an orientation of a measurement target object, the method comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the measurement target object;

a captured image acquisition step of acquiring a captured image;

an index detection step of detecting indices included in the captured image; and

a position/orientation calculation step of calculating the position and the orientation of the measurement target using information associated with image coordinates of the detected indices, and the orientation measurement value,

wherein in the position/orientation calculation step,

the position and the orientation of the image capture device are calculated using, as evaluation criteria, a difference between actually measured values of the information associated with the image coordinates of the detected indices and theoretical values of the information associated with the image coordinates of the detected indices, which are calculated based on estimated values of the position and the orientation of the image capture device, and

a difference between the estimated values and at least some parameters of the orientation measurement value.

28. A computer-readable medium storing commands for causing a computer to perform a position/orientation measurement method for measuring a position and an orientation of an image capture device as a measurement target object, the method comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the image capture device;

an image input step of inputting an image captured by the image capture device;

an index detection step of detecting indices included in the captured image;

a weight setting step of setting a weight input by an operator; and

a position/orientation calculation step of calculating the position and the orientation of the image capture device using information associated with image coordinates of the indices detected in the index detection step, and the orientation measurement value,

wherein in the position/orientation calculation step, the position and the orientation of the image capture device are calculated by changing influences of the information associated with the image coordinates of the indices and the orientation measurement value in accordance with the weight set in the weight setting step.

29. A computer-readable medium storing commands for causing a computer to perform a position/orientation measurement method for measuring a position and an orientation of an image capture device as a measurement target object, the method comprising:

an orientation measurement value input step of inputting an orientation measurement value from an orientation sensor attached to the image capture device;

an image input step of inputting an image captured by the image capture device;

an index detection step of detecting indices included in the captured image;

a mode selection step of selecting one of a first mode focused on the information associated with the image coordinates of the indices, a second mode focused on the orientation measurement value, and a third mode as an intermediate mode between the first mode and the second mode; and

a position/orientation calculation step of calculating the position and the orientation of the image capture device by applying a method according to the mode selected in the mode selection step using information associated with image coordinates of the indices detected in the index detection step, and the orientation measurement value.