INDEX LAYOUT MEASUREMENT METHOD, POSITION AND ORIENTATION ESTIMATION METHOD, INDEX LAYOUT MEASUREMENT APPARATUS, AND POSITION AND ORIENTATION ESTIMATION APPARATUS

• US 20070091125A1
• Filed: 12/15/2006
• Published: 04/26/2007
• Est. Priority Date: 02/02/2005
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Uchiyama, Shinji, Takemoto, Rika
• Granted Patent: US 7,664,341 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

The layout relationship between indices included in a first image and those included in a second image are calculated using the layout relationship between a common index commonly included in the first and second images, and the indices included in the first image, and that between the common index and the indices included in the second image.

Claims

1. An index layout measurement method of measuring a relative layout relationship between a first index or index group which exists in a given first region in a physical space and a second index or index group which exists in a second region even when the first index or index group which exists in the first region and the second index or index group which exists in the second region have a layout or condition in which an image thereof cannot be sensed under a condition that suffices to measure a relative layout relationship in a method of measuring a relative layout relationship between indices by sensing an image of a plurality of indices in a physical space by a movable image sensing device, characterized by comprising:

a first image sensing step of sensing an image so as to simultaneously include both some or all of the first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

a second image sensing step of sensing an image so as to simultaneously include both some or all of the second index/index group and the auxiliary index; and

a layout calculation step of calculating a relative layout relationship among the first index/index group, the auxiliary index, and the second index/index group from one or a plurality of images obtained by performing the first image sensing step once or repeating it a plurality of number of times, and one or a plurality of images obtained by performing the second image sensing step once or repeating it a plurality of number of times.

3. The method according to claim 1, characterized in that a visual feature of the auxiliary index is determined in advance.

4. The method according to claim 1, characterized in that the auxiliary index is removed after the layout relationship among the respective indices laid out in the physical space are calculated.

5. The method according to claim 1, characterized in that the auxiliary index is an auxiliary marker which is physically laid out or is optically laid out by projecting onto a given surface in the physical space.

10. A program characterized by making a computer execute an index layout measurement method according to claim 1.

12. A computer readable storage medium characterized by storing a program according to claim 10.

2. An index layout measurement method of measuring a relative layout relationship between a first index or index group which exists in a given first region in a physical space and a second index or index group which exists in a second region even when the first index or index group which exists in the first region and the second index or index group which exists in the second region have a layout or condition in which an image thereof cannot be sensed under a condition that suffices to measure a relative layout relationship in a method of measuring a relative layout relationship between indices by sensing an image of a plurality of indices in a physical space by a movable image sensing device, characterized by comprising:

a first image sensing step of sensing an image so as to simultaneously include both some or all of the first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

a first layout calculation step of calculating a relative layout relationship between the first index/index group and the auxiliary index from one or a plurality of images obtained by performing the first image sensing step once or repeating it a plurality of number of times;

a second image sensing step of sensing an image so as to simultaneously include both some or all of the second index/index group and the auxiliary index;

a second layout calculation step of calculating a relative layout relationship between the second index/index group and the auxiliary index from one or a plurality of images obtained by performing the second image sensing step once or repeating it a plurality of number of times; and

a third layout calculation step of calculating a relative layout relationship between the first index/index group and the second index/index group from results of the first layout calculation step and the second layout calculation step.

6. A position and orientation estimation method of estimating a position and/or orientation of a movable image sensing device based on indices which exist in a physical space, characterized by comprising:

a first image sensing step of sensing an image so as to simultaneously include both some or all of a first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

a first layout calculation step of calculating a relative layout relationship between the first index/index group and the auxiliary index from one or a plurality of images obtained by performing the first image sensing step once or repeating it a plurality of number of times;

a second image sensing step of sensing an image so as to simultaneously include both some or all of a second index/index group and the auxiliary index;

a second layout calculation step of calculating a relative layout relationship between the second index/index group and the auxiliary index from one or a plurality of images obtained by performing the second image sensing step once or repeating it a plurality of number of times;

a third layout calculation step of calculating a relative layout relationship between the first index/index group and the second index/index group from results of the first layout calculation step and the second layout calculation step; and

an image sensing device position and orientation estimation step of estimating the position and orientation of the image sensing device using layouts of the first index/index group and the second index/index group without using the auxiliary index.

11. A program characterized by making a computer execute a position and orientation estimation method according to claim 6.

7. An index layout measurement apparatus for measuring a relative layout relationship between a first index or index group which exists in a given first region in a physical space and a second index or index group which exists in a second region even when the first index or index group which exists in the first region and the second index or index group which exists in the second region have a layout or condition in which an image thereof cannot be sensed under a condition that suffices to measure a relative layout relationship in a method of measuring a relative layout relationship between indices by sensing an image of a plurality of indices in a physical space by a movable image sensing device, characterized by comprising:

first image sensing unit adapted to sense an image so as to simultaneously include both some or all of the first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

second image sensing unit adapted to sense an image so as to simultaneously include both some or all of the second index/index group and the auxiliary index; and

layout calculation unit adapted to calculate a relative layout relationship among the first index/index group, the auxiliary index, and the second index/index group from one or more images obtained by sensing once or more by said first image sensing unit, and one or more images obtained by sensing once or more by said second image sensing unit.

8. An index layout measurement apparatus for measuring a relative layout relationship between a first index or index group which exists in a given first region in a physical space and a second index or index group which exists in a second region even when the first index or index group which exists in the first region and the second index or index group which exists in the second region have a layout or condition in which an image thereof cannot be sensed under a condition that suffices to measure a relative layout relationship in a method of measuring a relative layout relationship between indices by sensing an image of a plurality of indices in a physical space by a movable image sensing device, characterized by comprising:

first image sensing unit adapted to sense an image so as to simultaneously include both some or all of the first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

first layout calculation unit adapted to calculate a relative layout relationship between the first index/index group and the auxiliary index from one or more images obtained by sensing once or more by said first image sensing unit;

second image sensing unit adapted to sense an image so as to simultaneously include both some or all of the second index/index group and the auxiliary index;

second layout calculation unit adapted to calculate a relative layout relationship between the second index/index group and the auxiliary index from one or more images obtained by sensing once or more by said second image sensing unit; and

third layout calculation unit adapted to calculate a relative layout relationship between the first index/index group and the second index/index group from results of said first layout calculation unit and said second layout calculation unit.

9. A position and orientation estimation apparatus for estimating a position and/or orientation of a movable image sensing device based on indices which exist in a physical space, characterized by comprising:

first image sensing unit adapted to sense an image so as to simultaneously include both some or all of a first index/index group and an auxiliary index which is temporarily laid out in the physical space and a position and orientation in a three-dimensional space of which can be defined;

first layout calculation unit adapted to calculate a relative layout relationship between the first index/index group and the auxiliary index from one or more images obtained by sensing once or more by said first image sensing unit;

second image sensing unit adapted to sense an image so as to simultaneously include both some or all of a second index/index group and the auxiliary index;

second layout calculation unit adapted to calculate a relative layout relationship between the second index/index group and the auxiliary index from one or more images obtained by sensing once or more by said second image sensing unit;

third layout calculation unit adapted to calculate a relative layout relationship between the first index/index group and the second index/index group from results of said first layout calculation unit and said second layout calculation unit; and

image sensing device position and orientation estimation unit adapted to estimate the position and orientation of the image sensing device using layouts of the first index/index group and the second index/index group without using the auxiliary index.