Placement information estimating method and information processing device

• US 7,529,387 B2
• Filed: 05/11/2005
• Issued: 05/05/2009
• Est. Priority Date: 05/14/2004
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kotake, Daisuke, Uchiyama, Shinji
• Primary Examiner(s): Strege; John B
• Application Number:
  US11/126,436
• Publication Number:
  US 20060004280A1
• Time in Patent Office: 1,455 Days
• Field of Search: 382/103, 382/287, 382/154, 345/633, 600/414
• US Class Current: 382/103
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

A method is provided for estimating the placement information of a plurality of types of markers of which the amounts of the placement information differ respectively, detects a plurality of types of markers from multiple images, calculates the position where the detected marker is projected on an image surface based on the approximate values of the positions and orientations of an imaging device when multiple images are photographed, obtains the correction value of the placement information of a marker so as to reduce a sum of error between the position of the detected marker on the image surface and the projected position of the marker calculated, corrects the placement information of the marker, and also calculates the placement information of a plurality of types of markers by repeating these processes.

Claims

1. A marker placement information estimating method for estimating placement information of a plurality of types of markers, the marker placement information estimating method comprising:

using a processor to carry out the method of;

a marker detecting step for detecting a plurality of types of markers, of which the placement information has different amounts of information, from multiple images in which scenes including markers are photographed, the plurality of types of markers including a first marker type of which the placement information is described with the position of one point within 3-D space, and a second marker type of which the placement information is described with the position and orientation within 3-D space;

a marker projected position calculating step for calculating a projected position where a marker is projected on an image surface based on an approximate value of the position and orientation of an imaging unit when the multiple images were photographed and an approximate value of the placement information of each type of marker;

a correction value calculating step for obtaining a correction value as to the placement information of each type of marker so as to reduce a sum of error between the position of each type of marker on the image surface detected in the marker detecting step and the projected position of the marker on the image surface obtained in the marker projected position calculating step;

a placement information correcting step for correcting an arrangement of each type of marker based on the correction value obtained in the correction value calculating step; and

a repeated calculating step for calculating the placement information of each type of marker by repeatedly performing the marker projected position calculating step, the correction value calculating step, and the placement information correcting step using the placement information of the marker corrected in the placement information correcting step instead of the approximate value of the marker projected position calculating step.

2. The marker placement information estimating method according to claim 1, wherein the correction value calculating step obtains a correction value corresponding to the placement information of each type of marker by giving a different weight for each type of marker.

3. The marker placement information estimating method according to claim 1, wherein the second marker type is a plane polygon region of which the color or brightness is different from the surroundings, and is a polygon marker that can be detected by obtaining the perimeter shape thereof from an image.

4. The marker placement information estimating method according to claim 1, further comprising a step for outputting the marker placement information of the estimated marker to a file.

5. A computer-readable medium storing a program for realizing the marker placement information estimating method according to claim 1 using a computer.

6. A marker placement information estimating method for estimating placement information of a plurality of types of markers, the marker placement information estimating method comprising:

using a processor to carry out the method of;

a marker detecting step for detecting a plurality of types of markers, of which the placement information has different amounts of information, from multiple images in which scenes including markers are photographed, the plurality of types of markers including a first marker type of which the placement information is described with the position of one point within 3-D space, and a second marker type of which the placement information is described with the position and orientation within 3-D space;

a marker type selecting step for selecting one or more types of marker from the plurality of types of markers;

a marker projected position calculating step for calculating a position where a marker is projected on an image surface based on an approximate value of the position and orientation of an imaging unit when the multiple images were photographed and an approximate value of the placement information of the marker having the type selected in the marker type selecting step;

a placement information correcting step for obtaining the placement information of the selected types of marker so as to reduce a sum of error between the position on the image surface of the marker, which has been detected in the marker detecting step, having the type selected in the marker type selecting step and the projected position on the image surface of the marker obtained in the marker projected position calculating step, and correcting the placement information;

a first repeated calculating step for calculating the placement information of the marker corrected in the placement information correcting step by repeatedly performing the marker projected position calculating step and the placement information correcting step using the placement information of the marker corrected in the placement information correcting step instead of the approximate value of the marker projected position calculating step; and

a second repeated calculating step for selecting one or more markers having the type not selected in the marker type selecting step again, and repeatedly performing the marker projected position calculating step, the placement information correcting step, and the first repeated calculating step.

7. The marker placement information estimating method according to claim 6, wherein the second marker type is a plane polygon region of which the color or brightness is different from surroundings, and is a polygon marker that can be detected by obtaining the perimeter shape thereof from an image.

8. The marker placement information estimating method according to claim 6, further comprising a step for outputting the marker placement information of the estimated marker to a file.

9. A computer-readable medium storing a program for realizing the marker placement information estimating method according to claim 6 using a computer.

10. An information processing device for estimating placement information of a plurality of types of markers, the information processing device comprising:

an imaging unit for, upon instruction by a user, capturing a plurality of images of a scene having a plurality of markers, the plurality of types of markers including a first marker type of which the placement information is described with the position of one point within 3-D space, and a second marker type of which the placement information is described with the position and orientation within 3-D space;

an image capturing unit for inputting the plurality of images of the scene having the plurality of markers captured by the imaging unit;

a marker detecting unit adapted to detect a plurality of types of markers, of which the placement information has different amounts of information, from the plurality of images input by the image capturing unit;

a marker projected position calculating unit adapted to calculate a position where a marker is projected on an image surface based on an approximate value of the position and orientation of an imaging unit when the multiple images were photographed and an approximate value of the placement information of each type of marker;

a correction value calculating unit adapted to obtain a correction value as to the placement information of each type of marker so as to reduce a sum of error between the position of each type of marker on the image surface detected by the marker detecting unit and the projected position of the marker on the image surface obtained by the marker projected position calculating unit;

a placement information correcting unit adapted to correct an arrangement of each type of marker based on the correction value obtained by the correction value calculating unit; and

a repeated calculating unit adapted to calculate the placement information of each type of marker by repeatedly performing the processing at the marker projected position calculating unit, the processing at the correction value calculating unit, and the processing at the placement information correcting unit using the placement information of the marker corrected by the placement information correcting unit instead of the approximate value of the marker projected position calculating unit.

11. An information processing device for estimating placement information of a plurality of types of markers, the information processing device comprising:

an imaging unit for, upon instruction by a user, capturing a plurality of images of a scene having a plurality of markers, the plurality of types of markers including a first marker type of which the placement information is described with the position of one point within 3-D space, and a second marker type of which the placement information is described with the position and orientation within 3-D space;

an image capturing unit for inputting the plurality of images of the scene having the plurality of markers captured by the imaging unit;

a marker detecting unit adapted to detect a plurality of types of markers, of which the placement information has different amounts of information, from plurality of images input by the image capturing unit;

a marker type selecting unit adapted to select one or more types of marker from the plurality of types of markers;

a marker projected position calculating unit adapted to calculate a projected position where a marker is projected on an image surface based on an approximate value of the position and orientation of an imaging unit when the multiple images were photographed and an approximate value of placement information of the marker having the type selected by the marker type selecting unit;

a placement information correcting unit adapted to obtain placement information of the selected types of marker so as to reduce a sum of error between the position on the image surface of the marker, which has been detected by the marker detecting unit, having the type selected by the marker type selecting unit and the projected position on the image surface of the marker obtained by the marker projected position calculating unit, and correcting the placement information;

a first repeated calculating unit adapted to calculate the placement information of the marker corrected by the placement information correcting unit by repeatedly performing processing at the marker projected position calculating unit and processing at the placement information correcting unit using the placement information of the marker corrected by the placement information correcting unit instead of the approximate value of the marker projected position calculating unit; and

a second repeated calculating unit adapted to select one or more markers having the type not selected by the marker type selecting unit again, and repeatedly performing processing at the marker projected position calculating unit, processing at the placement information correcting unit, and processing at the first repeated calculating unit.