Three-dimensional position/orientation sensing apparatus, information presenting system, and model error detecting system

• US 6,697,761 B2
• Filed: 09/13/2001
• Issued: 02/24/2004
• Est. Priority Date: 09/19/2000
• Status: Active Grant

• Current Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Original Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Inventors: Saito, Akito, Shibasaki, Takao, Akatsuka, Yuichiro
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): SUAREZ, FELIX E
• Application Number:
  US09/951,873
• Publication Number:
  US 20020052709A1
• Time in Patent Office: 894 Days
• Field of Search: 702/150, 702/151, 702/152, 702/153, 396/70, 396/429, 714/48, 714/56, 714/57
• US Class Current: 702/151
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures

Abstract

There are disclosed a three-dimensional position/orientation sensing apparatus which can measure a broad region, when an image obtained by capturing a marker having a known position/orientation relation with an object is analyzed, a relative position/orientation relation between the marker and a capturing apparatus is obtained, and a position and orientation of the object are obtained, an information presenting system in which the captured image of an actual object can easily be compared with object data, and a model error detecting system.

Claims

1. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

an image input section which inputs the image obtained by capturing said object and a plurality of markers each having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

an identifying section which uses the geometric characteristic of at least one of the plurality of markers to identify said at least one of the markers in the image inputted from said image input section; and

a position/orientation detecting section by a single marker, which analyzes the image of one of said at least one of the markers identified by said identifying section, obtains a relative position/orientation relation between said one marker and said capturing apparatus, and obtains the three-dimensional position/orientation relation between said object and said capturing apparatus.

2. The three-dimensional position/orientation sensing apparatus according to claim 1, further comprising:

a position/orientation detecting section by a plurality of markers, which obtains the relative position/orientation relation between said one marker and said capturing apparatus by analyzing the plurality of markers, and obtains the three-dimensional position/orientation relation between said object and said capturing apparatus; and

a changing section which selectively changes said position/orientation detecting section by the single marker and said position/orientation detecting section by the plurality of markers.

3. The three-dimensional position/orientation sensing apparatus according to claim 2, wherein said changing section changes said position/orientation detecting section by the single marker and said position/orientation detecting section by the plurality of markers based on a visual field angle of the marker on the image.

4. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said position/orientation detecting section by the single marker detects a plurality of characteristic portions from the image of said marker, compares a position relation of said characteristic portion on the image with a position relation of said characteristic portion on an actual marker, and obtains the relative position/orientation relation between said marker and said capturing apparatus.

5. The three-dimensional position/orientation sensing apparatus according to claim 4, wherein a shape of said marker is a quadrangular or further polygonal, and said position/orientation detecting section by the single marker detects a vertex of a polygonal shape as said characteristic portion.

6. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker includes information indicating a size of the marker.

7. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker includes information indicating an attribute of a place of the marker.

8. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker is formed of a self-emission type material.

9. The three-dimensional position/orientation sensing apparatus according to claim 8, wherein the marker formed of said self-emission type material is lit in synchronization with an alarm lamp during emergency/disaster.

10. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker is formed of a reflective material for reflecting and reversing an incident light along an incident light path.

11. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker is formed of a fluorescent material.

12. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker is invisible to human eyes.

13. The three-dimensional position/orientation sensing apparatus according to claim 1, wherein said marker is projected by a projector.

14. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

image input means for inputting the image obtained by capturing said object and a plurality of markers each having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

identifying means for using the geometric characteristic of at least one of the plurality of markers to identify said at least one of the markers in the image inputted from said image input means; and

position/orientation detecting means by a single marker, for analyzing the image of one of said at least one of the markers identified by said identifying means, obtaining a relative position/orientation relation between said one marker and said capturing apparatus, and obtaining the three-dimensional position/orientation relation between said object and said capturing apparatus.

15. An information presenting system comprising:

an image input section which inputs an image obtained by capturing a marker whose three-dimensional position relation with an object is known by a capturing apparatus;

a position/orientation detecting section which uses a position of the marker on said image to obtain a position/orientation relation between said object and said capturing apparatus; and

a display section which displays three-dimensional model data of said object on said image with a position, a size, and a direction based on said position/orientation relation.

16. The information presenting system according to claim 15, further comprising a display section which displays attribute information of said object.

17. The information presenting system according to claim 15, further comprising an object indicating section which indicates a desired object from a plurality of objects.

18. The information presenting system according to claim 15, wherein at least one of said three-dimensional model data and the attribute information is superimposed and displayed on a screen with a color different from a color of said object.

19. The information presenting system according to claim 16, wherein at least one of said three-dimensional model data and the attribute information is superimposed and displayed on a screen with a color different from a color of said object.

20. The information presenting system according to claim 16, wherein display of said attribute information includes display of a name of the object and a code for indicating said object from the name display.

21. The information presenting system according to claim 16, further comprising a setting section which sets a display position such that the displays of said attribute information do not interfere with each other.

22. The information presenting system according to claim 15, wherein said three-dimensional model data is displayed in a coated surface in which a background is seen through.

23. The information presenting system according to claim 15, wherein said three-dimensional model data is displayed, even when said object is shielded by a shielding material, and is not displayed in said image.

24. The information presenting system according to claim 15, wherein said marker is a visual marker attached to at least one of the object and the vicinity of the object.

25. The information presenting system according to claim 15, wherein said marker is invisible to human eyes.

26. The information presenting system according to claim 15, wherein said marker utilizes a characteristic portion of the object as a marker.

27. The information presenting system according to claim 15, wherein said marker is projected by a projector.

28. The information presenting system according to claim 16, wherein said attribute information is displayed in a least one of a wall surface and a ceiling surface around the object on the image.

29. A model error detecting system in which three-dimensional model data of an object can be corrected based on an image of the object, said system comprising:

an image input section which inputs the image obtained by capturing said object by a capturing apparatus;

a position/orientation detecting section which obtains a position/orientation relation between said object and said capturing apparatus;

a holding section which holds the model data of said object;

an error detecting section which compares the model data of the object extracted from said holding section with the image of said object based on said position/orientation relation, and detects an actual error from the model data of said object; and

a correcting section which corrects the model data of the holding section based on a detection result of said error detecting section.

30. The model error detecting system according to claim 29, wherein said position/orientation detecting section uses an image of a marker which is included in the input image and whose position relation with the object is known to obtain the position/orientation relation between the object and said capturing apparatus.

31. The model error detecting system according to claim 29, further comprising:

a display section which superimposes/displays the model data of the object onto the object on said image; and

a designating section which designates a place for comparison/collation on display in said display section,

wherein said error detecting section uses the image from a plurality of directions to perform a processing of obtaining an error in a two-dimensional projection coordinate in a visual field direction in the place designated by said designating section, and obtains a three-dimensional error.

32. The model error detecting system according to claim 29, wherein said error detecting section matches a pattern of the image obtained by two-dimensionally projecting the three-dimensional model data of said object with a pattern of the image of the object, and detects the error.

33. The model error detecting system according to claim 29, wherein the model data of the object is an image of a previously captured object, or is prepared based on the image.

34. The model error detecting system according to claim 29, wherein said holding section also holds position coordinate data of the object, and

said error detecting section uses a position coordinate of said object to obtain an error of the position coordinate data from a value obtained by actually measuring a distance between the object and the capturing apparatus by a measuring device.

35. The model error detecting system according to claim 34, wherein said holding section also holds volume data of the object, and

said error detecting section obtains the image of the object, a volume obtained by using said actually measured distance between the object and the capturing apparatus, and an error of volume data of said holding section.

36. An information presenting system comprising:

image input means for inputting an image obtained by capturing a marker whose three-dimensional position relation with an object is known by a capturing apparatus;

position/orientation detecting means for using a position of the marker on said image to obtain a position/orientation relation between said object and said capturing apparatus; and

display means for displaying three-dimensional model data of said object on said image with a position, a size, and a direction based on said position/orientation relation.

37. A model error detecting system in which three dimensional model data of an object can be corrected based on an image of the object, said system comprising:

image input means for inputting an image obtained by capturing said object by a capturing apparatus;

position/orientation detecting means for obtaining a position/orientation relation between said object and said capturing apparatus;

holding means for holding the model data of said object;

error detecting means for comparing the model data of the object extracted from said holding means with the image of said object based on said position/orientation relation, and detecting an actual error with the model data of said object; and

correcting means for correcting the model data of the holding means based on a detection result of said error detecting means.

38. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

an image input section which inputs the image obtained by capturing said object and at least one marker having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

an identifying section which uses the geometric characteristic of said at least one marker to identify said at least one marker in the image inputted from said image input section;

a position/orientation detecting section by a single marker, which analyzes the image of one of said at least one marker identified by said identifying section, obtains a relative position/orientation relation between said one marker and said capturing apparatus, and obtains the three-dimensional position/orientation relation between said object and said capturing apparatus;

a position/orientation detecting section by a plurality of markers, which analyzes images of plural markers identified by said identifying section to obtain three-dimensional position/orientation relations between said one marker, said object and said capturing apparatus; and

a changing section which effects selective switching between said position/orientation detecting section by the single marker and said position/orientation detecting section by the plurality of markers.

39. The three-dimensional position/orientation sensing apparatus according to claim 38, wherein said changing section effects switching between said position/orientation detecting section by the single marker and said position/orientation detecting section by the plurality of markers based on a visual field angle of the marker on the image.

40. The three-dimensional position/orientation sensing apparatus according to claim 38, wherein said position/orientation detecting section by the single marker detects a plurality of characteristic portions from the image of said marker, compares a position relation of each of said characteristic portions on the image with a position relation of said each characteristic portion on an actual marker, and obtains the relative position/orientation relation between said marker and said capturing apparatus.

41. The three-dimensional position/orientation sensing apparatus according to claim 40, wherein said marker is formed in a shape of a polygon having four or more sides, and said position/orientation detecting section by the single marker detects a vertex of the polygon as said characteristic portion.

42. The three-dimensional position/orientation sensing apparatus according to claim 38, wherein said position/orientation detecting section by the plurality of markers detects relative position/orientation relations between the capturing apparatus and the plurality of markers based on positions of the plurality of markers on the image.

43. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

image input means for inputting the image obtained by capturing said object and at least one marker having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

identifying means for identifying said at least one marker in the image inputted from said image input means, by using the geometric characteristic of said at least one marker;

position/orientation detecting means by a single marker, for analyzing the image of one of said at least one marker identified by said identifying means, obtaining a relative position/orientation relation between said one marker and said capturing apparatus, and obtaining the three-dimensional position/orientation relation between said object and said capturing apparatus;

position/orientation detecting means by a plurality of markers, for analyzing images of plural markers identified by said identifying means to obtain three-dimensional position/orientation relations between said one marker, said object and said capturing apparatus; and

changing means for effecting selective switching between said position/orientation detecting means by the single marker and said position/orientation detecting means by the plurality of markers.

44. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

an image input section which inputs the image obtained by capturing said object and one or more markers each having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

an identifying section which uses the geometric characteristic of said one or more markers to identify said one or more markers in the image inputted from said image input section;

a switching section which changes a number of said one or more markers to be analyzed; and

a position/orientation detecting section which analyzes the image or images of said one or more markers identified by said identifying section in the number set by said switching section, obtains relative position/orientation relations between said capturing apparatus and said one or more markers whose images are analyzed, and obtains the three-dimensional position/orientation relation between said object and said capturing apparatus.

45. A three-dimensional position/orientation sensing apparatus for measuring a relative three-dimensional position/orientation relation between an object and a capturing apparatus from an image captured by the capturing apparatus, said sensing apparatus comprising:

image input means for inputting the image obtained by capturing said object and one or more markers each having a known geometric characteristic and a known three-dimensional position/orientation relation with respect to said object, by using said capturing apparatus;

identifying means for identifying said one or more markers in the image inputted from said image input means by using the geometric characteristic of said one or more markers;

switching means for changing a number of said one or more markers to be analyzed; and

position/orientation detecting means for analyzing the image or images of said one or more makers identified by said identifying means in the number set by said switching means, obtaining relative position/orientation relations between said capturing apparatus and said one or more markers whose images are analyzed, and obtaining the three-dimensional position/orientation relation between said object and said capturing apparatus.