Information display member, position detecting method using the same, apparatus and method of presenting related information, and information presenting apparatus and information presenting method

• US 6,577,249 B1
• Filed: 10/16/2000
• Issued: 06/10/2003
• Est. Priority Date: 10/19/1999
• Status: Expired due to Fees

• Current Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Original Assignee: Olympus Optical Corporation Limited (Olympus Corporation)
• Inventors: Saito, Akito, Shibasaki, Takao, Akatsuka, Yuichiro
• Primary Examiner(s): Tweel, John
• Application Number:
  US09/688,925
• Time in Patent Office: 967 Days
• Field of Search: 340/988, 340/990, 340/995, 701/209, 701/211
• US Class Current: 340/988
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning

Abstract

An information display member has an information display face of its known size. On the information display face, a first pattern is displayed to represent coordinate information indicative of a position of the displayed information display member. On the information display face, a second pattern is displayed to represent direction information indicative of an direction in which the information display face is oriented. The information display face of the information display member is image-acquired by means of a camera, and an image is acquired. A direction in which the camera is oriented is obtained by a length ratio of aspect on the information display face mapped on the image and the direction information displayed on the information display face. A distance between the camera and the information display face is obtained by a size of the information display face mapped on the image. The position coordinate of the camera is obtained from the direction in which the camera is oriented the distance and the coordinate information displayed on the information display face.

Claims

1. An information display member having an information display face of its known size, said information display member comprising:

a first pattern displayed on said information display face in order to represent patterned coordinate information indicative of a position of said information display member; and

a second pattern displayed on said information display face in order to represent patterned direction information indicative of a direction in which said information display face is oriented.

2. An information display member according to claim 1, wherein said coordinate information is directed to latitude and longitude.

3. An information display member according to claim 1, wherein said information display face has a side face section, and information indicative of a direction in which said side face section is oriented is displayed at said side face section.

4. An information display member according to claim 1, further comprising:

a third pattern displayed on said information display face in order to represent patterned height information indicative of a height of said information display member; and

a fourth pattern displayed on said information display face in order to represent patterned vertical and horizontal directions of said information display member.

5. An information display member according to claim 4, wherein at least one of said first to fourth patterns expresses information according to disposition of regions with their two different densities.

6. An information display member according to claim 4, wherein at least one of said first to fourth patterns displays a character or a character describing information represented by each of said patterns together with the patterns.

7. A position detecting method comprising the steps of:

acquiring an image by acquiring the information display member displayed on the information display face with a camera by a first pattern for patterning and representing coordinate information indicative of a position of an information display member having an information display face of its known size and a second pattern for patterning and representing direction information indicative of a direction oriented on the information display face;

obtaining a length ratio of aspect on the information display face mapped on the image and a direction in which the camera is oriented from direction information displayed on the information display face;

obtaining a distance between the camera and the information display face from the size of the information display face mapped on the image; and

obtaining the position coordinate of the camera from the direction in which the camera is oriented, the distance, and the coordinate information displayed on the information display face.

8. An position detecting method according to claim 7, wherein a third pattern for patterning and representing height information indicative of a height of said information display member and a fourth pattern for patterning and representing the vertical and horizontal directions of said information display member are displayed on said information display face of said information display member,

said position detecting method further comprising the steps of:

obtaining height information displayed on an information display face mapped on said image; and

obtaining a height of said camera from said height information and an elevation angle of said camera.

9. An information display member, wherein said information display member is formed in a flat plate shape or a curved face plate shape expressing patterned information by using a combination of geometrical shapes and a combination of colors, and spatial position information and/or direction information is contained in the contents of the patterning expression information.

10. An information display member according to claim 9, wherein a combination of geometrical shapes of said information display member is mainly expressed in binary number.

11. An information display member according to claim 9, wherein said information display member is composed of an electronic display body, and a display patterning information can be changed freely by electrical signal operation.

12. An information display member according to claim 9, wherein the spatial position information and/or direction information on said information display member consists of characters indicative of longitude, latitude, direction, height, and size of the information display member.

13. A position and orientation detecting method, comprising the steps of:

acquiring an image relative to an outer frame of an information display member or external shapes thereof formed in a flat plate shape or curved face plate shape expressing patterning information by using a combination of geometrical shapes and a combination of colors, and including the spatial position information and/or direction information in the contents of the patterning and expression information, and a combination of geometrical patterns that designates the information display member position expressed in the information display member; and

computing a distance from a ratio of a real size of the known information display member to a size of the information display member image-acquired together with an image angle of said image acquisition apparatus, thereby to know the image acquisition position.

14. A position and orientation detecting method according to claim 13, wherein an elevation angle of said image acquisition apparatus is estimated by using the display information on which the vertical upward or downward directions and horizontal direction and orientation are displayed in an information display face of said information display member, and a height difference between said image acquisition position and said information display member is obtained from the already computed distance information.

15. An associated information presenting apparatus for superimposing and displaying associated information on each section of said object at the corresponding position on an image on which an object is image-acquired, said apparatus comprising:

an image input section for including in an image acquisition range a mark on which a pattern representing position information on four or more points of said object is displayed, thereby inputting an image on which said object is image-acquired;

a position information management section for reading said pattern from an image inputted from said image input section, thereby acquiring said position information;

a position information acquisition section for obtaining the position and orientation of said image input section from the acquired position information acquired by said position information management section;

an associated information generating section for obtaining the position of each section of each object on said image based on the position and orientation of said image input section obtained by said position information acquisition section; and

an information superimposing and displaying section for superimposing and displaying said associated information at the corresponding position based on the position on said image obtained by said associated information generating section.

16. A position and orientation detecting method according to claim 15, wherein position information on said mark itself is contained in position information on four or more points represented by said pattern displayed on said mark.

17. A method for superimposing and displaying the associated information on each section of said object at the corresponding position on an image on which an object is image-acquired, said method comprising the steps of:

including in an image acquisition range a mark on which a pattern representing position information on four or more points of said object is displayed, thereby inputting an image on which said object is image-acquired by an image input section;

reading said pattern, thereby acquiring said position information;

obtaining the position and orientation of said image input section from said position information;

obtaining a position on said image of each section of said object from the position and orientation of said image input section; and

superimposing and displaying said associated information at the corresponding position based on the position on said image.

18. An associated information presenting apparatus for superimposing and displaying the associated information on each section of said object at the corresponding position on an image on which an object is image-acquired, said apparatus comprising:

an image input section for including in an image acquisition range a mark on which a pattern representing position information on four or more points of said object is displayed, thereby inputting an image on which said object is image-acquired;

a position information management section for reading said pattern, thereby acquiring said position information;

a position information acquisition section for obtaining the position and orientation of said image input section from said position information;

an associated information generating section for obtaining a position on said image of each section of said object from the position and orientation of said image input section; and

an information superimposing and displaying section for superimposing and displaying said associated information at the corresponding position based on the result of said associated information generating section.

19. An associated information presenting apparatus according to claim 18, wherein position information on said mark itself is contained in position information on four or more points represented by said pattern displayed on said mark.

20. An associated information presenting apparatus for superimposing and displaying the associated information on each section of an object at the corresponding position on an image on which said object is image-acquired, said apparatus comprising:

an image input section for image-acquiring said object, thereby inputting an image;

a position information management section for storing position information on four or more points of said object predetermined from a plurality of images on which the position and orientation of an image input section are known;

a position information acquisition section for obtaining the position and orientation of said image input section from said position information;

an associated information generating section for obtaining a position on said image of each section of said object from the position and orientation of said image input section; and

an information superimposing and displaying section for superimposing and displaying said associated information at the corresponding position based on the result of said associated information generating section.

21. An associated information presenting apparatus according to claim 20, wherein the position and orientation of the image input section in said plurality of images are obtained by including in an image acquisition range a mark on which a pattern representing the size of, and position information on, the mark is displayed;

and image-acquiring an object at said image input section;

reading said pattern, thereby acquiring the size of, and position information on, the mark; and

using the thus acquired size of, and position information of, the mark.

22. An associated presenting apparatus comprising:

an image input section;

a mark recognition section for recognizing a mark in an input image from the image input section, thereby acquiring position information on the mark included in the recognized mark;

a feature point recognition section for recognizing a feature point in a screen;

a position information management section for performing association with the mark acquired by said mark recognition section in advance, thereby acquiring position information corresponding to the feature point in the screen recognized by the feature point recognition section;

a position information acquisition section for acquiring position and orientation information on said image input section from position information on the feature point acquired by a mark and/or position information management section acquired by said mark recognition section;

an associated information generating section for generating associated information in a field of view for an input image based on the position and orientation information acquired by said position information acquiring section; and

a superimposing and displaying section for superimposing and displaying on an input image the associated information generated by said associated information generating section.

23. An associated information presenting apparatus according to claim 22, wherein a feature point for performing said association is not created to be intentionally employed as a mark.

24. An associated information presenting apparatus according to claim 22, wherein a plurality of feature points are subjected to said association.

25. An associated information presenting apparatus according to claim 22, wherein said position information acquisition section employs a mark that contains one feature point and a mark that contains a plurality of feature points.

26. An associated information presenting apparatus comprising:

an input section for inputting an image;

a feature point recognition section for recognizing a feature point in a screen on which an image inputted by the input section is displayed;

a mark recognition section for recognizing a mark in the image inputted from said input section, thereby acquiring position information on the mark included in the recognized mark;

a position information management section for performing association with position information on a mark acquired by said mark recognition section in advance, thereby acquiring position information corresponding to the feature point in the screen recognized by said feature point recognizing section;

an orientation information acquisition section for acquiring position and orientation information on said input section from position information on a mark and/or a feature point;

an associated information generating section for generating associated information in a field of view for an image based on the position and orientation information acquired by said orientation information acquisition section; and

a superimposing and displaying section for superimposing and displaying the associated information generated by said orientation information acquiring section on an image inputted by said input section.

27. An associated information presenting apparatus according to claim 26, wherein a feature point for performing said association is not created to be intentionally employed as a mark.

28. An associated information presenting apparatus according to claim 26, wherein a plurality of feature points are subjected to said association.

29. An associated information presenting apparatus according to claim 26, wherein said position information acquisition section employs a mark that contains one feature point and a mark that contains a plurality of feature points.

30. An information presenting apparatus comprising:

an image input section for inputting an image;

a pattern recognizing section for recognizing a pattern representing information in an image inputted by said image input section;

an information reproducing section for reproducing information represented by a pattern recognized by said pattern recognition section;

an image superimposing section for superimposing information reproduced by said information reproducing section in an image inputted by said image input section; and

an image display section for displaying an output from said image superimposing section.

31. An information display apparatus according to claim 30, wherein said information reproducing section is adopted to set an information display attribute based on display attribute information contained in information reproduced by the information reproducing section.

32. An information presenting method comprising the steps of:

inputting an image;

recognizing a pattern representing information in an input image

reproducing information represented by a recognized pattern;

superimposing reproduced information on said input image; and

displaying an image on which said reproduced information is superimposed.

33. An image presenting apparatus comprising:

an image input section for inputting an image;

a pattern recognition section for recognizing a predetermined pattern in an image inputted by said input image section;

an information reproducing section for reproducing predetermined pattern information recognized by said pattern recognition section;

an image superimposing section for superimposing information reproduced by said information reproducing section on an image inputted by said image input section; and

an image display section for displaying an output from said image superimposing section.

34. An information presenting method according to claim 33, wherein said information reproducing section is adopted to set an information display attribute based on display attribute information contained in reproduced information.

35. An associated information presenting apparatus for superimposing and displaying the associated information on each section of said object at the corresponding position on an image on which said object is image-acquired, said apparatus comprising:

image input means for including in an image acquisition range a mark in which a pattern representing position information on four or more points of said object is displayed, thereby inputting an image on which said object is image-acquired;

position information management means for reading said pattern from an image inputted by said image input means, thereby acquiring said position information;

position information acquiring means for obtaining the position and orientation of said image input means from acquisition position information acquired by said position information management means;

associated information generating means for obtaining a position on said image of each section of said object based on the position and orientation of said image input means obtained by said position information acquiring means; and

information superimposing and displaying means for superimposing and displaying said associated information at the corresponding position based on the position on said image obtained by said associated information generating means.

36. An associated information presenting apparatus for superimposing and displaying the associated information on each section of said object at the corresponding position on an image on which said object is image-acquired, said apparatus comprising:

image input means for including in an image acquisition range a mark on which a pattern representing position information on four or more points of said object is displayed, thereby inputting an image on which said object is image-acquired;

position information management means for reading said pattern, thereby acquiring said position information;

position information acquiring means for obtaining the position and orientation of said image input means from said position information;

associated information generating means for obtaining a position on said image of each section of said object from the position and orientation of said image input means; and

information superimposing and displaying means for superimposing and displaying said associated information at the corresponding position based on the result of said associated information generating means.

37. An associated information presenting apparatus for superimposing and displaying the associated information on each section of said object at the corresponding position on an image on which said object is image-acquired, said apparatus comprising:

image input means for acquiring said object, thereby inputting an image;

position information management means for storing position information on four or more points of said object predetermined from a plurality of images on which the position and orientation of image input means are known;

position information acquiring means for obtaining the position and orientation of said image input means from said position information;

associated information generating means for obtaining a position on said image of each section of said object from the position and orientation of said image input means; and

information superimposing and displaying means for superimposing and displaying said associated information at the corresponding position based on the result of said associated information generating means.

38. An associated information presenting apparatus, comprising:

image input means;

mark recognition means for recognizing a mark in an input image from the image input means, thereby acquiring position information on the mark included in the recognized mark;

feature point recognition means for recognizing a feature point in a screen;

position information management means for performing association with a mark acquired by said mark recognition means, thereby acquiring position information corresponding to the feature point in the screen recognized by the feature point recognition means;

position information acquiring means for acquiring the position and orientation information on said image input means from position information on a feature point acquired by a mark acquired by said mark recognition means and/or said position information management means;

associated information generating means for generating information associated with the inside of the field of view for an input image based on the position and orientation information acquired by said position information acquiring means; and

superimposing and displaying means for superimposing and displaying on an input image the associated information generated by said associated information generating means.

39. An associated information presenting apparatus comprising:

image input means;

feature point recognition means for recognizing a feature point in a screen in which an image inputted by said input means is displayed;

mark recognition means for recognizing a mark in an input image from the image input means, thereby acquiring position information on the mark included in the recognized mark;

position information management means for performing association with a mark acquired by said mark recognition means, thereby acquiring position information corresponding to the feature point in the screen recognized by the feature point recognition means;

position information acquiring means for acquiring the position and orientation information on said input means from position information on a mark and/or a feature point;

associated information generating means for generating information associated with the inside of the field of view based on position and orientation information acquired by said position information acquiring means; and

superimposing and displaying means for superimposing and displaying associated information generated by said associated information generating means on an image inputted by said input means.

40. An information presenting apparatus comprising:

image input means for inputting an image;

pattern recognition means for recognizing a pattern presenting information in an image inputted by said image input means;

information reproducing means for reproducing information representing a pattern recognized by said pattern recognition means;

image superimposing means for superimposing information reproduced by said information reproducing means on an image inputted by said image input means; and

image display means for displaying an output from said image superimposing means.

41. An information presenting apparatus comprising:

image input means for inputting an image;

pattern recognition means for recognizing a predetermined pattern in an image inputted by said image input means;

information reproducing means for reproducing predetermined pattern information recognized by said pattern recognition means;

image superimposing means for superimposing information reproduced by said information reproducing means on an image inputted by said image input means; and

image display means for displaying an output from said image superimposing means.