System for and method of recognizating and tracking target mark

• US 5,617,335 A
• Filed: 01/18/1996
• Issued: 04/01/1997
• Est. Priority Date: 01/30/1992
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Hashima, Masayoshi, Watanabe, Ichiro, Okabayashi, Keiju, Murase, Yuichi, Kanda, Shinji, Sawasaki, Naoyuki, Hasegawa, Fumi
• Primary Examiner(s): Ramirez, Ellis B.
• Assistant Examiner(s): PEESO, THOMAS R
• Application Number:
  US08/588,320
• Time in Patent Office: 439 Days
• Field of Search: 364/516, 364/559, 340/815.57, 340/815.68, 382/10, 382/14, 382/18, 382/25, 382/51, 33/293
• US Class Current: 340/815.57
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B64G 1/646 Docking or rendezvous systems refueling in space B64G1/4024
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G05B 2219/39387 Reflex control, follow movement, track face, work, hand, visual servoing
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
• CPC Class Codes: G05B 2219/40613 Camera, laser scanner on end effector, hand eye manipulator, local
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis

Abstract

A system for and a method of recognizing and tracking a target mark with a video camera is disclosed. The system includes a target mark (10) disposed on an object (1) and composed of a black circle and a white triangle mounted centrally on the black circle and three-dimensionally shifted from the black circle, a video camera (20) for imaging the target mark (10), a robot (30) supporting the video camera (20) and movable in directions with six degrees of freedom, an image processor (40) for processing image data of the target mark which is produced by the video camera (20), a shift calculating unit (50) for detecting a shift of the target mark (10) from projected histogram information of the target mark (10) which is produced by the image processor (40), and a robot controller (60) for controlling movement of the robot depending on the shift to enable the video camera (20) to track the target mark (10). The system is capable of tracking the target mark (20) attached to the object (1) on a real-time basis. Mark recognizing apparatus capable of accurately recognizing target marks of other shapes is also disclosed.

Claims

1. A target mark attitude detecting method of detecting the attitude of a target mark to detect the attitude of an object about the direction of a camera based on an image produced by the camera of a target mark which is composed of at least a triangle of a particular shape, comprising the steps of:

determining projected histograms in X and Y directions of the image of the triangle of said target mark;

determining the positions of the centers of gravity in the X and Y directions of the image of the triangle of said target mark in said projected histograms;

determining maximum histogram values and X- and Y-axis values in said projected histograms;

determining which of classified and preset attitude patterns the attitude of the triangle of said target mark belongs to based on the positions of the centers of gravity, the maximum histogram values, the X- and Y-axis values, and known geometrical data of said target mark; and

calculating the attitude of the triangle of the target mark in the determined attitude pattern about the direction of the camera.

2. A target mark attitude detecting method according to claim 1, wherein the attitude of the triangle of the target mark about the direction of the camera is compared with a reference attitude of the triangle of the target mark about the direction of the camera to determine a shift of a roll angle of the triangle of the target mark about the direction of the camera.

3. A target mark attitude detecting method according to claim 1, wherein if the classified and preset attitude patterns include a plurality of different attitudes, then the attitude of the target mark is detected again after rolling the attitude of the camera slightly.

4. A target mark attitude detecting method according to claim 1, wherein said target mark includes a disk disposed behind said triangle, and the projected histograms of said triangle in the X and Y directions are determined by deducing measured projected histograms in the X and Y directions of the image of said target mark from calculated projected histograms in the X and Y directions of an image of only said disk.

5. A target mark attitude detecting method according to claim 4, wherein said projected histograms in the X and Y directions of the image of only said disk are calculated based on known geometrical data of said disk by determining two coordinates where histogram values are predetermined values, in each of the measured projected histograms in the X and Y directions of the image of said target mark, and regarding a center of the determined two coordinates as the center of said disk.

6. A target mark attitude detecting method according to claim 4, comprising the steps of:

determining the positions of the centers of gravity of the measured projected histograms in the X and Y directions of the image of said target mark which is composed of the triangle and the disk disposed behind the triangle;

determining central coordinates of the image of said disk;

determining the ratio between the area of said disk and the area of said triangle; and

calculating the position of the center of gravity of said triangle based on the positions of the centers of gravity of the target mark, the central coordinates of said disk, and said ratio.

7. A position and attitude measuring apparatus for measuring the position and the attitude of an object based on an image of a target mark, comprising:

a target mark disposed on a particular flat surface of an object and composed of a circle and a central point thereof;

a camera for imaging said target mark to generate an image of the circle and the central point thereof;

feature extracting means for extracting feature points required to measure the position and attitude of said object, from the image of said target mark; and

calculating means for calculating the position and attitude of said target mark in an object space according to projective geometrical calculations on said feature points.

8. A position and attitude measuring apparatus according to claim 7, wherein said feature extracting means extracts two image points for defining a diameter parallel to an image plane of the target mark among diameters of the circle in the object space, from the image of the circle of said target mark, and said calculating means calculates the position of said target mark in said object space based on the distance between the extracted two image points, the value of the diameter of the circle of said target mark, and the coordinates in an image space of the central point of said target mark imaged by said camera.

9. A position and attitude measuring apparatus according to claim 8, wherein said feature extracting means determines a diameter which is divided into two equal parts by the image of the central point of said target mark in the image plane of said target mark, among the diameters of the circle of said target mark to thereby extract said two image points.

10. A position and attitude measuring apparatus according to claim 7, wherein said calculating means calculates a diametrical vector in the object space of said target mark from the coordinates in the image plane of optional two points on a circumference defining the diameter of the circle of said target mark and the central point of the circle, and wherein a normal vector of said target mark in said object space is determined from at least two diametrical vectors with respect to said target mark thereby to determine the attitude of the object to which said target mark is attached.

11. A position and attitude of an object measuring apparatus according to claim 10, wherein said calculating means determines a diametrical vector parallel to the image plane of said target mark from the outer product of a normal vector of said target mark and a normal vector of the image plane thereby to calculate the position of said target mark in said object space based on the distance between the two image points in the image space of the two points on the circumference defining the diameter of the circle of said target mark which is determined by said diametrical vector, the value of the diameter of the circle of said target mark, and the coordinates in the image space of the central point of said target mark imaged by said camera.

12. A position and attitude measuring apparatus according to claim 7, wherein the central point of the circle of said target mark is indicated by two intersecting line segments.

13. A position and attitude measuring apparatus according to claim 7, wherein the circle of said target mark is divided into regions by two diameters.

14. A distance and attitude measuring apparatus for measuring the distance up to and the attitude of an object, comprising:

four disk-shaped marks disposed on an object and having respective centers of gravity positioned in one plane, at least one of said disk-shaped marks having a radius different from the radii of the other disk-shaped marks;

imaging means for imaging said disk-shaped marks;

center-of-gravity calculating means for calculating the positions of the centers of gravity of the respective disk-shaped marks based on image data of said four disk-shaped marks which are outputted by said imaging means; and

calculating means for solving a four-point perspective problem to calculate the distance up to and the attitude of the object based on the positions of the centers of gravity calculated by said center-of-gravity calculating means.

15. A distance and attitude measuring apparatus according to claim 14, wherein said center-of-gravity calculating means converts the image data of said four disk-shaped marks which are outputted by said imaging means to binary image data, generates vertically and horizontally projected histograms of the disk-shaped marks, and calculates the positions of the centers of gravity of the disk-shaped marks by way of weighted means.

16. A distance and attitude measuring apparatus for measuring the distance up to and the attitude of an object, comprising:

first and second disk-shaped marks having respective different radii and disposed on a first plane;

third and fourth disk-shaped marks disposed on a second plane which is different from said first plane;

imaging means for imaging the first through fourth disk-shaped marks;

center-of-gravity calculating means for calculating the positions of the centers of gravity of said first and second disk-shaped marks based on image data of said first through fourth disk-shaped marks which are outputted by said imaging means, and projecting said third and fourth disk-shaped marks toward said imaging means to calculate the positions of the centers of gravity in respective projected figures on said first plane; and

calculating means for solving a four-point perspective problem to calculate the distance up to and the attitude of the object based on the four positions of the centers of gravity calculated by said center-of-gravity calculating means.

17. A distance and attitude measuring apparatus according to claim 16, wherein said third and fourth disk-shaped marks are positioned on a line which is perpendicular to a plane passing through the centers of gravity of said first and second disk-shaped marks and which passes through the centers of gravity of said first and second disk-shaped marks.

18. A distance and attitude measuring apparatus according to claim 17, wherein said third and fourth disk-shaped marks have respective radii smaller than the radii of said first and second disk-shaped marks.

19. A distance and attitude measuring apparatus for measuring the distance up to and the attitude of an object, comprising:

first through fourth disk-shaped marks mounted on an object and having respective centers of gravity positioned on one plane, at least one of said disk-shaped marks having a radius different from the radii of the other disk-shaped marks, said centers of gravity of said first through fourth disk-shaped marks being positioned on a first plane;

fifth and sixth disk-shaped marks disposed on a second plane which is different from said first plane;

imaging means for imaging the first through sixth disk-shaped marks;

first center-of-gravity calculating means for calculating the positions of the centers of gravity of said third and fourth disk-shaped marks based on image data of said third through sixth disk-shaped marks which are outputted by said imaging means, and projecting said fifth and sixth disk-shaped marks toward said imaging means to calculate the positions of the centers of gravity in respective projected figures on said first plane;

second center-of-gravity calculating means for calculating the positions of the centers of gravity of the disk-shaped marks based on image data of said first through fourth disk-shaped marks which are outputted by said imaging means; and

calculating means for solving a four-point perspective problem to calculate the distance up to and the attitude of the object based on the four positions of the centers of gravity calculated by said first or second center-of-gravity calculating means.

20. A distance and attitude measuring apparatus according to claim 19, wherein said calculating means solves a four-point perspective problem to calculate the distance up to and the attitude of the object based on the four positions of the centers of gravity calculated by said first center-of-gravity calculating means while the four positions of the centers of gravity are being calculated by said first center-of-gravity calculating means, and solves a four-point perspective problem to calculate the distance up to and the attitude of the object based on the four positions of the centers of gravity calculated by said second center-of-gravity calculating means when said first center-of-gravity calculating means cannot calculate the four positions of the centers of gravity.