System for and method of recognizing and tracking target mark

• US 5,521,843 A
• Filed: 09/28/1993
• Issued: 05/28/1996
• Est. Priority Date: 01/30/1992
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Hashima, Masayoshi, Watanabe, Ichiro, Okabayashi, Keiju, Murase, Yuichi, Kanda, Shinji, Sawasaki, Naoyuki, Hasegawa, Fumi
• Primary Examiner(s): Voeltz, Emanuel T.
• Assistant Examiner(s): PEESO, THOMAS R
• Application Number:
  US08/119,228
• Time in Patent Office: 973 Days
• Field of Search: 364/167.01, 364/559, 364/516, 340/815.54, 340/815.57, 340/815.68, 382/10, 382/14, 382/18, 382/25, 382/51, 382/65, 250/203 CT, 33/293, 348/94
• US Class Current: 700/253
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B64G 1/646 Docking or rendezvous systems refueling in space B64G1/4024
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G05B 2219/39387 Reflex control, follow movement, track face, work, hand, visual servoing
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
• CPC Class Codes: G05B 2219/40613 Camera, laser scanner on end effector, hand eye manipulator, local
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis

Abstract

A system for and a method of recognizing and tracking a target mark with a video camera is disclosed. The system includes a target mark (10) disposed on an object (1) and composed of a black circle and a white triangle mounted centrally on the black circle and three-dimensionally shifted from the black circle, a video camera (20) for imaging the target mark (10), a robot (30) supporting the video camera (20) and movable in directions with six degrees of freedom, an image processor (40) for processing image data of the target mark which is produced by the video camera (20), a shift calculating unit (50) for detecting a shift of the target mark (10) from projected histogram information of the target mark (10) which is produced by the image processor (40), and a robot controller (60) for controlling movement of the robot depending on the shift to enable the video camera (20) to track the target mark (10). The system is capable of tracking the target mark ( 20) attached to the object (1) on a real-time basis. Mark recognizing apparatus capable of accurately recognizing target marks of other shapes is also disclosed.

Claims

1. A target mark tracking system for tracking a target mark with a video camera, comprising:

a target mark disposed on an object and composed of a black circle and a white triangle centered within said black circle and three-dimensionally shifted from the black circle;

a video camera for imaging said target mark;

a moving mechanism supporting said video camera and movable in directions with six degrees of freedom;

image processing means for processing image data of said target mark from a projected position based on histogram information of said target mark which is produced by said image processing means; and

moving mechanism control means for controlling movement of said moving mechanism depending on said shift to enable said video camera to track said target mark;

wherein said image processing means processes the image data of said target mark in an image coordinate system to recognize the target mark, store the image data, convert the image data to binary image data, and generate X- and Y-projected histograms.

2. A target mark tracking system according to claim 1, wherein said shift detecting means detects shifts in X- and Y-axis directions of said target mark from the difference between the position of the center of gravity of said target mark which is determined from said X- and Y-projected histograms and a predetermined target position for said target mark.

3. A target mark tracking system according to claim 1, wherein said shift detecting means detects a shift in a Z-axis direction of said target mark from the difference between the area of said target mark which is determined from said X- or Y-projected histogram and a predetermined target area for said target mark.

4. A target mark tracking system according to claim 1, wherein said shift detecting means determines a shift of the white triangle with respect to the black circle of said target mark from a peak determined from said X-projected histogram, and regards the determined shift as a shift in a yaw direction of said target mark.

5. A target mark tracking system according to claim 1, wherein said shift detecting means determines a shift of the white triangle with respect to the black circle of said target mark from a peak determined from said Y-projected histogram, and regards the determined shift as a shift in a pitch direction of said target mark.

6. A target mark tracking system according to claim 1, wherein said shift detecting means determines a shift in a roll direction of said target mark from a peak and a low peak determined from said X-projected histogram.

7. A target mark tracking system according to claim 1, wherein said image processing means determines the position and size of said window from the position of the center of gravity and the size of said target mark which are determined from said X- and Y-projected histograms.

8. A visual target mark tracking control system for imaging a target mark with a video camera and processing image data of the target mark produced by the video camera to hold said video camera in a predetermined positional relationship to the target mark at all times, comprising:

image change detecting means for detecting a change from a target position and attitude for said target mark in an image of said target mark produced by said video camera;

actual change detecting means for detecting a relative actual change from the predetermined positional relationship between said video camera and said target mark; and

relating means for experimentally shifting the predetermined positional relationship between said video camera and said target mark, and relating values which are detected by said image change detecting means and said actual change detecting means when the predetermined positional relationship is experimentally shifted, to each other.

9. A visual target mark tracking control system according to claim 8, wherein said relating means has first deriving means for relating a detected value representing the area of said target mark in the image thereof, among the values detected by said image change detecting means, to a distance between said video camera and said target mark, among the values detected by said actual change detecting means, to derive the distance between said video camera and said target mark from said detected value representing the area of said target mark.

10. A visual target mark tracking control system according to claim 9, wherein said relating means has second deriving means for relating the values detected by said image change detecting means to the distance between said video camera and said target mark among the values detected by said actual change detecting means and also to the values detected by said actual change detecting means, and deriving a relative actual change from the predetermined positional relationship between said video camera and said target mark from said distance derived by said first deriving means and the values detected by said image change detecting means.

11. A visual target mark tracking control system for imaging a target mark with a video camera and processing image data of the target mark produced by the video camera to hold said video camera in a predetermined positional relationship to the target mark at all times, comprising:

image change detecting means for detecting a change from a target position and attitude for said target mark in an image of said target mark produced by said video camera;

actual change detecting means for detecting a relative actual change from the predetermined positional relationship between said video camera and said target mark;

deriving means for experimentally shifting the predetermined positional relationship between said video camera and said target mark, relating values which are detected by said image change detecting means and said actual change detecting means when the predetermined positional relationship is experimentally shifted, to each other, and deriving a relative actual change from the predetermined positional relationship between said video camera and said target mark from the values detected by said image change detecting means; and

control means for multiplying said actual change derived by said deriving means by a proportional gain, calculating a speed command in a coordinate system which is shifted by the distance between said video camera and said target mark with respect to a rotational component, and using the calculated speed command as a speed command for positionally controlling said video camera.

12. A visual target mark tracking control system for imaging a target mark with a video camera and processing image data of the target mark produced by the video camera to hold said video camera in a predetermined positional relationship to the target mark at all times, comprising:

image change detecting means for detecting a change from a target position and attitude for said target mark in an image of said target mark produced by said video camera;

actual change detecting means for detecting a relative actual change from the predetermined positional relationship between said video camera and said target mark;

deriving means for experimentally shifting the predetermined positional relationship between said video camera and said target mark, relating values which are detected by said image change detecting means and said actual change detecting means when the predetermined positional relationship is experimentally shifted, to each other, and deriving a relative actual change from the predetermined positional relationship between said video camera and said target mark from the values detected by said image change detecting means; and

control means for determining a shift of the attitude of said video camera in a coordinate system fixed to said video camera from the actual change derived by said deriving means, and multiplying the determined shift by a constant gain to produce a speed command for positionally controlling said video camera.

13. A method of detecting a target mark comprising the steps of:

converting an original image to binary images;

grouping the binary images into images with joined pixels;

determining X- and Y-projected histograms of the grouped images;

counting extreme values of the X- and Y-projected histograms of the grouped images; and

comparing the counted extreme values with predetermined extreme values of X- and Y-histograms of a target mark to determine whether the grouped images represent the target mark.

14. A method according to claim 13, wherein the target mark is composed of at least two marks having different features of the numbers of extreme values and combined in a predetermined positional relationship, each of said steps is carried out with respect to each mark in said target mark to detect each mark and measure coordinate points thereof, and an n-point perspective problem is solved with respect to the coordinate points to measure the position and attitude of the target mark.