Automatic calibration of cameras and structured light sources

• US 6,101,455 A
• Filed: 05/14/1998
• Issued: 08/08/2000
• Est. Priority Date: 05/14/1998
• Status: Expired due to Fees

• Current Assignee: Michael S. Davis
• Original Assignee: Michael S. Davis
• Inventors: Davis, Michael S.
• Primary Examiner(s): Hoff, Marc S.
• Assistant Examiner(s): BUI, BRYAN
• Application Number:
  US09/079,422
• Time in Patent Office: 817 Days
• Field of Search: 702/94, 702/95, 702/150, 702/151, 702/152, 702/153, 901/42, 901/47, 700/245, 700/253, 700/254, 700/259, 700/251, 382/153, 382/154
• US Class Current: 702/94
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 7/4972 Alignment of sensor
• CPC Class Codes: G05B 2219/39008 Fixed camera detects reference pattern held by end effector
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19

Abstract

A work cell containing robot(s), video camera(s), and structured lighting source(s) is calibrated by observing targets with the camera(s) as a robot is displaced through a set of offsets. Complete information is recovered about the camera(s) calibration data and the structure of illumination from the light source(s). The robot is utilized to create known relative movements between the targets and the camera(s) and light source(s). Therefore, this technique is applicable to both the fixed and moving camera cases. Except for the target surface profile, there is no requirement to externally determine any absolute or relative positions, or any relationships either within or between the camera(s), targets, light source(s), and robot(s). Either single or multiple cameras (acting independently or in stereo) are calibrated to the robot'"'"'s coordinate frame, and then optionally used as measuring devices to determine the position and form of the structured light.

Claims

1. In a work cell of the type containing one or more robots, each having a coordinate frame, and one or more video cameras, a method of calibrating at least one of the cameras, comprising the steps of:

equipping one of the robots with one or more fiducials movable in three space;

moving the robot and/or the camera relative to one another through a series of offsets while observing and recording the image of each fiducial using the camera to be calibrated; and

determining the relationship between the relative position of the camera and the robot'"'"'s coordinate frame using only fiducial observations as the robot is moved through the series of offsets.

2. The method of claim 1, wherein the camera to be calibrated remains stationary while the target is moved through the series of offsets.

3. The method of claim 1, wherein the camera is moved while the target remains stationary.

4. The method of claim 1, including the step of calibrating a plurality of cameras through a series of offsets between each camera and at least one target.

5. The method of claim 1, wherein the work cell contains at least one structured light source, and wherein the target includes a surface profile, the method further including the step of using the camera being calibrated as measuring a device to determine the position and form of the structured light based upon the surface profile.

6. The method of claim 1, wherein the step of equipping one of the robots with a plurality of fiducials movable in three space includes the step of providing the robot with a target including such fiducials in the form of surface features.

7. The method of claim 1, wherein the relationship between the robot'"'"'s coordinate frame and at least one of the fiducials is known, such that the offset need only include a series of translations.

8. The method of claim 1, wherein the offsets include at least one rotation to determine the relationship between the robot'"'"'s coordinate frame and at least one of the fiducials.

9. The method of claim 1, wherein the offsets include a series of translations.

10. The method of claim 1, wherein the offsets include a series of rotations.

11. The method of claim 1, wherein the offsets include moving a combination of translations and rotations.

12. A method of automatically calibrating a single camera having a field of view in work-cell environment including a robotic manipulator with an end effector, comprising the steps of:

providing a calibration target relative to the end effector having at least three distinct features that can be resolved in the camera'"'"'s field of view;

moving the target in three space while remaining in the field of view of the camera by sending a series of offsets to the robot;

recording each offset position of the target in three space by observing the positions of the target features;

computing a first set of camera calibration data with respect to a temporary coordinate frame and the three-dimensional offset target positions;

aligning the temporary coordinate frame with the robot'"'"'s frame;

displacing the target through a second set of offsets in three dimensions while keeping the target in the camera'"'"'s field of view;

computing the positions of the features in three space;

determining the origin of the robot'"'"'s frame in accordance with the recorded offsets and computed feature positions.

13. The method of claim 12, wherein the step of displacing the target through a second set of offsets in three dimensions includes performing at least one rotation in three space.

14. The method of claim 12, wherein the step of displacing the target through a second set of offsets in three dimensions includes the step of performing a series of translations.

15. The method of claim 12, wherein at least two distinct positions of the target are viewed by the camera for each of the target orientations.

16. The method of claim 12, wherein the work cell includes a structured light source, and wherein the method further includes a process for determining the shape of structured light, comprising the steps of:

moving the target over a set of positions such that the light striking the target is in the camera'"'"'s field of view;

observing and recording the shape of the light striking the target surface at each position; and

computing the shape of the light in three space in accordance with the recorded position and orientation data.

17. The method of claim 16, wherein the target surface is a planar surface.

18. The method of claim 16, wherein the target surface is non-planar.

19. The method of claim 18, wherein the structured light is a plane of light, enabling the camera to image a line in the vicinity of where the two planes intersect.

20. The method of claim 16, wherein the relationship between the end effector and the target is known, enabling offsets to be used in each target position in the set.

21. The method of claim 16, wherein the relationship between the end effector and the target is unknown, the method further including the step of performing a relative offset between the camera and the target at a fixed orientation to ascertain the target'"'"'s position in space over the set of positions.