SYSTEM AND METHOD FOR CAMERA-BASED AUTO-ALIGNMENT

• US 20140100694A1
• Filed: 10/04/2013
• Published: 04/10/2014
• Est. Priority Date: 10/05/2012
• Status: Abandoned Application

• Current Assignee: Beckman Coulter Incorporated (Danaher Corporation)
• Original Assignee: Beckman Coulter Incorporated (Danaher Corporation)
• Inventors: Rueckl, Stefan, Streibl, Sebastian, Sickert, Manuel
• Application Number:
  US14/046,829
• Publication Number:
  US 20140100694A1
• Time in Patent Office: Days
• US Class Current: 700/254
• CPC Class Codes: B25J 9/1692 Calibration of manipulator
• CPC Class Codes: G05B 2219/39045 Camera on end effector detects reference pattern
• CPC Class Codes: G05B 2219/39057 Hand eye calibration, eye, camera on hand, end effector
• CPC Class Codes: G05B 2219/40613 Camera, laser scanner on end effector, hand eye manipulator, local

Abstract

A camera-based auto-alignment process can include gripping a first calibration tool by a gripper unit of a robotic arm. Images of the first calibration tool can be captured by a camera coupled to the gripper unit. The gripper unit and camera unit can be aligned on two roughly parallel axes. The images can be analyzed to calibrate the axis of view of the camera with the gripper axis, providing an XY calibration of the robotic arm. The gripper unit can be calibrated on a Z-axis using optical calibration with landmarks provided on a second calibration tool, and/or by moving the gripper unit towards the work surface until it makes contact with the work surface and stops. Once calibrated, the camera can be used to identify one or more landmarks at known locations on the work surface to align the robotic arm with the work surface.

Claims

1. A method for auto-alignment, comprising:

gripping an X-Y calibration tool by a gripper unit of a robotic arm at a first height above a work surface on a first axis;

acquiring images of the X-Y calibration tool at the first height by a camera on a second axis, wherein the camera is coupled to the gripper unit; and

analyzing the images of the first calibration tool at the first height to determine an offset between the second axis and the first axis.

2. The method of claim 1, further comprising:

rotating the X-Y calibration tool through a field of view of the camera, wherein the X-Y calibration tool includes a substantially flat portion including a plurality of landmarks; and

wherein analyzing the images of the X-Y calibration tool comprises

identifying a plurality of elliptical paths, each corresponding to one of the landmarks in the images,

determining a center point of the plurality of elliptical paths corresponding to the first axis,

based on the center point of the plurality of elliptical paths, determining an offset between the first axis and second axis.

3. The method of claim 2, further comprising:

positioning the robotic arm such that the camera is centered over a landmark and recording a first position of the landmark in pixels;

moving the robotic arm a predetermined number of steps in an X direction and a Y direction, and recording a second position of the landmark in pixels;

determining a step to pixel conversion ratio based on a difference between the first position and the second position and the predetermined number of steps.

4. The method of claim 2, further comprising:

gripping the X-Y calibration tool at a second height;

acquiring images of the X-Y calibration tool at the second height by the camera;

analyzing the images of the X-Y calibration tool at the second height to determine a second offset between the second axis and the first axis;

gripping the X-Y calibration tool at a third height;

acquiring images of the X-Y calibration tool at the third height by the camera;

analyzing the images of the X-Y calibration tool at the third height to determine a third offset between the second axis and the first axis; and

determining a linear offset function using the three offsets and three heights.

5. The method of claim 1, wherein at least one landmark on the X-Y calibration tool is a periodically repeating pattern and wherein analyzing the images of the X-Y calibration tool to determine an offset between the second axis and the first axis further comprises:

analyzing the images to determine at least one distortion correction parameter using the periodically repeating pattern, wherein the at least one distortion correction parameter can be used to correct lens-related distortions in the images; and

applying the at least one distortion correction parameter to the images to produce distortion corrected images.

6. The method of claim 1, further comprising:

identifying one or more landmarks on one or more elements on a work surface to align the robotic arm to the work surface.

7. The method of claim 1, wherein the camera is coupled to the gripper unit such that the camera is maintained at a fixed height as the gripper unit moves along a Z axis orthogonal to the work surface.

8. A method for auto-alignment, comprising:

positioning a gripper unit of a robotic arm at a predetermined height above a Z calibration tool, wherein the Z-calibration tool includes a plurality of landmarks on a plurality of levels;

calibrating the gripper unit along a Z-axis over a first landmark on the Z calibration tool, wherein calibrating includes

moving the gripper unit toward the first landmark on the Z calibration tool along the Z-axis until the gripper unit makes contact with the Z calibration tool; and

determining a first number of steps traveled by the gripper unit from the predetermined height to making contact with the Z calibration tool.

9. The method of claim 8, wherein calibrating the gripper unit along a Z-axis over a first landmark further includes:

determining a first height in pixels of the gripper unit over the first landmark on the Z calibration tool using triangulation.

10. The method of claim 8, further comprising:

calibrating the gripper unit along the Z-axis over at least two additional landmarks on the Z calibration tool; and

based on calibration results for each of the landmarks, determining a distance function that converts height in pixels to height in steps.

11. The method of claim 8, wherein at least one landmark on the Z calibration tool is a periodically repeating pattern and wherein calibrating the gripper on a Z-axis using a Z calibration tool further comprises:

analyzing images of the Z calibration tool to determine at least one distortion correction parameter using the periodically repeating pattern, wherein the at least one distortion correction parameter can be used to correct lens-related distortions in the images; and

applying the at least one distortion correction parameter to the images to produce distortion corrected images.

12. A method for auto-alignment, comprising:

gripping a calibration tool by a gripper unit on a first axis;

acquiring images of the calibration tool by a camera on a second axis, wherein the camera is connected to the gripper;

analyzing the images to determine at least one distortion correction parameter, wherein the at least one distortion correction parameter can be used to correct lens-related distortions in the images; and

applying the at least one distortion correction parameter to the images to produce distortion corrected images.

13. The method of claim 12, further comprising:

analyzing the distortion corrected images to determine an offset between the second axis and the first axis using one or more landmarks on a X-Y calibration tool shown in the distortion corrected images.

14. The method of claim 13, further comprising:

calibrating the gripper on a Z-axis using the distortion corrected images using one or more landmarks provided on a Z calibration tool, wherein calibrating the gripper on the Z-axis comprises triangulating landmarks provided on the Z calibration tool.

15. The method of claim 14, wherein calibrating the gripper on a Z-axis further comprises physically calibrating the gripper by moving the gripper toward the Z calibration tool along the Z-axis until the gripper makes contact with the Z calibration tool.

16. An assembly, comprising:

a robotic arm, including a gripper unit, wherein the robotic arm is configured to move in three dimensions over a work surface; and

a camera coupled to the gripper unit such that the camera is maintained at a fixed height as the gripper unit moves along a Z axis substantially orthogonal to the work surface.

17. The assembly of claim 16, further comprising:

an auto-alignment system, including one or more controllers coupled to the robotic arm and camera, wherein the auto-alignment system is configured to instruct the robotic arm to grip an X-Y calibration tool and rotate the X-Y calibration tool through a field of vision of the camera, and wherein the auto-alignment system is configured to instruct the camera to capture images of the X-Y calibration tool as it is rotated; and

wherein the auto-alignment system is further configured to analyze the images of the X-Y calibration tool to determine an offset between a first axis corresponding to the gripper unit and a second axis corresponding to the camera.

18. The assembly of claim 17, wherein the auto-alignment system is further configured to:

identify a plurality of elliptical paths, each corresponding to one of the landmarks in the images,

determine a center point of the plurality of elliptical paths corresponding to the first axis,

based on the center point of the plurality of elliptical paths, determine an offset between the first axis and second axis.

19. The assembly of claim 18, wherein the auto-alignment system is further configured to:

instruct the robotic arm to grip the X-Y calibration tool at a second height;

instruct the camera to acquire images of the X-Y calibration tool at the second height;

analyze the images of the X-Y calibration tool at the second height to determine a second offset between the second axis and the first axis;

instruct the robotic arm to grip the X-Y calibration tool at a third height;

acquiring images of the X-Y calibration tool at the third height by the camera;

instruct the camera to the images of the X-Y calibration tool at the third height to determine a third offset between the second axis and the first axis; and

determine a linear offset function using the three offsets and three heights.

20. The assembly of claim 17, wherein the auto-alignment system is further configured to:

position the robotic arm such that the camera is centered over a landmark and recording a first position of the landmark in pixels;

move the robotic arm a predetermined number of steps in an X direction and a Y direction, and recording a second position of the landmark in pixels;

determine a step to pixel conversion ratio based on a difference between the first position and the second position and the predetermined number of steps.

21. The assembly of claim 17, wherein the auto-alignment system is further configured to:

instruct the camera to capture images of a landmark including a periodically repeating pattern;

analyze the images of the landmark including the periodically repeating pattern to determine at least one distortion correction parameter using the periodically repeating pattern, wherein the at least one distortion correction parameter can be used to correct lens-related distortions in the images; and

apply the at least one distortion correction parameter to the images of the X-Y calibration tool to produce distortion corrected images.

22. The assembly of claim 17, wherein the auto-alignment system is further configured to:

position the gripper unit of the robotic arm at a predetermined height above a Z calibration tool, wherein the Z calibration tool includes a plurality of landmarks on a plurality of levels;

calibrate the gripper unit along a Z-axis over a first landmark on the Z calibration tool, wherein calibrating includes

determining a first height in pixels of the gripper unit over the first landmark on the Z calibration tool using triangulation;

moving the gripper unit toward the first landmark on the Z calibration tool along the Z-axis until the gripper unit makes contact with the Z calibration tool; and

determining a first number of steps traveled by the gripper unit from the predetermined height to making contact with the Z calibration tool.

23. The assembly of claim 22, wherein the auto-alignment system is further configured to:

calibrate the gripper unit along the Z-axis over at least two additional landmarks on the Z calibration tool; and

based on calibration results for each of the landmarks, determine a distance function that converts height in pixels to height in steps.