Automated guidance and recognition system and method of the same

• US 8,923,602 B2
• Filed: 07/22/2008
• Issued: 12/30/2014
• Est. Priority Date: 07/22/2008
• Status: Active Grant

• Current Assignee: Recognition Robotics, Inc., Comau LLC (Stellantis NV)
• Original Assignee: Recognition Robotics, Inc., Comau Incorporated (Stellantis NV)
• Inventors: Melikian, Simon, Falcone, Maximiliano A., Cyrek, Joseph
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US12/177,529
• Publication Number:
  US 20100021051A1
• Time in Patent Office: 2,352 Days
• Field of Search: 382106-108, 382141-145, 382151-155, 382159-161, 382/181, 382/190, 382/195, 382/203, 382/274, 382/276, 382/286, 382/289, 382/312, 382/321, 700252-254, 600/130, 435/5, 348/86, 348/127
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/37561 Move camera until image corresponds to stored image of same workpiece
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
• CPC Class Codes: G05B 2219/40003 Move end effector so that image center is shifted to desired position
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 2201/11 Technique with transformation invariance effect

Abstract

Disclosed herein are embodiments and methods of a visual guidance and recognition system requiring no calibration. One embodiment of the system comprises a servo actuated manipulator configured to perform a function, a camera mounted on the face plate of the manipulator, and a recognition controller configured to acquire a two dimensional image of the work piece. The manipulator controller is configured to receive and store the face plate position at a distance “A” between the reference work piece and the manipulator along an axis of the reference work piece when the reference work piece is in the camera'"'"'s region of interest. The recognition controller is configured to learn the work piece from the image and the distance “A”. During operation, a work piece is recognized with the system, and the manipulator is accurately positioned with respect to the work piece so that the manipulator can accurately perform its function.

Claims

1. A system for moving a servo actuated manipulator without a calibration target, comprising:

a servo actuated manipulator configured to perform a function;

a camera mounted on or in fixed relation with a face plate of the manipulator;

a manipulator controller configured to move the manipulator; and

a recognition system configured to;

signal the manipulator controller to move the manipulator from a work position at a reference work piece to a distance “

A”

between the reference work piece and the manipulator along a coordinate axis of the reference work piece,

acquire a two dimensional image via the camera of at least a portion of the reference work piece,

learn the reference work piece based on the two dimensional image and the distance “

A”

associated with the reference work piece, and

with the introduction of a production workpiece, signal the manipulator controller to move the manipulator to a work position at the production work piece based on the learning of the reference work piece and the distance “

A”

associated with the reference work piece.

2. The system of claim 1, wherein the recognition system is further configured to center the portion of the reference work piece in a region of interest by moving the manipulator along a coordinate axis parallel to the reference work piece prior to acquiring the two dimensional image.

3. The system of claim 1, wherein with the introduction of a production work piece to the manipulator, the recognition system is further configured to:

with the servo actuated manipulator moved to a position corresponding to the distance “

A”

associated with the reference work piece, acquire a two dimensional image via the camera of at least a portion of the production work piece, and

signal the manipulator controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on dimensional data offsets between the two dimensional images of the reference work piece and the production work piece.

4. The system of claim 3, wherein the recognition system is further configured to repeat calculation of the dimensional data offsets until a threshold offset limit is reached.

5. The system of claim 3, wherein the manipulator is configured to move dx, dy, dz, rx, ry, rz around a center of rotation located either on the face plate or along the coordinate axis.

6. The system of claim 3, wherein when the recognition system signals the manipulator controller to move the manipulator, the manipulator moves in two steps, the first step by an amount of dx, dy, dz, and the second step by an amount of rx, ry, rz based on the dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.

7. The system of claim 1, wherein the recognition system is further configured to learn and store in a database a large number of different reference work pieces each with an associated two dimensional image and a distance “

A”

.

8. The system of claim 7, wherein, when a production work piece is introduced to the manipulator, the recognition system is further configured to acquire a two dimensional image via the camera of at least a portion of the production work piece and to recognize the production work piece as one of the large number of different reference work pieces in the database.

9. The system of claim 8, wherein, if the production work piece is not recognized, the manipulator controller moves the manipulator along the coordinate axis until the distance “

A”

is such that recognition system recognizes the production work piece.

10. The system of claim 8, wherein, once the production work piece is recognized, the recognition system is further configured to use the existing two dimensional image of the production work piece and signal the manipulator controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.

11. The system of claim 10, wherein the recognition system is further configured to repeat calculation of the dimensional data offsets until a threshold offset limit is reached.

12. The system of claim 1, further comprising:

the recognition system comprising a database of at least one reference work piece; and

wherein the recognition system is further configured to acquire a two dimensional image via the camera of at least a portion of a production work piece, match the image of the production work piece with the reference work piece in the database, calculate a dimensional data offset between the image of the production work piece and an image associated with the reference work piece, and signal the manipulator controller to move the manipulator to decrease the offset until the offset is below a threshold limit.

13. A method for controlling the movement of a servo actuated manipulator without the need for calibration, using a camera, comprising:

positioning a reference work piece;

moving the manipulator to a work position at the reference work piece;

moving the manipulator along a coordinate axis of the reference work piece from the work position to a distance “

A”

wherein at least a portion of the reference work piece is in a region of interest of the camera;

acquiring a two dimensional image via the camera of at least a portion of the reference work piece;

learning the reference work piece based on the two dimensional image and the distance “

A”

associated with the reference work piece; and

with the introduction of a production workpiece, moving the manipulator to a work position at the production work piece based on the learning of the reference work piece and the distance “

A”

associated with the reference work piece.

14. The method of claim 13 further comprising:

centering the portion of the reference work piece in the region of interest by moving the manipulator along a coordinate axis parallel to the reference work piece prior to acquiring the two dimensional image.

15. The method of claim 13 further comprising, with the instruction of a production work piece:

moving the manipulator to a position corresponding to the distance “

A”

associated with the reference work piece;

acquiring a two dimensional image via the camera of at least a portion of the production work piece;

calculating dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece; and

moving the manipulator by an amount of dx, dy, dz, rx, ry, rz based on the dimensional data offsets.

16. The method of claim 15, further comprising repeating calculating the dimensional data offsets and moving the manipulator until a threshold offset limit is reached.

17. The method of claim 13 further comprising:

repeating the method for each of a plurality of different reference work pieces; and

creating a database of each two dimensional image and distance “

A”

associated with each of the plurality of difference reference work pieces.

18. The method of claim 17 further comprising, with the introduction of a production work piece:

moving the manipulator to a position corresponding to the distance “

A”

associated with the reference work piece;

acquiring a two dimensional image via the camera of at least a portion of the production work piece;

comparing the two dimensional image of the production work piece with the database of reference work pieces;

recognizing the production work piece from a reference work piece in the database;

calculating dimensional data offsets between the two dimensional images of the reference work piece and the production work piece; and

moving the manipulator by an amount of dx, dy, dz, rx, ry, rz based on the dimensional data offsets.

19. The method of claim 18, wherein if the production work piece is not recognized, moving the manipulator along the coordinate axis to change the distance “

A”

such that a reference work piece from the database is recognized.

20. The method of claim 18, further comprising repeating calculating the dimensional data offsets and moving the manipulator controller until a threshold offset limit is reached.

21. The method of claim 18, wherein the manipulator moves in two steps, the first step by an amount of dx, dy, dz, and the second step by an amount of rx, ry, rz based on dimensional data offsets between the two dimensional images of the reference work piece and the production work piece.