Automated Guidance and Recognition System and Method of the Same

• US 20100021051A1
• Filed: 07/22/2008
• Published: 01/28/2010
• Est. Priority Date: 07/22/2008
• Status: Active Grant

• Current Assignee: Recognition Robotics, Inc., Comau LLC (Stellantis NV)
• Original Assignee: Recognition Robotics, Inc., Comau Incorporated (Stellantis NV)
• Inventors: Falcone, Maximiliano A., Melikian, Simon, Cyrek, Joseph
• Granted Patent: US 8,923,602 B2
• Time in Patent Office: Days
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/37561 Move camera until image corresponds to stored image of same workpiece
• CPC Class Codes: G05B 2219/39397 Map image error directly to robot movement, position with relation to world, base not needed, image based visual servoing
• CPC Class Codes: G05B 2219/40003 Move end effector so that image center is shifted to desired position
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 2201/11 Technique with transformation invariance effect

Abstract

Disclosed herein are embodiments and methods of a visual guidance and recognition system requiring no calibration. One embodiment of the system comprises a servo actuated manipulator configured to perform a function, a camera mounted on the face plate of the manipulator, and a recognition controller configured to acquire a two dimensional image of the work piece. The manipulator controller is configured to receive and store the face plate position at a distance “A” between the reference work piece and the manipulator along an axis of the reference work piece when the reference work piece is in the camera'"'"'s region of interest. The recognition controller is configured to learn the work piece from the image and the distance “A”. During operation, a work piece is recognized with the system, and the manipulator is accurately positioned with respect to the work piece so that the manipulator can accurately perform its function.

Claims

1. A visual guidance and recognition system for recognizing work pieces and guiding a servo actuated manipulator without a calibration target, the system comprising:

a servo actuated manipulator configured to perform a function;

a camera mounted on or in fixed relation with a face plate of the manipulator;

a recognition system configured to acquire a two dimensional image via the camera of at least a portion of at least one reference work piece; and

a manipulator controller configured to receive and store a position of the face plate at a distance “

A”

between the reference work piece and the manipulator along a coordinate axis of the reference work piece when the reference work piece image is acquired;

wherein the recognition system is further configured to learn the reference work piece based on the two dimensional image and the distance “

A”

associated with the reference work piece.

2. The visual guidance and recognition system of claim 1, wherein the recognition system is further configured to center the portion of the reference work piece in a region of interest by moving the servo actuated manipulator along a coordinate axis parallel to the reference work piece prior to acquiring the two dimensional image.

3. The visual guidance and recognition system of claim 1, wherein, when used during operation, a production work piece is introduced to the servo actuated manipulator, and

wherein the recognition controller is further configured to acquire a two dimensional image via the camera of at least a portion of the production work piece and signal the manipulator controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.

4. The visual guidance and recognition system of claim 3, wherein the recognition controller is further configured to repeat calculation of the dimensional data offsets until a threshold offset limit is reached.

5. The visual guidance and recognition system of claim 3, wherein the manipulator is configured to move dx, dy, dz, rx, ry, rz around a center of rotation located either on the face plate or along the coordinate axis.

6. The visual guidance and recognition system of claim 3, wherein when the recognition controller signals the manipulator controller to move the manipulator, the manipulator moves in two steps, the first step by an amount of dx, dy, dz, and the second step by an amount of rx, ry, rz based on the dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.

7. The visual guidance and recognition system of claim 1, wherein the recognition controller is further configured to learn and store in a database a large number of different reference work pieces each with an associated two dimensional image and a distance “

A”

.

8. The visual guidance and recognition system of claim 7, wherein, when a production work piece is introduced to the servo actuated manipulator, the recognition controller is further configured to acquire a two dimensional image via the camera of at least a portion of the production work piece and to recognize the production work piece as one of the large number of different reference work pieces in the database.

9. The visual guidance and recognition system of claim 8, wherein, if the production work piece is not recognized, the manipulator controller moves the manipulator along the coordinate axis until an the distance “

A”

is such that recognition controller recognizes the production work piece.

10. The visual guidance and recognition system of claim 8, wherein, once the production work piece is recognized, the recognition controller is further configured to use the existing two dimensional image of the production work piece and signal the manipulator controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.

11. The visual guidance and recognition system of claim 10, wherein the recognition controller is further configured to repeat calculation of the dimensional data offsets until a threshold offset limit is reached.

12. A visual guidance and recognition system for recognizing work pieces and guiding a servo actuated manipulator with out a calibration target comprising;

a recognition controller comprising a database of at least one reference work piece;

a servo actuated manipulator configured to perform a function;

a camera mounted on or in fixed relation with a face plate of the manipulator; and

a manipulator controller configured to control movement of the servo actuated manipulator, wherein the recognition system is configured to acquire a two dimensional image via the camera of at least a portion of a production work piece, match the image of the production work piece with the reference work piece in the database, calculate a dimensional data offset between the image of the production work piece and an image associated with the reference work piece, and signal the manipulator controller to move the manipulator to decrease the offset until the offset is below a threshold limit.

13. A method for visual guidance and recognition of a work piece without the need for calibration, using a camera and a servo actuated manipulator, comprising:

positioning a reference work piece;

storing a manipulator work position at the reference work piece in a manipulator controller;

moving the manipulator along a coordinate axis of the reference work piece to a distance “

A”

wherein at least a portion of the reference work piece is in a region of interest of the camera;

storing the position of a face plate of the manipulator at the distance “

A”

in the manipulator controller;

acquiring a two dimensional image with a recognition controller via the camera of at least a portion of at least one reference work piece; and

learning the reference work piece with the recognition controller based on the two dimensional image and the distance “

A”

associated with the reference work piece.

14. The method of claim 13 further comprising:

centering the portion of the reference work piece in the region of interest by moving the servo actuated manipulator along a coordinate axis parallel to the reference work piece prior to acquiring the two dimensional image.

15. The method of claim 13 further comprising:

introducing a production work piece;

acquiring a two dimensional image with the recognition controller via the camera of at least a portion of the production work piece;

calculating with the recognition controller dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece; and

signaling the manipulator controller with the recognition controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on the dimensional data offsets.

16. The method of claim 15, further comprising repeating calculating the dimensional data offsets and signaling the manipulator controller until a threshold offset limit is reached.

17. The method of claim 13 further comprising:

repeating the method for each of a plurality of different reference work pieces; and

creating a database with in the recognition controller of each two dimensional image and distance “

A”

associated with each of the plurality of difference reference work pieces.

18. The method of claim 17 further comprising:

introducing a production work piece;

acquiring a two dimensional image with the recognition controller via the camera of at least a portion of the production work piece;

comparing the two dimensional image of the production work piece with the database of reference work pieces;

recognizing the production work piece from a reference work piece in the database;

calculating with the recognition controller dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece; and

signaling the manipulator controller with the recognition controller to move the manipulator by an amount of dx, dy, dz, rx, ry, rz based on the dimensional data offsets.

19. The method of claim 18, wherein if the production work piece is not recognized, moving the manipulator with the manipulator controller along the coordinate axis to change the distance “

A”

such that a reference work piece from the database is recognized.

20. The method of claim 18, further comprising repeating calculating the dimensional data offsets and signaling the manipulator controller until a threshold offset limit is reached.

21. The method of claim 18, wherein when the recognition controller signals the manipulator controller to move the manipulator, the manipulator moves in two steps, the first step by an amount of dx, dy, dz, and the second step by an amount of rx, ry, rz based on dimensional data offsets between the two dimensional images of the recognized reference work piece and the production work piece.