Robot vision using target holes, corners and other object features

• US 5,956,417 A
• Filed: 02/28/1994
• Issued: 09/21/1999
• Est. Priority Date: 02/16/1982
• Status: Expired due to Term

• Current Assignee: LMI Technologies Inc.
• Original Assignee: Sensor Adaptive Machines, Inc.
• Inventors: Pryor, Timothy R.
• Primary Examiner(s): Kelley, Christopher S.
• Application Number:
  US08/203,603
• Time in Patent Office: 2,031 Days
• Field of Search: 382/8, 382/1, 382/95, 382/169, 382/103, 382/106, 382/153, 382/154, 382/286, 382/287, 382/288, 382/291, 348/86, 348/94, 348/95, 348/169, 348/42, 348/47, 348/119, 356/3, 356/12, 356/14, 395/119-125
• US Class Current: 382/154
• CPC Class Codes: A01B 69/008 automatic
• CPC Class Codes: B23P 19/10 Aligning parts to be fitted together
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: F02F 2007/0063 Head bolts; Arrangements of cylinder head bolts
• CPC Class Codes: G01B 11/00 Measuring arrangements characterised by the use of optical techniques
• CPC Class Codes: G05B 19/401 characterised by control arrangements for measuring, e.g. calibration and initialisation, measuring workpiece for machining purposes G05B19/19 takes precedence
• CPC Class Codes: G05B 2219/34033 Control processor and signal processor cooperate
• CPC Class Codes: G05B 2219/39393 Camera detects projected image, compare with reference image, position end effector
• CPC Class Codes: G05B 2219/45064 Assembly robot

Abstract

A method and apparatus uses naturally occurring object features as targets in a machine vision based robot (or other) guidance system. In one embodiment, a special high speed processor finds at least 3 pre-selected feature points on an object, which when considered with the data base for the object, allows a rapid 6 degree of freedom solution for the object location relative to the camera and thence connected automation. While primarily envisioned for industrial robots, the invention is useful for all sorts of machines including vehicles, construction machinery and the like--anywhere the target object has a known data base relative to the features in question.

Claims

1. An apparatus for locating an object feature comprising:

a data base of an object including locations in three dimensions of at least three preselected features of the object relative to one another and relative to other features of the object, at least one of said preselected features being a natural feature of the object;

a storing means for storing said data base;

an electro-optical sensor means comprising multiple cameras for obtaining and scanning respective images of the object;

a location determining means for determining, from the scanned images, the location of said at least three preselected features in three-dimensional space;

a determining means for determining, from the data base and from the determined three dimensional locations of the at least three preselected features, a location of at least one of said other features of the object.

3. An apparatus as claimed in claim 1 and further including a moving means for moving the object with the robot to position the portion of the object at a desired location.

4. An apparatus as claimed in claim 1 wherein said location determining means includes a photogrammetric analysis means for photogrammetrically solving for the location of the preselected features identified in three dimensional space.

5. An apparatus as claimed in claim 1 wherein said electro-optical sensor means includes a TV camera.

6. An apparatus as claimed in claim 1 wherein said electro-optical sensor means includes a stereo pair of TV cameras which view the object from angularly displaced fields of view, and wherein said data analysis means determines the location of the preselected natural target feature seen by each TV camera and checks that the location determined for the preselected natural target feature from each TV camera is the same.

7. An apparatus as claimed in claim 1 wherein said location determining means is a front end processor which includes a selection means for selecting an additional preselected feature to be optionally identified and a sensing means for sensing if one of the three preselected features is unidentifiable and for causing said selection means to select the additional preselected feature to be identified in place thereof.

8. An apparatus as claimed in claim 7 wherein said electro-optical sensor means includes a pair of TV cameras each having a fixed viewing axis for viewing different sides of the object such that said location determining means can locate preselected natural features in three-dimensional space on either side as desired.

9. An apparatus as claimed in claim 1 wherein said location determining means identifies preselected features by size.

10. An apparatus as claimed in claim 1 wherein said location determining means identifies preselected features by shape.

11. An apparatus as claimed in claim 1 wherein said location determining means includes a search means for locating a first one of the preselected features and for then searching for the other preselected features in predetermined locations relative to the located position of the first preselected feature.

12. An apparatus as claimed in claim 1 wherein the object is moving;

and wherein said location determining means includes a tracking means for tracking the moving preselected features.

13. An apparatus as claimed in claim 1 wherein the preselected natural features are three dimensional.

14. An apparatus as claimed in claim 13 wherein the at least one of the preselected natural features is selected from one of holes, corners, edge segments, notches, protrusions and indentations found on the object which are functionally related to the purpose of the object and not provided on the object merely as a two dimensional target feature.

26. An apparatus for locating an object feature as claimed in claim 1 and further including a robot which is operatively related to said object and which is moved based on the determined location of said at least one other feature.

28. An apparatus for locating an object feature as claimed in claim 1 wherein said data base is obtained from a CAD data base of the object.

2. A method for locating an object feature comprising the steps of:

creating a data base of an object including locations in three dimensions of at least three preselected features of the object relative to one another and relative to other features of the object, at least one of the preselected features being a natural feature of the object;

storing of the data base in a storage device;

obtaining and scanning respective images of the object with an electro-optical sensor device comprising multiple cameras;

location determining, from the scanned images, the location of said at least three preselected features in three-dimensional space with a location determining device;

determining from the data base from the determined three dimensional locations of the at least three preselected features with a determining device a location of at least one of the other features of the object.

15. A method as claimed in claim 2 and further including the step of moving the object with the robot to position the portion of the object at a desired three dimensional location.

16. A method as claimed in claim 2 wherein said location determining step includes the step of photogrammetrically solving for the location of the preselected features in three-dimensional space with a photogrammetric analysis device.

17. A method as claimed in claim 2 wherein said location determining step includes the steps of viewing the object from angularly displaced fields of view with a stereo pair of TV cameras, determining the location in three-dimensional space of the preselected natural feature seen by each TV camera and checking that the location determined for the preselected natural feature from each TV camera is the same.

18. A method as claimed in claim 2 wherein said location determining step includes the steps of selecting with a front end processor an additional preselected feature to be optionally identified, sensing if one of the three preselected features is unidentifiable during the location determining step, and then selecting the additional preselected feature to be located in place thereof.

19. A method as claimed in claim 18 wherein said location determining step includes the step of viewing different sides of the object with a pair of TV cameras each having a fixed viewing axis such that during said location determining step preselected natural features on either side are determined as desired.

20. A method as claimed in claim 2 wherein said location determining step locates preselected features by size.

21. A method as claimed in claim 2 wherein said location determining step locates preselected features by shape.

22. A method as claimed in claim 2 wherein said location determining step includes the steps of locating a first one of the preselected features with a search device and then searching for the other preselected features in predetermined locations relative to the located position of the first preselected feature.

23. A method as claimed in claim 2 wherein the object is moving;

and wherein said location determining step includes the step of tracking the moving preselected features with a tracking device.

24. A method as claimed in claim 2 wherein the preselected natural features are three dimensional.

25. A method as claimed in claim 24 wherein the three-dimensional natural feature is selected from one of holes, corners, edge segments, notches, protrusions and indentations found on the object which are functionally related to the purpose of the object and not provided on the object merely as a two dimensional target feature.

27. A method for locating an object feature as claimed in claim 2 and further including the step of moving a robot which is operatively related to the object based on the determined location of said at least one other feature.

29. A method for locating an object feature as claimed in claim 2 and further including the step of downloading data concerning said object from a CAD system.