Method and system for the high-precision positioning of at least one object in a final location in space

• US 8,346,392 B2
• Filed: 12/17/2008
• Issued: 01/01/2013
• Est. Priority Date: 12/27/2007
• Status: Active Grant

• Current Assignee: Leica Geosystems AG (Hexagon AB)
• Original Assignee: Leica Geosystems AG (Hexagon AB)
• Inventors: Walser, Bernd, Metzler, Bernhard, Aebischer, Beat, Siercks, Knut, Pettersson, Bo
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): LANG, MICHAEL DEAN
• Application Number:
  US12/810,797
• Publication Number:
  US 20100274390A1
• Time in Patent Office: 1,476 Days
• Field of Search: 700/259
• US Class Current: 700/259
• CPC Class Codes: B25J 9/1697 Vision controlled systems

Abstract

The invention relates to a method and a system for the high-precision positioning of at least one object in a final location in space. An object (12) is gripped and held by the industrial robot (11) within a gripping tolerance. A compensating variable, which corrects the gripping tolerance, is determined for the industrial robot (11). The object (12) is adjusted with high precision into a final location by the following steps, which repeat until reaching the final location at a predetermined tolerance: recording of image recordings by recording units (1_a, 1_b); determining the current location of the object (12) in the spatial coordinate system from the positions (P_a, P_b) of the recording units (1_a, 1_b), the angular orientations of cameras (2_a, 2_b) of the recording units (1_a, 1_b) which are detected by angle measuring units (4_a, 4_b), the image recordings, and the knowledge of features (13) on the object (12); calculating the location difference between the current location of the object (12) and the final location; calculating a new target position of the industrial robot (11) in consideration of the compensating variable from the current position of the industrial robot (11) and a variable which is linked to the location difference; adjusting the industrial robot (11) into the new target position.

Claims

1. A method for the high-precision positioning of at least one object in a final location in space by means of an industrial robot, with a first industrial robot which can be adjusted into predefinable positions and a first optical recording means which is calibrated in a three-dimensional space coordinate system and positioned in a known first position having a known orientation, with an optically calibrated first camera for recording images within a determined first field of vision, a first drive unit for orienting the first camera causing an adjustment of the first field of vision and a first angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the first camera, so that the first field of vision can be determined in the space coordinate system, at least one second optical recording means which is calibrated in the three-dimensional space coordinate system and positioned in a known second position having a known orientation, with an optically calibrated second camera for recording images within a determined second field of vision, a second drive unit for orienting the second camera causing an adjustment of the second field of vision and a second angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the second camera, so that the second field of vision can be determined in the space coordinate system the at least two positions being set apart in such a way as to allow a three-dimensional image recording of the at least one object by means of the at least two recording means through at least partly overlapping fields of vision, the method comprising:

the first industrial robot gripping and holding a first object within a gripping tolerance, wherein the first object has a known optically detectable first feature;

determining a first compensating variable of a type which corrects the gripping tolerance, for the first industrial robot, so that the first object is adjustable in a compensated manner in the space coordinate system by predefining a position of the first industrial robot, the first compensating variable being determined by the steps of

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the first features of the first object which is held in a first compensating position of the first industrial robot;

recording first image recordings;

determining the location of the first object in the space coordinate system in the first compensating position of the first industrial robot from the positions of the recording means, the angular orientations of the cameras, the angular orientations detected by the angle measuring units, the first image recordings, and the knowledge of the first features on the first object; and

determining the first compensating variable by adducing the first compensating position of the first industrial robot and at least the determined location of the first object in the first compensating position of the first industrial robot; and

adjusting the first object with high precision into a first final location by the steps, which are repeated until the first final location is reached at a predefined tolerance, of;

recording further first image recordings;

determining the current location of the first object in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations detected by the angle measuring units, the further first image recordings, and the knowledge of the first features on the first object;

calculating the location difference between the current location of the first object and the first final location;

calculating a new setpoint position of the first industrial robot in consideration of the first compensating variable from the current position of the first industrial robot and a variable linked to the location difference; and

adjusting the first industrial robot into the new setpoint position.

2. A method according to claim 1, wherein, after the gripping of the first object, the first industrial robot is adjusted into the first compensating position of the first industrial robot for determining the first compensating variable.

3. A method according to claim 1, wherein, before the steps which are repeated until the first final location is reached at a predefined tolerance:

the first industrial robot is adjusted, in consideration of the first compensating variable, from the first compensating position into a position in which the first object is positioned at a first approach location close to the first final location; and

the two cameras are oriented, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the first features of the first object which is positioned in the first approach location.

4. A method according to claim 1, wherein, before the high-precision adjustment of the first object into the first final location, a second object is gripped by a second industrial robot or by hand and placed into an object holder in a second final location in the space coordinate system.

5. A method according to claim 4, wherein the object holder is embodied in a positively centring manner in such a way that the second object is placed with high precision in the predefined second final location.

6. A method according to one of claims 5, wherein the first final location for the first object is calculated from the second final location of the second object and a predefined relative location between the first object and the second object.

7. A method according to claim 4, wherein the second object has known optically detectable second features and, after the placing of the second object in the object holder, the second final location of the second object in the space coordinate system is determined by the steps:

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the second features of the second object;

recording second image recordings; and

determining the second final location of the second object in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the second image recordings, and the knowledge of the second features on the second object.

8. A method according to claim 1, wherein, before the high-precision adjustment of the first object into the first final location:

a second object, which has known optically detectable second features, is gripped and is held by a second industrial robot within a gripping tolerance, the second industrial robot is adjusted into a final position of the second industrial robot in which the second object is in a second final location, and

the second final location of the second object in the space coordinate system is determined by the steps of;

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the second features of the second object;

recording second image recordings; and

determining the second final location of the second object in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the second image recordings, and the knowledge of the second features on the second object.

9. A method according to claim 1, wherein, before the high-precision adjustment of the first object into the first final location:

a second object, which has known optically detectable second features, is gripped and is held by a second industrial robot within a gripping tolerance;

a second compensating variable of this type, which corrects the gripping tolerance, is determined for the second industrial robot, so that the second object is adjustable in a compensated manner in the space coordinate system by predefining a position of the second industrial robot, the second compensating variable being determined by the steps;

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the second features of the second object which is held in a second compensating position of the second industrial robot;

recording second image recordings;

determining the location of the second object in the space coordinate system in the second compensating position of the second industrial robot from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the second image recordings and the knowledge of the second features on the second object, and

determining the second compensating variable by adducing the second compensating position of the second industrial robot and at least the determined location of the second object in the second compensating position of the second industrial robot, and

the second object is adjusted with high precision into a second final location by the steps, which are repeated until the second final location is reached at a predefined tolerance, of;

recording further second image recordings;

determining the current location of the second object in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the further second image recordings and the knowledge of the second features on the second object;

calculating the location difference between the current location of the second object and the second final location; and

calculating a new setpoint position of the second industrial robot in consideration of the second compensating variable from the current position of the second industrial robot and a variable linked to the location difference, and

adjusting the second industrial robot into the new setpoint position.

10. A method according to claim 9, wherein, after the gripping of the second object, the second industrial robot is adjusted into the second compensating position of the second industrial robot for determining the second compensating variable.

11. A method according to claim 9, wherein, before the steps which are repeated until the second final location is reached at a predefined tolerance:

the second industrial robot is adjusted, in consideration of the second compensating variable, from the second compensating position into a position in which the second object is positioned at a second approach location close to the second final location, and

the two cameras are oriented, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the second features of the second object which is positioned in the second approach location.

12. A method according to claim 1, wherein:

a third object, which is embodied as a machining tool, is held by a third industrial robot within a holding tolerance;

the machining tool or a part coupled thereto of the third industrial robot has known optically detectable third features; and

a third compensating variable of a type, which corrects the holding tolerance, is determined for the third industrial robot, so that machining tool is adjustable in a compensated manner in the space coordinate system by predefining a position of the third industrial robot, the third compensating variable being determined by the steps of;

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the third features, the machining tool being held in a third compensating position of the third industrial robot;

recording third image recordings;

determining the location of the machining tool in the space coordinate system in the third compensating position of the third industrial robot from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the third image recordings and the knowledge of the third features; and

determining the third compensating variable by adducing the third compensating position of the third industrial robot and at least the determined location of the machining tool in the third compensating position of the third industrial robot.

13. A method according to claim 12, wherein:

the machining tool is adjusted with high precision into a third final location by the steps, which are repeated until the third final location is reached at a predefined tolerance, of;

recording further third image recordings;

determining the current location of the machining tool in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the further third image recordings and the knowledge of the third features;

calculating the location difference between the current location of the third object and the third final location;

calculating a new setpoint position of the third industrial robot in consideration of the third compensating variable from the current position of the third industrial robot and a variable linked to the location difference; and

adjusting the third industrial robot into the new setpoint position.

14. A method according to claim 1, wherein the variable linked to the location difference is formed by the location difference multiplied by a factor of less than or equal to 1.

15. A method according to claim 1, wherein:

the features are substantially known from a model which can be processed by an electronic data processing means;

by means of image processing carried out on an electronic data processing means, the features in the model and/or in the image recordings are identified and the features from the model and also the features from the image recordings are associated with one another; and

the location of the object in the space coordinate system is determined from the detected position of the recorded features in the space coordinate system and the associated features.

16. A method according to claim 15, wherein the location of the recorded features relative to one another is determined and compared with the location of the features substantially known from the model relative to one another.

17. A method according to claim 16, wherein an error message is output on exceeding of a divergence between the relative location of the recorded features and the relative location of the features substantially known from the model.

18. A method according to claim 17, wherein the object is exchanged for a new object in the case of the error message.

19. A method according to claim 16, wherein the model is adapted to the detected object.

20. A method according to claim 19, wherein the adapted model determines the final location in the space coordinate system.

21. A method according to claim 1, wherein the at least two recording means are each embodied in such a way that in each case:

the camera is orientable by means of the drive unit about in each case a tilt axis, which is horizontal in relation to the space coordinate system, and a vertical standing axis; and

the horizontal angular orientation about the standing axis and the vertical angular orientation about the tilt axis are each detected in the space coordinate system by the angle measuring unit.

22. A method according to claim 21, wherein the horizontal tilt axis and the vertical standing axis substantially intersect.

23. A method according to claim 22, wherein the at least two recording means are each embodied in such a way that:

the projection centre of the camera is arranged in each case at the intersection of the horizontal tilt axis and the vertical standing axis;

the position of the camera is coupled to the position of the recording means in each case independently of the angular orientation; and

the location in the space coordinate system is determined from the positions of the cameras, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the image recordings, and the knowledge of the features.

24. A method according to claim 22, wherein the at least two recording means are each embodied in such a way that:

the projection centre of the camera is arranged in each case outside the intersection of the horizontal tilt axis and the vertical standing axis;

the positions of the camera are determined in each case from the predefined eccentricity of the projection centre of the camera from the intersection, the angular orientations and the positions of the recording means; and

the location in the space coordinate system is determined from the positions of the recording means, the eccentricities of the projection centres of the cameras, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the image recordings and the knowledge of the features.

25. A method according to claim 1, wherein at least the first recording means has a first emitter, which is embodied for emitting a first structured light within the first field of vision of the first camera, which is orientable in conjunction with the first camera by means of the first drive unit and the angular orientation of which can be detected with high precision by means of the first angle measuring unit which is calibrated in the space coordinate system, wherein, in this step of recording the image recordings, the first structured light is projected from the first emitter onto the object in such a way that the first structured light lies in the field of vision of the second camera of the second recording means, and wherein the location of the object in the space coordinate system is additionally determined by means of triangulation from:

the positions of the recording means;

the angular orientations of the first emitter and the second camera, the angular orientations being detected by the angle measuring units; and

the image recordings of the second camera that image the first structured light.

26. A method according to claim 25, wherein at least the second recording means has a second emitter, which is embodied for emitting a second structured light within the second field of vision of the second camera, which is orientable in conjunction with the second camera by means of the second drive unit and the angular orientation of which can be detected with high precision by means of the second angle measuring unit which is calibrated in the space coordinate system, wherein, in the step of recording the image recordings:

the second structured light is projected from the second emitter onto at least the part of the features of the object in such a way that the second structured light lies in the field of vision of the first camera of the first recording means; and

the location of the object in the space coordinate system is additionally determined by means of triangulation from the positions of the recording means, the angular orientations of the second emitter and the first camera, the angular orientations being detected by the angle measuring units, and the image recordings of the first camera that image the second structured light and the knowledge of the features on the object.

27. A method according to claim 25, wherein the structured light is embodied as a projected laser line.

28. A method according to claim 25, wherein the structured light is embodied as a laser spot which is projected so as to be fanned out in the form of a laser line.

29. A method according to claim 25, wherein the structured light is embodied as a two-dimensionally projected pattern, in particular a laser raster.

30. A method according to claim 1, with an emission means which is arranged in a known third position, has a known orientation and has a third emitter which is embodied for emitting a third structured light, which is orientable by means of a third drive unit and the angular orientation of which can be detected with high precision by means of a third angle measuring unit which is calibrated in the space coordinate system, wherein, in the step of recording the image recordings, the third structured light is projected from the third emitter onto the object in such a way that the third structured light lies in the field of vision of the first camera of the first recording means, and wherein the location of the object in the space coordinate system is additionally determined by means of triangulation from:

the position of the first recording means;

the angular orientation of the first camera, the angular orientation being detected by the first angle measuring unit;

the angular orientation of the third emitter, the angular orientation being detected by the third angle measuring unit; and

the image recordings of the first camera that image the third structured light.

31. A method according to claim 1, wherein at least the first recording means has a first laser distance measuring means which is orientable in conjunction with the first camera by means of the first drive unit and the angular orientation of which can be detected with high precision by means of the first angle measuring unit which is calibrated in the space coordinate system.

32. A method according to claim 31, wherein the position of the first recording means in the space coordinate system is determined by aiming at stationary target marks by means of the first laser distance measuring means.

33. A method according to claim 31, wherein the location of the object in the space coordinate system is additionally determined by aiming at the features on the object.

38. A system for the high-precision positioning of at least one object in a final location in space by means of an industrial robot, comprising:

a first industrial robot which can be adjusted into predefinable positions;

a first optical recording means which is calibrated in a three-dimensional space coordinate system and positioned in a known first position having a known orientation, with an optically calibrated first camera for recording images within a determined first field of vision, a first drive unit for orienting the first camera causing an adjustment of the first field of vision, and a first angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the first camera, so that the first field of vision can be determined in the space coordinate system;

at least one second optical recording means which is calibrated in the three-dimensional space coordinate system and positioned in a known second position having a known orientation, with an optically calibrated second camera for recording images within a determined second field of vision, a second drive unit for orienting the second camera causing an adjustment of the second field of vision, a second angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the second camera, so that the second field of vision can be determined in the space coordinate system, the at least two positions being set apart in such a way as to allow a three-dimensional image recording of the at least one object by means of the at least two recording means through at least partly overlapping fields of vision, and

a control device having a data processing means embodied for image processing, wherein the control device is data-connected to the first industrial robot and the at least two optical recording means in such a way that the image recordings recorded by the cameras are supplied to the control device, the angular orientations of the cameras that are detected by the angle measuring units are supplied to the control device, the drive units are activated by means of the control device for orienting the cameras and the first industrial robot is adjusted into positions which are predefined by the control device, the control device and the data processing means thereof being configured to carry out the method according to claim 1.

39. A system for the high-precision positioning of at least one object in a final location in space by means of an industrial robot, comprising:

a first industrial robot which can be adjusted into predefinable positions;

a second industrial robot which can be adjusted into predefinable positions;

a first optical recording means which is calibrated in a three-dimensional space coordinate system and positioned in a known first position having a known orientation, with an optically calibrated first camera for recording images within a determined first field of vision, a first drive unit for orienting the first camera causing an adjustment of the first field of vision and a first angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the first camera, so that the first field of vision can be determined in the space coordinate system;

at least one second optical recording means which is calibrated in the three-dimensional space coordinate system and positioned in a known second position having a known orientation, with an optically calibrated second camera for recording images within a determined second field of vision, a second drive unit for orienting the second camera causing an adjustment of the second field of vision; and

a second angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the second camera, so that the second field of vision can be determined in the space coordinate system, the at least two positions being set apart in such a way as to allow a three-dimensional image recording of the at least one object by means of the at least two recording means through at least partly overlapping fields of vision; and

a control device having a data processing means embodied for image processing, wherein the control device is data-connected to the first industrial robot, the second industrial robot and the at least two optical recording means in such a way that the image recordings recorded by the cameras are supplied to the control device, the angular orientations of the cameras that are detected by the angle measuring units are supplied to the control device, the drive units are activated by means of the control device for orienting the cameras and the first industrial robot and the second industrial robot are adjusted into positions which are predefined by the control device, wherein the control device and the data processing means thereof are configured for carrying out the method according to claim 4.

40. A system for the high-precision positioning of at least one object in a final location in space by means of an industrial robot, comprising:

a first industrial robot which can be adjusted into predefinable positions;

a second industrial robot which can be adjusted into predefinable positions;

a third industrial robot which can be adjusted into predefinable positions and holds a third object, which is embodied as a machining tool, within a holding tolerance, the machining tool or a part coupled thereto of the third industrial robot having known optically detectable third features;

a first optical recording means which is calibrated in a three-dimensional space coordinate system and positioned in a known first position having a known orientation, with an optically calibrated first camera for recording images within a determined first field of vision, a first drive unit for orienting the first camera causing an adjustment of the first field of vision; and

a first angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the first camera, so that the first field of vision can be determined in the space coordinate system,

at least one second optical recording means which is calibrated in the three-dimensional space coordinate system and positioned in a known second position having a known orientation, with an optically calibrated second camera for recording images within a determined second field of vision, a second drive unit for orienting the second camera causing an adjustment of the second field of vision and a second angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the second camera, so that the second field of vision can be determined in the space coordinate system, the at least two positions being set apart in such a way as to allow a three-dimensional image recording of the at least one object by means of the at least two recording means through at least partly overlapping fields of vision; and

a control device having a data processing means embodied for image processing, wherein the control device is data-connected to the first industrial robot, the second industrial robot, the third industrial robot and the at least two optical recording means in such a way that the image recordings recorded by the cameras are supplied to the control device, the angular orientations of the cameras that are detected by the angle measuring units are supplied to the control device, the drive units are activated by means of the control device for orienting the cameras, wherein the first industrial robot, the second industrial robot and the third industrial robot are adjusted into positions which are predefined by the control device, and the control device is configured for carrying out the method according to claim 12.

34. A system for the high-precision positioning of at least one object in a final location in space by means of an industrial robot, comprising:

a first industrial robot which can be adjusted into predefinable positions;

a first optical recording means which is calibrated in a three-dimensional space coordinate system and positioned in a known first position having a known orientation, with an optically calibrated first camera for recording images within a determined first field of vision, a first drive unit for orienting the first camera causing an adjustment of the first field of vision and a first angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the first camera, so that the first field of vision can be determined in the space coordinate system;

at least one second optical recording means which is calibrated in the three-dimensional space coordinate system and positioned in a known second position having a known orientation, with an optically calibrated second camera for recording images within a determined second field of vision, a second drive unit for orienting the second camera causing an adjustment of the second field of vision and a second angle measuring unit, which is calibrated in the space coordinate system, for the high-precision detection of the angular orientation of the second camera, so that the second field of vision can be determined in the space coordinate system, the at least two positions being set apart in such a way as to allow a three-dimensional image recording of the at least one object by means of the at least two recording means through at least partly overlapping fields of vision; and

a control device having a data processing means embodied for image processing, wherein the control device is data-connected to the first industrial robot and the at least two optical recording means in such a way that the image recordings recorded by the cameras are supplied to the control device, the angular orientations of the cameras that are detected by the angle measuring units are supplied to the control device, the drive units are activated by means of the control device for orienting the cameras and the first industrial robot is adjusted into positions which are predefined by the control device, the control device and the data processing means thereof being embodied in such a way that a first object, which has optically detectable first features known to the control device, is gripped and is held by the first industrial robot within a gripping tolerance, a first compensating variable of this type, which corrects the gripping tolerance, is determined for the first industrial robot by the control device, so that the first object is adjustable in a compensated manner in the space coordinate system by predefining a position of the first industrial robot, the first compensating variable being determined by means of the control device by the steps of;

orienting the at least two cameras, in each case by means of the drive unit with at least partly overlapping fields of vision, onto at least a part of the first features of the first object which is held in a first compensating position of the first industrial robot;

recording first image recordings;

determining the location of the first object in the space coordinate system in the first compensating position of the first industrial robot from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the first image recordings and the knowledge of the first features on the first object, and

determining the first compensating variable by adducing the first compensating position of the first industrial robot and at least the determined location of the first object in the first compensating position of the first industrial robot and

wherein the first object is adjusted with high precision by the control device into a first final location by the steps, which are repeated until the first final location is reached at a predefined tolerance, of;

recording further first image recordings;

determining the current location of the first object in the space coordinate system from the positions of the recording means, the angular orientations of the cameras, the angular orientations being detected by the angle measuring units, the further first image recordings and the knowledge of the first features on the first object;

calculating the location difference between the current location of the first object and the first final location;

calculating a new setpoint position of the first industrial robot in consideration of the first compensating variable from the current position of the first industrial robot and a variable linked to the location difference; and

adjusting the first industrial robot into the new setpoint position.

35. A system according to claim 34, wherein the at least two recording means are each embodied in such a way that:

the camera is orientable by means of the drive unit about in each case a tilt axis, which is horizontal in relation to the space coordinate system, and a vertical standing axis; and

the horizontal angular orientation about the standing axis and the vertical angular orientation about the tilt axis are each detected in the space coordinate system by the angle measuring unit.

36. A system according to claim 35, wherein the horizontal tilt axis and the vertical standing axis substantially intersect.

37. A system according to claim 36, wherein the at least two recording means are each embodied as video theodolite.