CAMERA-BASED MONITORING OF MACHINES WITH MOBILE MACHINE ELEMENTS FOR COLLISION PREVENTION

• US 20090222134A1
• Filed: 01/30/2009
• Published: 09/03/2009
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: PILZ GmbH & Company KG
• Original Assignee: PILZ GmbH & Company KG
• Inventors: Franke, Andre, Krueger, Lars, Woehler, Christian
• Granted Patent: US 9,122,266 B2
• Time in Patent Office: Days
• US Class Current: 700/251
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/39098 Estimate stop, brake distance in predef time, then verify if in safe distance
• CPC Class Codes: G05B 2219/49137 Store working envelop, limit, allowed zone

Abstract

In modern industrial robots which move at considerable speeds, collisions generally cause serious damage to the robots and the collision objects. This can lead to injuries and expensive production stoppages. In a method for collision-free interaction between a machine having mobile machine elements and objects in its vicinity, safety regions are established and monitored using the knowledge of the current position and the dynamic behavior. In particular, image data of the mobile machine element are recorded by means of an image acquisition system and are correlated with a database, which has been compiled in a training phase with image data of at least one mobile machine element. The database contains image data relating to a plurality of movement phases within the movement process of the machine. In the scope of correlating the currently recorded image data with this database, both the current position of the mobile machine element is ascertained and a future position, reachable within the stopping time, is estimated. The dynamic behavior of the machine in the event of collision risk is optimized on the basis of this estimation.

Claims

1. A method for collision-free interaction between a machine having a machine element moveable through a plurality of movement phases in accordance with a dynamic behavior and an object in the vicinity of the mobile machine element, the machine having a drive for driving the mobile machine element and at least one object sensor for detecting the object, the method comprising the steps of

providing an image acquisition system for recording image data of the machine element,

providing a database containing reference image data relating to the plurality of movement phases,

moving the machine element through the movement phases,

providing a stopping time defining a time period required to bring the moving machine element to rest,

recording instantaneous image data of the machine element by means of the image acquisition system,

correlating the instantaneous image data and the reference image data in order to determine a current position of the moving machine element and in order to estimate a future position of the machine element, wherein the future position is a maximum reachable position within the stopping time,

establishing a safety region around the machine element as a function of the estimated future position,

detecting any entry of the object into the safety region by means of the at least one object sensor, and

controlling the drive in order to modify the dynamic behavior of the machine element whenever an object entry into the safety region is detected.

2. The method according to claim 1, wherein the image acquisition system comprises a multi-ocular camera system for recording the image data.

3. The method according to claim 1, wherein the current position is determined by identifying reference image data which is most similar to the instantaneous image data, and by determining a movement phase allocated to the identified reference image data.

4. The method according to claim 1, wherein instantaneous image data are subjected to a transformation in order to obtain transformed instantaneous image data, and wherein the current position is determined as a function of the transformed instantaneous image data.

5. The method according to claim 1, wherein the future position of the machine element is estimated by identifying a path of movement along the movement phases.

6. The method according to claim 1, wherein the safety region is established by determining a plurality of future positions on the basis of the database, and by expanding the plurality of future positions by a safety region.

7. The method of claim 6, wherein the safety region is determined as a function of a distance which the object can travel within the stopping time.

8. The method according to claim 7, wherein the future positions are expanded on all sides in order to establish the safety region.

9. The method according to claim 1, wherein the at least one object sensor comprises a camera sensor.

10. The method according to claim 1, wherein the at least one object sensor generates 3D sensor data including movement information of the object.

11. The method according to claim 1, wherein controlling the drive comprises stopping the machine element.

12. The method according to claim 1, wherein controlling the drive comprises a speed change of the machine element.

13. The method according to claim 12, wherein the speed change is determined as a function of an estimated collision probability defining a probability for a collision between the object and the machine element, with the collision probability being estimated after detection of entry of the object into the safety region.

14. The method according to claim 13, wherein a space passed through by the mobile machine element is subjected to quantization, an individual collision probability being determined for each quantization element, and the actual collision probability being derived from the sum of the individual collision probabilities over all quantization elements.

15. A device for the collision-free interaction between a machine having at least one machine element moveable through a plurality of movement phases and an object in the vicinity of the mobile machine element, the device comprising a first unit for determining a current position of the mobile machine element and for estimating a future position of the machine element, which future position is reachable by the machine element within a stopping time, comprising a second unit for determining at least one safety region as a function of the future position, comprising at least one object sensor for detecting an entry of the object into the safety region, and comprising a third unit for controlling the machine element when entry of the object into the safety region is detected in order to modify a dynamic movement behavior of the machine element, wherein the first unit comprises an image acquisition system for recording image data of the machine element, a database containing reference image data relating to the plurality of movement phases, and a correlator for correlating the instantaneously recorded image data with the reference image data from the database, the reference image data corresponding to the plurality of movement phases, and the correlator being designed to estimate both the current position and the future position of the machine element.