Camera-based monitoring of machines with mobile machine elements for collision prevention

• US 9,122,266 B2
• Filed: 01/30/2009
• Issued: 09/01/2015
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: PILZ GmbH & Company KG
• Original Assignee: PILZ GmbH & Company KG
• Inventors: Franke, Andre, Krueger, Lars, Woehler, Christian
• Primary Examiner(s): Nguyen, John Q
• Assistant Examiner(s): KIM, KYUNG J
• Application Number:
  US12/362,733
• Publication Number:
  US 20090222134A1
• Time in Patent Office: 2,405 Days
• Field of Search: 700/251, 700/255, 901/2, 901/46, 901/49
• US Class Current: 1/1
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/39098 Estimate stop, brake distance in predef time, then verify if in safe distance
• CPC Class Codes: G05B 2219/49137 Store working envelop, limit, allowed zone

Abstract

In a method for collision-free interaction between a machine having mobile machine elements and objects in its vicinity, safety regions are established and monitored using the knowledge of the current position and the dynamic behavior. In particular, image data of the mobile machine element are recorded by means of an image acquisition system and are correlated with a database compiled in a training phase with image data of at least one mobile machine element. The database contains image data relating to a plurality of movement phases within the movement process of the machine. Using the database, both the current position of the mobile machine element is ascertained and a future position, reachable within the stopping time, is estimated. The dynamic behavior of the machine in the event of collision risk is optimized on the basis of this estimation.

Claims

1. A method for failsafely controlling an automated machine having a machine element moveable through a plurality of predefined movement phases, the machine having a drive for driving the mobile machine element in accordance with said predefined movement phases and at least one object sensor for detecting an object in the vicinity of the mobile machine element, the method comprising the steps of:

providing an image acquisition system for acquiring image data of the mobile machine element;

moving the mobile machine element through the predefined movement phases,

providing a database containing reference image data acquired by the image acquisition system corresponding to the plurality of predefined movement phases;

determining a stopping time corresponding to the time period required from a stop command to bring the mobile machine element to rest;

thereafter monitoring the operation of the automated machine via the image acquisition system by acquiring instantaneous image data of the mobile machine element;

correlating the instantaneous image data and the reference image data in order to determine a current position of the mobile machine element, wherein the current position is determined as a position allocated to one of said plurality of predefined movement phases whose reference image data is most similar to the instantaneous image data;

estimating a future position of the mobile machine element relative to the current position of the mobile machine element using the reference image data in the database, wherein the future position is a maximum reachable position from the current position within the stopping time;

establishing a safety region around the current position of the mobile machine element, wherein a dimension of the safety region is determined as a function of the estimated future position;

detecting an entry of the object into the safety region by means of the at least one object sensor;

generating a stop command when an object entry into the safety region is detected; and

controlling said drive in response to said stop command.

2. The method according to claim 1, wherein the image acquisition system comprises a multi-ocular camera system for acquiring the image data.

3. The method according to claim 1, wherein the current position is determined by identifying reference image data which is most similar to the instantaneous image data, and by determining a movement phase allocated to the identified reference image data.

4. The method according to claim 1, wherein instantaneous image data are subjected to a transformation in order to obtain transformed instantaneous image data, and wherein the current position is determined as a function of the transformed instantaneous image data.

5. The method according to claim 1, wherein the future position of the mobile machine element is estimated by identifying a path of movement along the movement phases.

6. The method according to claim 1, wherein the dimension of the safety region is established by determining a plurality of future positions on the basis of the database, and by expanding the plurality of future positions by a predetermined distance.

7. The method of claim 6, wherein the dimension of the safety region is determined as a function of a distance which the object can travel within the stopping time.

8. The method according to claim 7, wherein the future positions are expanded on all sides in order to establish the safety region.

9. The method according to claim 1, wherein the at least one object sensor generates 3D sensor data including movement information of the object.

10. The method of claim 1, wherein the at least one object sensor comprises the image acquisition system used to monitor the operation of the automated machine.

11. A device for failsafely controlling an automated machine having a drive for driving at least one machine element moveable through a plurality of predefined movement phases, the device comprising:

an image acquisition system for monitoring the operation of the automated machine and acquiring image data of the mobile machine element;

a database containing reference image data acquired by the image acquisition system corresponding to the plurality of predetermined movement phases of the mobile machine element;

a first unit for determining a current position of the mobile machine element by correlating instantaneous image data acquired by the image acquisition system of the mobile machine element with the reference image data from the database, wherein the first unit determines the current position of the mobile machine element as a position allocated to a movement phase whose reference image data is most similar to the instantaneous image data;

a second unit for estimating a future position of the mobile machine element relative to the current position, wherein the future position is the maximum reachable position from the current position within a predetermined stopping time defined as the time period required from a stop command to bring the mobile machine element to rest;

at least one object sensor for detecting an object in the vicinity of the mobile machine element;

a third unit for establishing a safety region around the current position of the mobile machine element as a function of the future position; and

a controller for generating a stop command when said at least one object sensor detects an entry of the object into the safety region and controlling said drive in response to said stop command.

12. The device of claim 11, wherein the at least one object sensor comprises the image acquisition system used to monitor the operation of the automated machine.