Method for observation of a person in an industrial environment

• US 8,154,590 B2
• Filed: 01/30/2009
• Issued: 04/10/2012
• Est. Priority Date: 08/02/2006
• Status: Active Grant

• Current Assignee: PILZ GmbH & Company KG
• Original Assignee: PILZ GmbH & Company KG
• Inventors: Kressel, Ulrich, Krueger, Lars, Progscha, Werner, Woehler, Christian, Kummert, Franz, Schmidt, Joachim, Ott, Rainer, Sagerer, Gerhard
• Primary Examiner(s): BHATIA, AJAY M
• Application Number:
  US12/362,745
• Publication Number:
  US 20090237499A1
• Time in Patent Office: 1,166 Days
• Field of Search: 348/77
• US Class Current: 348/77
• CPC Class Codes: F16P 3/142 using image capturing devices
• CPC Class Codes: G05B 2219/40202 Human robot coexistence
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A method for observing a person in an industrial environment using a multicamera system to acquire image data about the position and alignment of a person, including a person'"'"'s body parts. These image data are then examined with regard to the imaging of a person so that whenever a person has been detected in the image data, an articulated virtual 3D model of the human body is matched to this person hypothesis. Subsequently, this virtual body model is continuously matched to the movement behavior of the person detected in the image data. A hazard potential is determined using knowledge of the position and the movement behavior of the virtual body model in space. The hazard potential thus determined is subjected to a threshold value comparison in order to act upon the movement control of the machine or the machine part in the event of this threshold value being exceeded.

Claims

1. A method for observation of a person in an industrial environment comprising a moveable machine element, the method comprising the steps of

acquiring image data of the person by means of a multicamera system,

analyzing the image data in order to produce a person hypothesis representing the person,

providing an articulated virtual 3D model of a human body,

matching the articulated virtual 3D model of the human body to the person hypothesis in order to generate a movement behavior representation,

determining an instantaneous position of the machine element,

determining a hazard potential depending on the position of the machine element and the movement behavior representation, and

controlling the machine element as a function of the hazard potential.

2. The method of claim 1, wherein the machine element is slowed down or shut down whenever the hazard potential exceeds a predefined threshold.

9. The method of claim 1, wherein the articulated virtual 3D model is created on the basis of 3D point data including information relating to spatial coordinates.

11. The method of claim 9, wherein two individual clusters are combined to form a common cluster representing the person, whenever the two individual clusters do not overlap and the resulting common cluster falls within the person-specific limiting values.

12. The method of claim 9, wherein the industrial environment comprises a floor, and a projection of the cluster onto the floor is determined and tracked in order to continuously match the articulated virtual 3D model the person during movement.

13. The method of claim 12, wherein the projection is tracked by means of linear prediction and Kalman filtering.

14. The method of claim 1, wherein a probability of match is determined, the probability of match representing a probability with which a given posture of the articulated, virtual 3D model represent the current shape of the person.

3. A method for observation of a person in an industrial environment comprising a moveable machine element, the method comprising the steps of

acquiring image data of the person by means of a multicamera system,

analyzing the image data in order to produce a person hypothesis representing the person,

providing an articulated virtual 3D model of a human body,

matching the articulated virtual 3D model to the person hypothesis,

providing a database containing a plurality of reference data representing a reference movement cycle of the person, the reference data having been determined from shape and position of the articulated virtual 3D model during a plurality of reference movement phases,

generating current data representing a current movement of the person as a function of a current shape and position of the articulated virtual 3D model,

correlating the current data with the reference data from the database,

wherein a current movement phase is detected whenever the current data exhibits a predefined degree of similarity to the reference data,

wherein a movement cycle is assessed as having been completely carried out by the person whenever a specific sequence of current movement phases has been detected, and

wherein a signal is produced whenever an incomplete movement cycle is determined.

4. The method of claim 3, wherein the signal comprises in indication of a missing part in the incomplete movement cycle.

5. The method of claim 4, further comprising a statistical acquisition of missing parts.

6. The method of claim 3, wherein the reference data are generated by means of a plurality of training cycles.

7. The method of claim 3, wherein the reference data are generated by means of simulation.

8. The method of claim 3, wherein the reference data are subjected to a transformation in order to produce transformed reference data, with the step of correlating being performed on the basis of the transformed reference data.

10. The method of claim 8, wherein the 3D point data are combined in a cluster as a function of predefined person-specific limiting values.