Method for the monitoring of a monitored zone

• US 7,505,620 B2
• Filed: 04/12/2005
• Issued: 03/17/2009
• Est. Priority Date: 04/14/2004
• Status: Expired due to Fees

• Current Assignee: Sick AG (Sick)
• Original Assignee: Sick AG (Sick)
• Inventors: Grabinger, Jorg, Braune, Ingolf
• Primary Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US11/103,880
• Publication Number:
  US 20050232465A1
• Time in Patent Office: 1,435 Days
• Field of Search: 382/100, 382/103, 382/106, 382/107, 382/141, 382/149, 382/152, 382/153, 382/154, 382/181, 382/190, 382/195, 348/154, 348/155, 348/143, 345/619, 345/629, 345/632, 345/633
• US Class Current: 382/152
• CPC Class Codes: F16P 3/142 using image capturing devices
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

In a method for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven movable part, video images are used which were detected in time sequence by at least two video cameras whose fields of view overlap at least partly in an overlap region in the monitored zone. On the basis of the first video images detected by a first one of the video cameras, a first recognition and/or tracking of virtual objects is carried out and, on the basis of the second video images detected by a second one of the video cameras, a second recognition and/or tracking of virtual objects is carried out, the virtual objects corresponding to real objects at least in the overlap region. For the classification and/or determination of the position and/or the movement of at least one real object in the overlap region, virtual objects corresponding to the real object from the first and second virtual object recognition and/or tracking processes are determined and data determined in each case for both virtual objects on their virtual object recognition and/or tracking are evaluated. At least one monitoring signal is formed in dependence on the evaluation.

Claims

1. A method for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven, movable part wherein video images are used which were detected in a time sequence by at least two video cameras whose fields of view overlap at least in part in an overlap region in the monitored zone;

on the basis of the first video images detected by a first one of the video cameras, a first recognition and/or tracking of virtual objects is carried out and, on the basis of the second video images detected by a second one of the video cameras, a second recognition and/or tracking of virtual objects is carried out, the virtual objects corresponding to real objects at least in the overlap region;

for the classification and/or determination of the position and/or the movement of at least one real object in the overlap region, virtual objects corresponding to the real object from the first and second virtual object recognition and/or tracking processes are determined and data determined in each case for both virtual objects on their virtual object recognition and/or tracking are evaluated; and

at least one monitoring signal is formed in dependence on the evaluation; and

a fusion object is associated with virtual objects of the first and second object recognition and/or tracking processes corresponding to the same object.

2. A method in accordance with claim 1, wherein, for the definition of virtual objects in the video images, connected areas are determined which correspond to surface areas on an object corresponding to a virtual object which have similar surface properties in accordance with at least one pre-determined criterion.

3. A method in accordance with claim 1, wherein, for the definition of the virtual objects in the video images, connected areas are determined which have similar gray values and/or colors and/or textures in accordance with at least one pre-determined criterion.

4. A method in accordance with claim 1, wherein at least one virtual object class is assigned to the virtual objects.

5. A method in accordance with claim 1, wherein pre-determined features on virtual objects are recognized;

and wherein at least one virtual object class is associated with the virtual objects in dependence on the recognized pre-determined features.

6. A method in accordance with claim 1, wherein only moving virtual objects are recognized and/or tracked in the virtual object recognition and/or tracking.

7. A method in accordance with claim 6, wherein a background which does not move and/or change over the course of time is eliminated from the used video images for the determination of at least one moving virtual object.

8. A method in accordance with claim 1, wherein the position and/or movement and/or properties of said fusion object being determined in dependence on data with respect to the positions and/or movements and/or properties of the virtual objects corresponding to the real object.

9. A method in accordance with claim 8, wherein, after formation of a fusion object in a cycle, its data with respect to its position and/or movement and/or properties are up-dated in dependence on the data of the virtual objects corresponding to the real object with respect to their positions and/or movements and/or properties in at least one following cycle.

10. A method in accordance with claim 8, wherein, if, in the virtual object tracking of a virtual object associated with a fusion object, said virtual object is not recognized in a current video image, only the data of another virtual object associated with the fusion object with respect to its position and/or movement and/or properties are used for the determination of data with respect to the position and/or movement and/or properties of the fusion object.

11. A method in accordance with claim 8, wherein a fusion object class or a class for virtual object groupings or fusion object groupings is associated with at least one fusion object and/or one virtual object grouping or fusion object grouping.

12. A method in accordance with claim 11, wherein pre-determined features on objects are recognized;

and wherein at least one fusion object class or a class for virtual object groupings or fusion object groupings is associated with the fusion objects and/or virtual object groupings or fusion object groupings in dependence on the recognized, pre-determined features on virtual objects associated with the fusion objects or virtual object groupings or fusion object groupings.

13. A method in accordance with claim 8, wherein a protected zone is assigned to at least one moving fusion object and/or to one moving fusion object grouping;

and wherein a check is made whether another fusion object and/or another fusion object grouping is entering into the protected zone.

14. A method in accordance with claim 13, wherein a class is assigned to at least one fusion object and/or one fusion object grouping;

and wherein a protected zone is assigned to the fusion object or to the fusion object grouping at least in dependence on the assigned class.

15. A method in accordance with claim 13, wherein a protected zone is assigned to a fusion object and/or to a fusion object grouping at least in dependence on its position and/or movement, in particular on its movement direction and/or movement speed.

16. A method in accordance with claim 13, wherein a movement of at least one fusion object and/or of one fusion object grouping at at least one future point in time is predicted;

wherein a check is made whether a fusion object and/or a fusion grouping enters into the protected zone or the fusion object or the fusion object grouping enters into the protected zone of another fusion object and/or of another fusion object grouping; and

wherein a corresponding monitoring signal is emitted on the finding of an entry.

17. A method in accordance with claim 1, wherein one virtual object grouping or one fusion object grouping is formed from at least two virtual objects or fusion objects whose movement is coupled.

18. A method in accordance with claim 1, wherein a class for virtual objects or fusion objects or virtual object groupings or fusion object groupings is associated with a virtual object and/or a fusion object and/or a virtual object grouping and/or a fusion object grouping in dependence on whether at least one section of the virtual object or of the fusion object or of the virtual object grouping or fusion object grouping is moved into a pre-determined section of the monitored zone.

19. A method in accordance with claim 18, wherein at least one class is used for fusion objects and/or fusion object groupings for fusion objects or fusion groupings critical to safety and at least one class is used for fusion objects and/or fusion object groupings for neutral fusion objects or fusion object groupings.

20. A method in accordance with claim 1, wherein a check is made whether a pre-determined section of the overlap region, in particular a danger zone, is hidden in at least one of the video images.

21. A method in accordance with claim 1, wherein at least one pre-determined marking is recognized on at least one virtual object, which corresponds to a real object of a pre-determined type, said marking being characteristic for the type of real object;

wherein parts of at least one video image corresponding to the virtual object are compared with reference data which are pre-determined for the type of real object; and

wherein changes of the conditions in at least one part of the monitored zone, in particular lighting conditions, are recognized in dependence on the result of the comparison and/or a problem with a video camera used for the detection of the corresponding video image is recognized in dependence on the result of the comparison.

22. A computer program with program code means embodied as computer executable instructions on a computer readable medium and adapted, when installed on a computer, to carry out a method for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven, movable part wherein video images are used which were detected in a time sequence by at least two video cameras whose fields of view overlap at least in part in an overlap region in the monitored zone;

on the basis of the first video images detected by a first one of the video cameras, a first recognition and/or tracking of virtual objects is carried out and, on the basis of the second video images detected by a second one of the video cameras, a second recognition and/or tracking of virtual objects is carried out, the virtual objects corresponding to real objects at least in the overlap region;

for the classification and/or determination of the position and/or the movement of at least one real object in the overlap region, virtual objects corresponding to the real object from the first and second virtual object recognition and/or tracking processes are determined and data determined in each case for both virtual objects on their virtual object recognition and/or tracking are evaluated; and

at least one monitoring signal is formed in dependence on the evaluation; and

a fusion object is associated with virtual objects of the first and second object recognition and/or tracking processes corresponding to the same object.

23. A computer program product with program code means which are stored on a computer readable medium, said computer program product being adapted, when the computer program product is installed on a computer, to carry out a method for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven, movable part wherein video images are used which were detected in a time sequence by at least two video cameras whose fields of view overlap at least in part in an overlap region in the monitored zone;

on the basis of the first video images detected by a first one of the video cameras, a first recognition and/or tracking of virtual objects is carried out and, on the basis of the second video images detected by a second one of the video cameras, a second recognition and/or tracking of virtual objects is carried out, the virtual objects corresponding to real objects at least in the overlap region;

for the classification and/or determination of the position and/or the movement of at least one real object in the overlap region, virtual objects corresponding to the real object from the first and second virtual object recognition and/or tracking processes are determined and data determined in each case for both virtual objects on their virtual object recognition and/or tracking are evaluated; and

at least one monitoring signal is formed in dependence on the evaluation; and

a fusion object is associated with virtual objects of the first and second object recognition and/or tracking processes corresponding to the same object.

24. An apparatus for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven movable part with at least two video cameras by means of which video images of two different fields of view can be detected which overlap at least partly in an overlap region in the monitored zone;

and a data processing apparatus which is connected to the video cameras and is made to carry out a method for the monitoring of a monitored zone next to and/or in an apparatus having at least one driven, movable part wherein video images are used which were detected in a time sequence by at least two video cameras whose fields of view overlap at least in part in an overlap region in the monitored zone;

on the basis of the first video images detected by a first one of the video cameras, a first recognition and/or tracking of virtual objects is carried out and, on the basis of the second video images detected by a second one of the video cameras, a second recognition and/or tracking of virtual objects is carried out, the virtual objects corresponding to real objects at least in the overlap region;

for the classification and/or determination of the position and/or the movement of at least one real object in the overlap region, virtual objects corresponding to the real object from the first and second virtual object recognition and/or tracking processes are determined and data determined in each case for both virtual objects on their virtual object recognition and/or tracking are evaluated; and

at least one monitoring signal is formed in dependence on the evaluation; and

a fusion object is associated with virtual objects of the first and second object recognition and/or tracking processes corresponding to the same object.