Method for evaluating a plurality of time-offset pictures, device for evaluating pictures, and monitoring system

• US 9,589,191 B2
• Filed: 02/17/2012
• Issued: 03/07/2017
• Est. Priority Date: 02/18/2011
• Status: Active Grant

• Current Assignee: Hella KGaA Hueck & Company
• Original Assignee: Hella KGaA Hueck & Company
• Inventors: Stefanovic, Marko, Badura, Rolf-Stephan, Karsch, Jan
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Tran, Mai
• Application Number:
  US14/000,107
• Publication Number:
  US 20140037141A1
• Time in Patent Office: 1,845 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08G 1/0116 from roadside infrastructure, e.g. beacons
• CPC Class Codes: G08G 1/0133 for classifying traffic situation

Abstract

The invention relates to a method for evaluating a plurality of chronologically staggered images, said method comprising the following steps:

• • detecting a plurality of objects in a first image and storing each of the plurality of objects as tracks with a first capture time and/or a first capture location, preferably in a track list,
  • detecting a plurality of objects in further images and identifying each of the detected objects as an object assigned to the respective stored track,
• wherein the respective track is updated by the current position of the identified object and, in the respective further images, objects detected for the first time are stored with assigned tracks, and
• wherein a covered path length, a distance and/or a time difference from the first capture time is determined as a capture period for each of the objects or tracks,
• wherein the path length, the distance and/or the capture period are compared with a respective predefined threshold value, and
• wherein the objects or tracks are classified according to the result of the comparison as objects or tracks to be taken into consideration or as objects or tracks not to be taken into consideration, wherein a number of stored tracks is determined for at least one of the images.

Claims

1. A method for evaluating a plurality of chronologically offset images, said method comprising the following steps:

detecting a plurality of objects in a first image and storing each of the plurality of objects as tracks with a first capture time and a first capture location,

detecting a plurality of objects in further images and identifying each of the objects detected in the further images as an object assigned to the respective stored track, wherein the respective track is updated by the current position of the object identified as the object assigned to this track and, in the respective further images, objects detected for the first time are stored with assigned tracks,

storing and outputting both a track list and a list of objects that have been assigned to one of the tracks and that have left an image region, wherein, for each of the objects in the list, the list contains at least one of the following;

an entry time into the image region, a residence period in the image region and an exit time from the image region, wherein objects that have left the image region are shifted from the track list into the list of objects that have left an image region, and

outputting a list filtered from the track list and/or a list filtered from the list of objects that have left an image region, wherein the filtering is based on an attribute of the objects.

2. The method according to claim 1, comprising determining a number of stored tracks for at least one of the images.

3. The method according to claim 1, comprising:

using the list to establish objects crossing counting lines, to assess re-entrants into a sub-region of the images, to separately assess the object numbers in a number of sub-regions of the images, to count people standing in queues, to create heat maps, and/or to create statistics concerning residence period.

4. The method according to claim 1, comprising outputting the list in a manner chronologically staggered with respect to the storing of said list.

5. The method according to claim 1, comprising outputting a plurality of lists in a chronologically staggered manner, wherein the plurality of lists are further processed together, in such a way that the lists are used together to establish objects crossing counting lines, to assess re-entrants into a sub-region of the images, to separately evaluate the object numbers in a plurality of sub-regions of the images, to count people standing in queues, to create heat maps, and/or to create statistics concerning residence period.

6. The method according to claim 1, comprising forming the list of objects as an output track list, wherein the output track list for each object specified in the list comprises information concerning a location at which the respective object is currently located, and wherein the output track list contains information concerning the time of the first capture of the objects, the first capture location, and/or the time at which the object was last captured.

7. The method according to claim 1, comprising forming the list as a list of objects that have left an image region, wherein, for each object in the list, an entry location into the image region, an exit location from the image region, and/or a last movement vector of the object is contained in the list.

8. The method according to claim 1, filtering out and outputting a list of the objects that were last shifted from the list of objects currently contained in the image into the list of objects that have left an image region.

9. The method according to claim 1, comprising outputting the list whenever a predefined trigger condition is met.

10. The method according to claim 1, comprising assigning the objects in the list a further attribute and/or sub-dividing the list into sub-lists according to a further attribute.

11. The method according to claim 1, comprising controlling an output of the list via an HMI or MAPI.

12. The method according to claim 1, comprising, for the objects or tracks, determining a covered path length, a distance, and/or a time difference from the first capture time as a capture period, wherein the path length, the distance and/or the capture period is/are compared with a respective predefined threshold value, and wherein the objects or tracks are classified according to the result of the comparison as either objects or tracks to be taken into consideration or as objects or tracks not to be taken into consideration.

13. The method according to claim 1, comprising automatically changing the predefined threshold values by a user and/or by an optimization algorithm before or during the detection of the at least one object.

14. The method according to claim 1, wherein the tracks are each then only taken into consideration when determining the number of tracks and/or are then only classified as tracks to be taken into consideration if the respective corresponding object is arranged in a predefined sub-region of the images.

15. The method according to claim 1, comprising outputting the specific number of tracks or objects continuously or in accordance with a plurality of images or upon the command of a user.

16. The method according to claim 1, comprising determining and storing an exit time from the plurality of images and/or the sub-region for each of the objects.

17. The method according to claim 14, comprising outputting the first capture time and the exit time and/or a time difference between the first capture time and exit time.

18. The method according to claim 1, comprising evaluating as images recorded images captured by an image capture system in real time.

19. The method according to claim 1, comprising determining a number, averaged over time, of tracks and/or a time difference, averaged over time, between the first capture time and exit time.

20. The method according to claim 1, wherein a plurality of objects are stored, wherein, once a predefined number of objects has been reached and/or once a predefined memory demand has been reached, an oldest object is overwritten.

21. The method according to claim 1, wherein the objects are people or items of luggage.

24. The method according to claim 1, wherein the attribute includes a capture location.

25. The method according to claim 1, wherein the attribute includes a capture time.

22. A system comprising a device for evaluating images, comprising an image processor circuit configured to:

detect a plurality of objects in a first image and storing each of the plurality of objects as tracks with a first capture time and a first capture location;

detect a plurality of objects in further images and identify each of the objects detected in the further images as an object assigned to the respective stored track, wherein the respective track is updated by the current position of the object identified as the object assigned to this track and, in the respective further images, objects detected for the first time are stored with assigned tracks;

store and output both a track list and a list of objects that have been assigned to one of the tracks and that have left an image region, wherein, for each of the objects in the list, the list contains at least one of the following;

an entry time into the image region, a residence period in the image region and an exit time from the image region, wherein objects that have left the image region are shifted from the track list into the list of objects that have left an image region, and

output a list filtered from the track list and/or a list filtered from the list of objects that have left an image region, wherein the filtering is based on an attribute of the objects.

23. A surveillance system comprising at least one image capture system, and a device for evaluating images, comprising an image processor circuit configured to:

detect a plurality of objects in a first image and storing each of the plurality of objects as tracks with a first capture time and a first capture location;

detect a plurality of objects in further images and identify each of the objects detected in the further images as an object assigned to the respective stored track, wherein the respective track is updated by the current position of the object identified as the object assigned to this track and, in the respective further images, objects detected for the first time are stored with assigned tracks;

store and output both a track list and a list of objects that have been assigned to one of the tracks and that have left an image region, wherein, for each of the objects in the list, the list contains at least one of the following;

an entry time into the image region, a residence period in the image region and an exit time from the image region, wherein objects that have left the image region are shifted from the track list into the list of objects that have left an image region, and output a list filtered from the track list and/or a list filtered from the list of objects that have left an image region, wherein the filtering is based on an attribute of the objects.

26. A method for evaluating a plurality of chronologically offset images, said method comprising the following steps:

detecting a plurality of objects in a first image and storing each of the plurality of objects as tracks with a first capture time and a first capture location,

detecting a plurality of objects in further images and identifying each of the objects detected in the further images as an object assigned to the respective stored track, wherein the respective track is updated by the current position of the object identified as the object assigned to this track and, in the respective further images, objects detected for the first time are stored with assigned tracks,

storing and outputting a mixed list of objects that have been assigned to one of the tracks, the list containing objects that have not yet left an image region and objects that have left an image region, wherein, for each of the objects in the list, the list contains at least one of the following;

an entry time into the image region, a residence period in the image region and an exit time from the image region, and wherein a piece of information is assigned to each of the objects in the list, said piece of information indicating whether the object has already left the image region or not and being adapted if the object leaves the image region, and

outputting a list filtered from the mixed list, wherein the filtering is based on an attribute of the objects.