METHOD AND SYSTEM FOR AUTOMATIC OBJECT DETECTION AND SUBSEQUENT OBJECT TRACKING IN ACCORDANCE WITH THE OBJECT SHAPE

• US 20120148103A1
• Filed: 08/22/2010
• Published: 06/14/2012
• Est. Priority Date: 08/23/2009
• Status: Active Grant

• Current Assignee: iAd Gesellschaft Automatisierung und Datenverarbeitungs GmbH
• Original Assignee: iAd Gesellschaft Automatisierung und Datenverarbeitungs GmbH
• Inventors: Hampel, Hermann, Berold, Ulrich, Quast, Katharina, Kaup, Andre
• Granted Patent: US 8,948,450 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method and system for automatic object detection and subsequent object tracking in accordance with the object shape in digital video systems having at least one camera for recording and transmitting video sequences. In accordance with the method and system, an object detection algorithm based on a Gaussian mixture model and expanded object tracking based on Mean-Shift are combined with each other in object detection. The object detection is expanded in accordance with a model of the background by improved removal of shadows, the binary mask generated in this way is used to create an asymmetric filter core, and then the actual algorithm for the shape-adaptive object tracking, expanded by a segmentation step for adapting the shape, is initialized, and therefore a determination at least of the object shape or object contour or the orientation of the object in space is made possible.

Claims

1. A method for automatic object detection and subsequent object tracking in digital video systems having at least one camera for recording and transmitting video sequences, as well as further processing or evaluation of video data, wherein an object detection algorithm based on a Gaussian mixture model and an expanded object tracking based on Mean-Shift to be combined with each other in the object detection, whereby

the object detection is expanded in accordance with a model of the background by improved removal of shadows,

the binary mask generated in this way is used to create an asymmetric filter core, and

then the actual algorithm for the shape-adaptive object tracking, expanded by a segmentation step for adapting the shape is initialized,

so that a determination of at least of the object shape or the object contour or the orientation of the object in space is made possible.

2. The method according to claim 1, wherein using a asymmetric filter core adapted to the object form, and in that working the mean shift algorithm only with the actual object characteristics color -, edges- or structure information so that during the initialization information about the contour of the object to be obtained.

3. The method according to claim 1, wherein adapting the localized object region as closely as possible to the actual object shape, a color segmentation or a Mean shift based color segmentation of the object region and its immediate object region is made, in that classifying as an object segments all segments which are falling at least 35% to 70% within the object region, and in that categorizing as an object segments also all the segments whose color information are included at least to 35% to 70% in the current target model.

4. The method according to claim 1, wherein taking into account in the model form at least one of the object characteristics color-, edges- or structural-information or the so-called SIFT features and in that integrating in the GMM method a limitation of the standard deviation as well as an exploiting of temporal and spatial correlation.

5. The method according method according to claim 1, wherein subtracting the current background model from the current image of the video sequence changing image regions are detected and in that subsequently determining a binary mask, which contains the moving image regions from the difference between the background and the current picture by a threshold decision.

6. The method according to claim 1, wherein the detection of penumbra and umbra takes place according to the equation

φ

<

(φ

₂−

φ

₁/r₂−

r₁)*(r−

r₁)+φ

₁

with a lower threshold r₁ of maximum allowed darkness, with an upper threshold r₂ of maximum allowed brightness, with a current brightness value r, and with an angle φ

₁ indicates the minimum allowable deviation of the color saturation of the current pixel from the background as well as an angle φ

2 indicates the maximum allowed deviation of the color saturation of the current pixel from the background and an angle φ

, which measures the current deviation of the color saturation of the current pixel from the background, so that a simultaneous removal of penumbra and umbra by totally only four parameters is possible.

7. The method according to claim 1, wherein making a so-called Connected Component Analysis of the binary mask after the shadow removal for the determination of related object regions.

8. The method according to claim 7, wherein classifying as reliable objects if in several consecutive images areas are recognized, and in that generating an asymmetric filter kernel for the reliably recognized objects based on the current binary mask by calculating the normalized distance to the object boundary for each pixel within the binary mask.

13. A computer program comprising program code means to carry out all steps of the method according to claim 1 if said program is running on a computer and/or a device comprising at least one camera for recording video sequences to which are connected a means for automatic object recognition and then a means for object tracking and to which means for further processing or analysis of the video signal of the camera is connected, wherein the video signal of the camera is supplied to both an object detection means based on a Gaussian Mixture Model object recognition algorithm and a means having an enlarged based on Mean shift object tracking, in that the object detection means comprises means for object recognition in accordance with a model of the background enlarged by an improved shadow removal, in that the at the output arising binary mask of the object recognition means is connected to the object tracking means, which comprises means for creating an asymmetric filter core and means for the adaptive object tracking expanded to a segmentation step comprises the shape adaptation, and in that at the output arising video signal of the object tracking means is connected to the processing/evaluating means for determining the shape of the object or the object contour or the orientation of the object.

9. A system comprising at least one camera for recording video sequences to which are connected a means for automatic object recognition and then a means for object tracking and to which means for further processing or analysis of the video signal of the camera is connected, wherein the video signal of the camera is supplied to both an object detection means based on a Gaussian Mixture Model object recognition algorithm and a means having an enlarged based on Mean shift object tracking, in that the object detection means comprises means for object recognition in accordance with a model of the background enlarged by an improved shadow removal, in that the at the output arising binary mask of the object recognition means is connected to the object tracking means, which comprises means for creating an asymmetric filter core and means for the adaptive object tracking expanded to a segmentation step comprises the shape adaptation, and in that at the output arising video signal of the object tracking means is connected to the processing/evaluating means for determining the shape of the object or the object contour or the orientation of the object.

10. The system according to claim 9, wherein the object detection means comprises in series means for limiting the standard deviation, means for taking into account the temporal correlation, means for shadow removal and means to take account of local correlation which is connected to an input of a subtraction circuit, in that to the other input of said subtraction circuit the video signal of the camera is connected and in that at the output of said subtraction circuit the binary mask arises.

11. The system according to claim 9, wherein the object tracking means comprises a means for creating the target model via histogram, a means for generating the filter core based on the binary mask and a means for form adaptive object tracking and in that the binary mask is supplied to the input of the means for generating the filter core, and the video signal of the camera is supplied to the input of the means to create the target model of histogram, wherein at the output of the means for form adaptive object tracking, which is connected to the output of the means for creating the target model via histogram, the input video signal connected to the processing/evaluating means arises.

12. The system according to claim 9, wherein the video signal of the camera is supplied both to a means for the calculation of the background and to a means for forming the difference of the object detection means, wherein the second input of the means for forming the difference is connected to the output of the means for calculating the background, and in that at the output of the means for forming the difference a means for threshholding is connected, comprising in series a means for shadow removal, a means for performing a connected component analysis and a means for comparison of components connected with persecuted objects, whereas at the output of said comparison means the binary mask arises.