Method and system for automatic object detection and subsequent object tracking in accordance with the object shape

• US 8,948,450 B2
• Filed: 08/22/2010
• Issued: 02/03/2015
• Est. Priority Date: 08/23/2009
• Status: Active Grant

• Current Assignee: iAd Gesellschaft Automatisierung und Datenverarbeitungs GmbH
• Original Assignee: IAD Gesellschaft Fur Informatik Automatisierung und Datenverarbeitung mbH
• Inventors: Hampel, Hermann, Berold, Ulrich, Quast, Katharina, Kaup, Andre
• Primary Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US13/391,790
• Publication Number:
  US 20120148103A1
• Time in Patent Office: 1,626 Days
• Field of Search: 382/103, 382/104, 382/164, 382/165, 382/173, 382/203, 382/204, 382/224, 382/260, 382/261, 382/267, 348/113, 348/169
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method and system for automatic object detection and subsequent object tracking in accordance with the object shape in digital video systems having at least one camera for recording and transmitting video sequences. In accordance with the method and system, an object detection algorithm based on a Gaussian mixture model and expanded object tracking based on Mean-Shift are combined with each other in object detection. The object detection is expanded in accordance with a model of the background by improved removal of shadows, the binary mask generated in this way is used to create an asymmetric filter core, and then the actual algorithm for the shape-adaptive object tracking, expanded by a segmentation step for adapting the shape, is initialized, and therefore a determination at least of the object shape or object contour or the orientation of the object in space is made possible.

Claims

1. A method for automatic object detection and subsequent object tracking in digital video systems having at least one camera for recording and transmitting video sequences, said method comprising:

supplying a video signal from the at least one camera to a control room;

supplying the video signal from the at least one camera to each of;

an object detection module; and

an object tracking module;

processing the video signal in the object detection module using an algorithm based on a Gaussian mixture model to detect one or more objects in the video sequence;

processing the video signal in the object tracking using Mean-Shift analysis to track one or more objects in the video sequence,

further processing the video signal in the object detection module using background modeling to improve the removal of shadows from the one or more objects detected in the video sequence,

generating a binary mask at an output of the object detection module that determines whether a newly identified object is present in the video sequence and creates an asymmetric filter core,

supplying an output of the binary mask to the object tracking module, and

initializing an object detection algorithm for shape-adaptive object tracking in the object tracking module, wherein the object detection algorithm for shape adaptive object tracking further comprises a segmentation step that comprises;

adapting a localized object region as closely as possible to actual object shape,

making a color segmentation or a Mean shift based color segmentation of the object and its immediate object region,

classifying as object segments all segments that fall at least 35% to 70% within the immediate object region, and

categorizing as object segments all segments whose color information falls at least to 35% to 70% within a target model, such that a determination of at least one of object shape or object contour or orientation of the object in space is made, and

supplying an output of the object tracking module to the control room with the video signal supplied by the at least one camera.

2. The method according to claim 1, wherein, during the initializing step, information about a contour of the object is obtained using the asymmetric filter core and the mean shift algorithm using only an actual object characteristic selected from the group consisting of color, edges and structure information.

3. The method according to claim 2, wherein the selected actual object characteristic or Scale invariant feature transform (SIFT) feature is considered in the asymmetric filter core of the binary mask, and wherein the method further comprises integrating, using a GMM method, a limitation of a standard deviation as well as an exploitation of temporal and spatial correlation in the object detection module.

4. The method according method according to claim 1, wherein changing image regions are detected by subtracting a current background model from a current image of the video sequence, and wherein the binary mask implements a threshold decision from a difference between the background model and the current image.

5. The method according to claim 1, wherein a detection of penumbra and umbra takes place according to the equation

φ

<

(φ

₂−

φ

₁)/(r₂−

r₁)*(r−

r₁)+φ

₁

wherein r₁ represents a lower threshold of maximum allowed darkness, wherein r₂ represents an upper threshold of maximum allowed brightness, wherein r represents a current brightness value, wherein φ

₁ represents an angle indicating a minimum allowable deviation of color saturation of a current pixel from background, and wherein φ

₂ represents an angle indicating a maximum allowed deviation of color saturation of the current pixel from the background and an angle φ

, which measures a current deviation of color saturation of the current pixel from the background, so that a simultaneous removal of penumbra and umbra by only four parameters occurs.

6. The method according to claim 1, further comprising performing a Connected Component Analysis of the binary mask after shadow removal to determine related object regions.

7. The method according to claim 6, wherein, if in several consecutive images areas are recognized, such areas are classified as reliable objects, and wherein an asymmetric filter kernel is generated for the reliable objects based on the current binary mask by calculating a normalized distance to an object boundary for each pixel within the binary mask.

12. A non-transitory computer-readable medium encoded with a computer program comprising program code for carrying out all steps of the method according to claim 1 when said program is running on a computer and/or a device comprising:

at least one camera for recording a video sequence and transmitting a video signal;

an object detection module for receiving the video signal from the camera and automatically detecting one or more objects in the video sequence, wherein the object detection module generates and outputs a binary mask, wherein object detection is based on a model of a background image that is generated by Gaussian Mixture Model algorithm, and wherein the object detection module further removes shadows from the background image;

an object tracking module for receiving the video signal the camera and an output from the binary mask, said module being configured to conduct mean-shift based analysis object tracking in order to output information about object position and object form from the video signal and the binary mask, wherein the object tracking module further comprises an asymmetric filter core and means for an adaptive object tracking that performs shape adaptation segmentation; and

a processing/evaluating means, in a control room, that is configured to receive the video signal from the camera, the information about object position and object form and thereby determine the shape of the object or the object contour or the orientation of the object.

8. A system comprising:

at least one camera for recording a video sequence and transmitting a video signal;

an object detection module that is configured to receive the video signal from the camera and detect one or more objects in the video sequence using a model of a background image that is generated by a Gaussian Mixture Model algorithm and also remove shadows from the background image and generates and outputs a binary mask;

an object tracking module that is configured to receive the video signal from the camera and an output from the binary mask and perform a mean-shift based analysis object tracking in order to output information about object position and object form from the video signal and the binary mask, wherein the tracking means comprises an asymmetric filter core and means for an adaptive object tracking that performs object shape adaptation segmentation; and

a processing/evaluating means, in a control room, that is configured to receive the video signal from the camera, the information about object position and object form and thereby determine the shape of the object or the object contour or the orientation of the object.

9. The system according to claim 8, wherein the object detection module comprises, in series, limitation means for limiting standard deviation, considering means for taking into account temporal correlation, removal means for shadow removal, and considering means for taking into account local correlation, wherein the object detection module is connected to an input of a subtraction circuit, wherein the video signal of the camera is connected to an other input of said subtraction circuit, and wherein the binary mask is formed at an output of said subtraction circuit.

10. The system according to claim 8, wherein the object tracking module comprises a creating means for creating the target model via a histogram, and a generating means for generating the filter core based on the binary mask, and wherein the binary mask is supplied to an input of the means for generating the filter core, and the video signal of the camera is supplied to an input of the means to create the target model via the histogram.

11. The system according to claim 8, wherein the video signal of the camera is supplied both to a means for calculating background and to a means for identifying a difference between objects and background, wherein a second input of the means for identifying the difference is connected to an output of the means for calculating the background, and wherein a means for thresholding is connected at the output of the means for identifying the difference, said means for thresholding comprising in series a means for shadow removal, a means for performing a connected component analysis and a means for comparison of components connected with persecuted objects, and wherein the binary mask is at the output of said comparison means.