Unusual event detection in wide-angle video (based on moving object trajectories)

• US 8,284,258 B1
• Filed: 09/18/2009
• Issued: 10/09/2012
• Est. Priority Date: 09/18/2008
• Status: Expired due to Fees

• Current Assignee: Grandeye Limited (Oncam Global Group AG)
• Original Assignee: Grandeye Limited (Oncam Global Group AG)
• Inventors: Cetin, A. Enis, Noyan, Timucin, Davey, Mark K
• Primary Examiner(s): Boutah, Alina N.
• Application Number:
  US12/562,715
• Time in Patent Office: 1,117 Days
• Field of Search: 348/36, 348/169, 348/E5.055, 382/103, 382106-107, 382/190
• US Class Current: 348/169
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Object images captured by a wide-angle camera are distorted due to the optical effects of the wide-angle lens. The disclosed innovations allow an automatic analysis on the corrected image distinguishing normal movement from an unusual event movement. The analysis is based on Markov Modeling on moving object trajectories and motion angles.

Claims

1. A method of monitoring moving objects in a wide-angle video, comprising the steps of:

determining moving object trajectories;

converting the moving object trajectories to a trajectory in a perspectively corrected image domain; and

interpreting the object trajectories for detecting unusual behavior using state transition probability models of non-hidden Markov models.

2. The method of claim 1, wherein the object trajectories are determined on the distorted image domain captured by the imaging sensor of the wide-angle camera.

3. The method of claim 1, wherein the said distorted wide-angle video is corrected inside the camera.

4. The method of claim 1, wherein the moving objects are determined in the distorted image domain using an image background subtraction method involving a recursive estimation of the background image in the distorted image domain.

5. The method of claim 1, wherein the detected moving objects are tracked in distorted image domain using a mean-shift object tracking method operating in the distorted image domain.

6. The method of claim 4, wherein the detected moving objects are tracked in distorted image domain using a mean-shift object tracking method operating in the distorted image domain.

7. The method of claim 5, wherein the said mean-shift tracking method computes a data object histogram from a real-valued discrete wavelet transform.

8. The method of claim 1, wherein the said moving object trajectories in corrected image domain are interpreted using Markov Models with states defined according to motion vector angles of moving objects for unusual event detection.

9. The method of claim 1, wherein the state transition probability models are trained with motion vectors in Cartesian coordinates and polar coordinates computed in corrected image domain.

10. A method of monitoring moving objects in a wide-angle video, comprising the steps of:

determining moving object trajectories; and

interpreting the object trajectories for detecting unusual behavior for an unusual event in a distorted image domain using a state transition analysis of Markov Models.

11. The method of claim 10, wherein the object trajectories are determined on the distorted image domain captured by the imaging sensor of the wide-angle camera.

12. The method of claim 10, wherein the moving objects are determined in the distorted image domain using an image background subtraction method involving a recursive estimation of the background image in the distorted image domain.

13. The method of claim 10, further comprising the step of:

tracking a detected moving object in a distorted image domain using a mean-shift object tracking method operating in the distorted image domain.

14. The method of claim 10, wherein the unusual event is detected when a moving object enters into a forbidden region pre-specified by an operator.

15. The method of claim 10, wherein the said forbidden region is specified on the perspectively corrected image domain.

16. A system for detecting unusual events in a wide-angle video, comprising:

an electronic camera that emulates PTZ function capturing wide-angle digital video images; and

an image data processor that calculates and converts moving pixel blob trajectories in a non-rectilinear image domain and analyzes the blob trajectories as unusual behavior using state transition probability models applied to a data object histogram;

wherein the state transition probability models are trained using prior trajectory data corresponding to regular and unusual motion trajectories of moving objects, and wherein the state transition probability models are non-hidden Markov models.

17. The system of claim 16, wherein the data object histogram is obtained from wavelet coefficients of moving objects.

18. The system of claim 16, wherein the pixel blobs are derived from image data by segmenting non-rectilinear image data into moving foreground pixels and stationary background pixels, wherein a background image is obtained by either recursively averaging the past images of the digital video or by median filtering.

19. The system of claim 16, further comprising:

a system classification output of an unusual event according to a conditional probability if ones of unusual state transition probability models exceed the conditional probability of ones of regular event state transition probability models.