Method and system for searching for events in video surveillance

• US 20050180603A1
• Filed: 01/25/2005
• Published: 08/18/2005
• Est. Priority Date: 01/27/2004
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Ramesh, Visvanathan, Paragios, Nikolaos, Zoghlami, Imad
• Granted Patent: US 7,555,046 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method for detecting events in a video sequence includes providing a video sequence, sampling the video sequence at regular intervals to form a series of snapshots of the sequence, measuring a similarity of each snapshot, measuring a similarity change between successive pairs of snapshots, wherein if a similarity change magnitude is greater than a predetermined threshold, a change event has been detected, verifying the change event to exclude a false positive, and completing the processing of the snapshot incorporating the verified change event.

Claims

1. A method for detecting events in a video sequence, said method comprising the steps of:

providing a video sequence;

sampling the video sequence at regular intervals to form a series of snapshots of the sequence;

measuring a similarity of each snapshot;

measuring a similarity change between successive pairs of snapshots, wherein if a similarity change magnitude is greater than a predetermined threshold, a change event has been detected;

verifying the change event to exclude a false positive; and

completing the processing of the snapshot incorporating the verified change event.

2. The method of claim 1, wherein the sampling interval is from a few seconds to a few minutes.

3. The method of claim 1, further comprising defining one or more windows-of-interest in each snapshot, and measuring the similarity in each window-of-interest in each snapshot.

4. The method of claim 3, wherein the similarity measure for a window-of-interest in a snapshot is defined as

$S_w^0$

( t ) = 1 

W 

⁢

∑

x ∈

W ⁢

( x t - x _ t ) ⁢

( x t - x _ t ) ,

where x_t represents the pixel intensity for a pixel in a window-of-interest W of snapshot t, and {overscore (x)}_t is a spatial intensity average in the window for the snapshot.

5. The method of claim 4, wherein the similarity measure is normalized to the spatial intensity scale of the window-of-interest according to the formula

$S_w$

( t ) = 1 

W 

* x _ t ⁢

∑

x ∈

W ⁢

( x t - x _ t ) * ( x t - x _ t ) .

6. The method of claim 1, wherein the change in the similarity measure is determined from the time derivative of the similarity measure.

7. The method of claim 1, wherein a false positive includes an occlusion.

8. The method of claim 7, further comprising eliminating an occlusion by weighting a time derivative of the similarity measure according to the definition

$f_w$

( t ) = g ⁡

( t ) * S . w ⁡

( t ) ⁢

 

⁢

wherein $g<br><br>\left(t\right)=h<br><br>\left(\underset{i\in <br><br>\left[n_1,n_2\right],j\in <br><br>\left[n_1,n_2\right]}{\min }<br><br>\mathrm{similarity}<br><br>\left(w_{t-i},w_{t+j}\right)\right),\mathrm{wherein}$ $\mathrm{similarity}<br><br>\left(w_i,w_j\right)=\frac{1}{n}<br><br>\underset{k=1}{\overset{n}{\sum <br><br>}}<br><br><br><br>{\mathrm{hist}}_i<br><br>\left[k\right]-{\mathrm{hist}}_j<br><br>\left[k\right]<br><br>,$

and wherein {dot over (S)}_w(t) is the similarity measure time derivative, w_i, w_j are corresponding windows-of-interest in a pair of successive snapshots, [n₁, n₂] is the duration neighborhood about the snapshot incorporating the occlusion over which similarity is being sought, h is a positive increasing function with h(1)=1, and hist is a histogram of spatial intensity values in the window-of-interest.

9. The method of claim 8, wherein h(x)∝

x².

10. The method of claim 1, wherein a false positive includes a change of illumination.

11. The method of claim 1, wherein the predetermined threshold is based on an analysis of the fluctuations in the similarity change between successive pairs of snapshots.

12. The method of claim 11, wherein the threshold is more than three standard deviations greater than the mean fluctuation magnitude of the similarity change between successive pairs of snapshots.

13. A method for searching for events in a video sequence, said method comprising the steps of:

providing a series of snapshots of a video sequence;

defining one or more windows-of-interest in each snapshot;

measuring a similarity of each window-of-interest in a snapshot according to the formula

$S_w^0<br><br>\left(t\right)=\frac{1}{<br><br>W<br><br>}<br><br>\sqrt{\underset{x\in <br><br>W}{\sum <br><br>}<br><br>\left(x_t-{\stackrel{\_}{x}}_t\right)<br><br>\left(x_t-{\stackrel{\_}{x}}_t\right)},$

 

where x_t represents the pixel intensity for a pixel in a window-of-interest W of snapshot t, and {overscore (x)}_t is a spatial intensity average in the window for the snapshot; and

measuring a similarity change between successive pairs of snapshots from a time derivative of the similarity measure, wherein if a similarity change magnitude is greater than a predetermined threshold, a change event has been detected, wherein the threshold is based on an analysis of the fluctuations in the similarity change between successive pairs of snapshots.

14. The method of claim 13, wherein the video sequence was sampled at regular intervals to form the series of snapshots.

15. The method of claim 14, wherein the sampling interval is from a few seconds to a few minutes.

16. The method of claim 13, wherein the similarity measure is normalized to the spatial intensity scale of the window-of-interest according to the formula

$S_w$

( t ) = 1 

W 

* x _ t ⁢

∑

x ∈

W ⁢

( x t - x _ t ) * ( x t - x _ t ) .

17. The method of claim 13, further comprising the steps of verifying the change event to eliminate a false positive, wherein a false positive includes an occlusion and a change of illumination, and completing the processing of the snapshot incorporating the verified change event.

18. The method of claim 17, further comprising eliminating an occlusion by weighting the time derivative of the similarity measure according to the definition

$\begin{array}{c}{f}_w\end{array}$

( t ) = g ⁡

( t ) * S . w ⁡

( t ) wherein g ⁡

( t ) = h ⁡

( min i ∈

[ n 1 , n 2 ] , j ∈

[ n 1 , n 2 ] ⁢

similarity ⁡

( w t - i , w t + j ) ) , wherein similarity ⁢

 

⁢

( w i , w j ) = 1 n ⁢

∑

k = 1 n ⁢

 

⁢



hist i ⁡

[ k ] - hist j ⁡

[ k ] 

,

and wherein {dot over (S)}_w(t) is the similarity measure time derivative, w_i, w_j are corresponding windows-of-interest in a pair of successive snapshots, [n₁,n₂] is the duration neighborhood about the snapshot incorporating the occlusion over which similarity is being sought, h is a positive increasing function with h(1)=1, and hist is a histogram of spatial intensity values in the window-of-interest.

19. The method of claim 18, wherein h(x)∝

x².

20. The method of claim 13, wherein the threshold is more than three standard deviations greater than the mean fluctuation magnitude of the similarity change between successive pairs of snapshots.

21. A program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform the method steps for detecting events in a video sequence, said method comprising the steps of:

providing a video sequence;

sampling the video sequence at regular intervals to form a series of snapshots of the sequence;

measuring a similarity of each snapshot;

measuring a similarity change between successive pairs of snapshots, wherein if a similarity change magnitude is greater than a predetermined threshold, a change event has been detected;

verifying the change event to exclude a false positive; and

completing the processing of the snapshot incorporating the verified change event.

22. The computer readable program storage device of claim 21, wherein the sampling interval is from a few seconds to a few minutes.

23. The computer readable program storage device of claim 21, wherein the method further comprises defining one or more windows-of-interest in each snapshot, and measuring the similarity in each window-of-interest in each snapshot.

24. The computer readable program storage device of claim 23, wherein the similarity measure for a window-of-interest in a snapshot is defined as

$S_w^0$

( t ) = 1 

W 

⁢

∑

x ∈

W ⁢

( x t - x _ t ) ⁢

( x t - x _ t ) ,

where x_t represents the pixel intensity for a pixel in a window-of-interest W of snapshot t, and {overscore (x)}_t is a spatial intensity average in the window for the snapshot.

25. The computer readable program storage device of claim 24, wherein the similarity measure is normalized to the spatial intensity scale of the window-of-interest according to the formula

$S_w$

( t ) = 1 

W 

* x _ t ⁢

∑

x ∈

W ⁢

( x t - x _ t ) * ( x t - x _ t ) .

26. The computer readable program storage device of claim 21, wherein the change in the similarity measure is determined from the time derivative of the similarity measure.

27. The computer readable program storage device of claim 21, wherein a false positive includes an occlusion.

28. The computer readable program storage device of claim 27, wherein the method further comprises eliminating an occlusion by weighting a time derivative of the similarity measure according to the definition

$\begin{array}{c}{f}_w\end{array}$

( t ) = g ⁡

( t ) * S . w ⁡

( t ) wherein g ⁡

( t ) = h ⁡

( min i ∈

[ n 1 , n 2 ] , j ∈

[ n 1 , n 2 ] ⁢

similarity ⁡

( w t - i , w t + j ) ) , wherein similarity ⁢

 

⁢

( w i , w j ) = 1 n ⁢

∑

k = 1 n ⁢

 

⁢



hist i ⁡

[ k ] - hist j ⁡

[ k ] 

,

and wherein {dot over (S)}_w(t) is the similarity measure time derivative, w_i, w_j are corresponding windows-of-interest in a pair of successive snapshots, [n₁,n₂] is the duration neighborhood about the snapshot incorporating the occlusion over which similarity is being sought, h is a positive increasing function with h(1)=1, and hist is a histogram of spatial intensity values in the window-of-interest.

29. The computer readable program storage device of claim 28, wherein h(x)∝

x².

30. The computer readable program storage device of claim 21, wherein a false positive includes a change of illumination.

31. The computer readable program storage device of claim 21, wherein the predetermined threshold is based on an analysis of the fluctuations in the similarity change between successive pairs of snapshots.

32. The computer readable program storage device of claim 31, wherein the threshold is more than three standard deviations greater than the mean fluctuation magnitude of the similarity change between successive pairs of snapshots.