Method for Detecting Objects Left-Behind in a Scene

• US 20080247599A1
• Filed: 04/05/2007
• Published: 10/09/2008
• Est. Priority Date: 04/05/2007
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Porikli, Fatih M., Ivanov, Yuri A.
• Granted Patent: US 7,813,528 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method detects an object left-behind in a scene by updating a set of background models using a sequence of images acquired of the scene by a camera. Each background model is updated at a different temporal scales ranging from short term to long term. A foreground mask is determined from each background model after the updating for a particular image of the sequence. A motion image is updated from the set of foreground masks. In the motion, image, each pixel has an associated evidence value. The evidence values are compared with a evidence threshold to detect and signal an object left behind in the scene.

Claims

1. A method for detecting an object left-behind in a scene, comprising the steps of:

updating a set of background models using a sequence of images acquired of a scene by a camera, each background model updated at a different temporal scales ranging from short term to long term;

determining a foreground mask from each background model after the updating for a particular image of the sequence;

updating a motion image from the set of foreground masks, in which each pixel in the motion image has an associated evidence value; and

comparing the evidence values with a evidence threshold to detect and signal an object left behind in the scene.

2. The method of claim 1, in which the detecting is based an inference of intensities of pixels in the sequence of images evolving over time.

3. The method of claim 1, in which the temporal scale is based on a frame rate the sequence of images is processed.

4. The method of claim 3, in which the temporal scale is short term if every image in the sequence is used during the updating, and the temporal scale is long term if the images to be processed are subsampled from the sequence.

6. The method of claim 1, in which the temporal scale is based on a frame rate the sequence of images are processed, and a learning rate at which parameters of the set of background models are updated.

7. The method of claim 6, in which the temporal scale is short term if the corresponding background models are updated more frequently than the background models for the long term at the same learning rate.

8. The method of claim 1, in which each background model is maintained as a set layers, each layer being a multivariate Gaussian distribution, and in which parameters define each Gaussian distributions at any point in time dining the updating.

9. The method of claim 8, in which the Gaussian distribution is multidimensional.

11. The method of claim 1, in which the updating of the motion image removes pixels in the motion image corresponding to slowly adapting background pixels and fast adapting foregrounds pixels, such that remaining pixels in the motion image correspond to the left-behind object.

12. The method of claim 1, in which the temporal scale depends on an application.

13. The method of claim 8, in which the updating of the set of background models uses a Bayesian mechanism comprising:

estimating in a probability distributions of a mean and a variance of the parameters of each layer.

14. The method of claim 13, in which an expectation of the mean and the variance for detecting the left-behind object, and a variance of the mean as a confidence score of the evidence values.

15. The method of claim 1, in which each foreground mask is a binary image having zero and one pixel values, and the one pixels values represent a possible foreground object in the sequence of images.

16. The method of claim 15, in which the set of foreground masks include a long-term mask F_L and a short-term mask F_S, and in which the updating of the motion image is according to evidence rules:

if F_L(x, y)=0 and F_S(x, y)=0, then the pixel (x, y) corresponds to a background in the scene, and a corresponding pixel 1(x, y) in a current image is represented in both the corresponding background models B_L and B_S;

if F_L(x, y)=1 and F_S(x, y)=1, then the pixel (x, y) corresponds to a moving object in the scene and is not represented in either of the background models B_L or B_S;

if F_L(x, y)=0 and FS(x, y)=1, then the pixel (x, y) corresponds to scene background, and the pixel 1(x, y) in the current image is only represented in the B_L model; and

if F_L(x, y)=1 and F_S(x, y)=0, then pixel (x, y) may correspond to the left-behind object.

17. The method of claim 16, in which the updating of the motion image M(x, y) is

$M$

( x , y ) = { M 

( x , y ) + 1 F L 

( x , y ) = 1 ⋀

F S 

( x , y ) = 0 M 

( x , y ) - k F L 

( x , y ) ≠

1 ⋁

F S 

( x , y ) ≠

0 max e M 

( x , y ) >

max e 0 M 

( x , y ) <

0 ,

where a threshold max_e and a delay constant k positive numbers.

20. The method of claim 12, in which k=max_e.

23. The method of claim 1, in which the set of background models are identical except for the temporal scale at the background models are updated.

24. The method of claim 1, further comprising:

applying connected components analysis on the pixels in the motion image and detecting and signaling the left-behind object according a size constraint.

25. The method of claim 1, further comprising:

applying connected components analysis on the pixels in the motion image and detecting and signaling the left-behind object according a pattern constraint.

5. The method of claim 5, in which the temporal scale is a learning rate at which parameters of the set of background models are updated.

10. The method of claim 5, in which the temporal scale is a rate at which the parameters are updated.

21. The method of claim 10, in which a subset of the background models is updated for any image in the sequence.

19. The method of claim 18, in which the evidence threshold max_e is in a range of [10, 300] successive images in the sequence.

22. The method of claim 18, in which a number of layers in each set is adapted according to the confidence score.