Method and apparatus for background segmentation based on motion localization

• US 20050104964A1
• Filed: 10/22/2001
• Published: 05/19/2005
• Est. Priority Date: 10/22/2001
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Molinov, Sergei A, Eruhimov, Viktor Lvovich, Bovyrin, Alexandr V
• Application Number:
  US10/451,517
• Publication Number:
  US 20050104964A1
• Time in Patent Office: Days
• US Class Current: 348/155
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

A system (1000) and method of detecting static background on a video sequence of images with moving foreground objects is described. The method includes localizing moving objects in each frame and training a background model using the rest of the image. The system is also capable of handling occasional background changes and camera movements.

Claims

1. A method of extracting a background image, comprising:

localizing a moving object in a video sequence based on a change in the moving object over a plurality of frames of the video sequence, the moving object occupying frame areas of changing color; and

training a background model for the plurality of frames outside of the frame areas of changing color.

2. The method of claim 1, wherein localizing comprises localizing the moving object using a change detection mask.

3. The method of claim 1, wherein localizing comprises:

determining a boundary for the moving object that is of homogenous color; and

constructing a hull around the moving object using the boundary.

4. The method of claim 3, wherein determining a boundary comprises:

determining a maximum contour of a plurality of contours of the moving object, the maximum contour having the largest area of the plurality of contours;

determining other contours of the moving object; and

joining the maximum contour with the other contours.

5. The method of claim 4, further comprising:

eliminating the smallest contour from joining with the maximum contour.

6. The method of claim 4, wherein joining comprises joining one of the other contours with the maximum contour if the distance between the maximum contour and the one of the other contours is less than a predetermined distance.

7. The method of claim 6, wherein the frames comprise a plurality of pixels and wherein the predetermined distance is based on a probability that a pixel of the plurality of pixels is considered moving given that it corresponds to the moving object.

8. The method of claim 7, wherein the predetermined distance is based on a probability that the pixel is considered moving given that it is static.

9. The method of claim 3, wherein the frames comprise a plurality of pixels and wherein the hull is constructed to contain only pixels of changing colors over consecutive frames.

10. The method of claim 3, wherein constructing the hull comprises:

determining all connected components in the boundary, wherein each of the components has a contour having an area;

filtering out a smallest area contour;

selecting a maximum area contour; and

joining the maximum area contour with other contours of the connected components.

11. The method of claim 1, wherein the frames comprise a plurality of pixels and wherein training comprises characterizing a pixel color at a given time with a value based on a state, each pixel corresponding to a state of a plurality of states.

12. The method of claim 11, wherein the plurality of states includes an untrained background state.

13. The method of claim 11, wherein the plurality of states includes a trained background state.

14. The method of claim 11, wherein the plurality of states includes a foreground state.

15. The method of claim 11, wherein the plurality of states includes an unknown background state.

16. The method of claim 11, wherein training comprises:

training the background model for the pixel in a foreground; and

changing the state of the pixel to an untrained background if the pixel represents a static behavior for a certain period of time.

17. The method of claim 16, further comprising changing the state to a trained background after a predetermined number of two frames.

18. The method of claim 1, wherein the video sequence is recorded with a video camera and wherein the method further comprises:

detecting a motion of the video camera; and

compensating for the motion of the video camera.

19. The method of claim 18, detecting the motion comprises:

selecting a frame feature; and

tracking the frame features over the plurality of frames.

20. The method of claim 19, wherein compensating comprises resetting the background model when the motion has stopped.

21. A machine readable medium having stored thereon instructions/which when executed by a processor, cause the processor to perform the following:

localizing a moving object in a video sequence based on a change in the moving object over a plurality of frames of the video sequence/the moving object occupying frame areas of changing color; and

training a background model for the plurality of frames outside of the frame areas of changing color.

22. The machine readable medium of claim 21, wherein localizing comprises localizing the moving object using a change detection mask.

23. The machine readable medium of claim 21, wherein localizing comprises:

determining a boundary for the moving object that is of homogenous color; and

constructing a hull around the moving object using the boundary.

24. The machine readable medium of claim 23, wherein determining a boundary comprises:

determining a maximum contour of a plurality of contours of the moving object, the maximum contour having the largest area of the plurality of contours;

determining other contours of the moving object; and

joining the maximum contour with the other contours.

25. The machine readable medium of claim 24, wherein the processor further performs:

determining a smallest contour of the plurality of contours; and

eliminating the smallest contour from joining with the maximum contour.

26. The machine readable medium of claim 24, wherein joining comprises joining one of the other contours with the maximum contour if the distance between the maximum contour and the one of the other contours is less than a predetermined distance.

27. The machine readable medium of claim 23, wherein the processor performing constructing the hull comprises the processor performing:

determining all connected components in the boundary, wherein each of the components has a contour having an area;

filtering out a smallest area contour;

selecting a maximum area contour; and

joining the maximum area contour with other contours of the connected components.

28. The machine readable medium of claim 21, wherein the frames comprise a plurality of pixels and wherein the processor performing training, comprises the processor performing characterizing a pixel color at a given time with a value based on a state, each pixel corresponding to a state of a plurality of states.

29. The machine readable medium of claim 28, wherein the processor performing training comprises the processor performing:

training the background model for the pixel in a foreground; and

changing the state of the pixel to an untrained background if the pixel represents a static behavior for a certain period of time.

30. The machine readable medium of claim 21, wherein the video sequence is recorded with a video camera and wherein the processor further performs:

detecting a motion of the video camera; and

compensating for the motion of the video camera.

31. The machine readable medium of claim 30, wherein the processor performing detecting the motion comprises the processor performing the following:

selecting a frame feature; and

tracking the frame features over the plurality of frames.

32. The machine readable medium of claim 30, wherein the processor performing compensating comprises the processor performing the following:

resetting the background model when the motion has stopped.

33. An apparatus for extracting a background image, comprising:

means for localizing a moving object in a video sequence based on a change in the moving object over a plurality of frames of the video sequence, the moving object occupying frame areas of changing color; and

means for training a background model for the plurality of frames outside of the frame areas of changing color.

34. The apparatus of claim 33, wherein the means for localizing comprises:

means for determining a boundary for the moving object that is of homogenous color; and

means for constructing a hull around the moving object using the boundary.

35. The apparatus of claim 33, wherein the video sequence is recorded with a video camera and wherein the apparatus further comprises:

means for detecting a motion of the video camera; and

means for compensating for the motion of the video camera.

36. An apparatus for extracting a background image, comprising:

a processor to execute one or more routines to localize a moving object in a video sequence based on a change in the moving object over a plurality of frames of the video sequence, the moving object occupying frame areas of changing color, and to train a background model for the plurality of frames outside of the frame areas of changing color; and

a storage device coupled with the processor, the storage device having stored therein the one or more routines to localize the moving object and train the background model.

37. The apparatus of claim 36, wherein the processor executes one or more routines to localize the moving object using a change detection mask.

38. The apparatus of claim 36, wherein the processor executes one or more routines to determine a boundary for the moving object that is of homogenous color and to construct a hull around the moving object using the boundary.

39. The apparatus of claim 36, further comprising a display coupled with the processor to display the plurality of frames of the video sequence.

40. The apparatus of claim 36, further comprising a camera coupled with the processor to record the plurality of frames of the video sequence.

41. The apparatus of claim 40, wherein the processor executes one or more routines to detect a motion of the video camera to compensate for the motion of the video camera.