VIDEO SEGMENTATION USING STATISTICAL PIXEL MODELING

• US 20130242095A1
• Filed: 05/03/2013
• Published: 09/19/2013
• Est. Priority Date: 03/23/2001
• Status: Active Grant

• Current Assignee: Avigilon Fortress Corporation
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Lipton, Alan J., Haering, Niels, Rasheed, Zeeshan, Zhang, Zhong, Yin, Weihong, Venetianer, Peter L., Myers, Gary W., Javed, Omar
• Granted Patent: US 9,020,261 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method for segmenting video data into foreground and background portions utilizes statistical modeling of the pixels. A statistical model of the background is built for each pixel, and each pixel in an incoming video frame is compared with the background statistical model for that pixel. Pixels are determined to be foreground or background based on the comparisons. The method for segmenting video data may be further incorporated into a method for implementing an intelligent video surveillance system. The method for segmenting video data may be implemented in hardware.

Claims

1. A method to perform a two-pass method of video segmentation for differentiating between foreground and background portions of video, comprising:

obtaining, by circuitry, a frame sequence from an input video stream;

executing, by the circuitry, a first-pass method for each frame of the frame sequence, the first-pass method comprising the steps of;

aligning the frame with a scene model; and

updating a background statistical model; and

finalizing the background statistical model;

executing, by the circuitry, a second-pass method for each frame of the frame sequence, the second-pass method comprising the steps of;

labeling each region of the frame; and

performing spatial/temporal filtering of the regions of the frame.

2. The method according to claim 1, wherein the step of updating a background statistical model comprises the steps of:

compiling values corresponding to the regions of the frames; and

computing a standard deviation for each region of the frames.

3. The method according to claim 1, wherein the step of finalizing the background statistical model comprises the step of:

for each region of the frames, determining a statistical mode based on the compiled values; and

assigning the statistical mode of each region as its value in the background statistical model.

4. The method according to claim 1, wherein the step of labeling each region of the frame comprises the steps of:

determining whether or not the region matches the background statistical model; and

assigning a label based on the result of the determining step.

5. The method according to claim 4, wherein the step of determining comprises the step of determining whether a value of the region of the frame falls within some range about a corresponding value of the region in the background statistical model;

and

wherein the step of assigning a label comprises the steps of;

assigning a first label if the value of the region of the frame does fall within the range; and

assigning a second label if the value of the region of the frame does not fall within the range.

6. The method according to claim 5, wherein said first label is background and said second label is foreground.

7. The method according to claim 4, wherein the step of determining comprises the steps of:

determining whether a value of the region of the frame falls within a first range about a corresponding value of the region in the background statistical model; and

if the value of the region of the frame does not fall within the first range, determining if it falls within a second range about said corresponding value of the region in the background statistical model; and

wherein the step of assigning a label comprises the steps of;

assigning a first label if the value of the region of the frame falls within the first range;

assigning a second label if the value of the region of the frame does not fall within the first range and does lie within the second range; and

assigning a third label if the value of the region of the frame does not fall within the second range.

8. The method according to claim 7, wherein said first label is background, said second label is foreground, and said third label is definite foreground.

9. The method according to claim 1, wherein the step of labeling each region of the frame comprises the steps of:

determining whether or not the region matches the background statistical model, comprising the step of performing determinations as to whether or not a value of the region of the frame falls within each of an arbitrary number of ranges about a corresponding value of the region in the background statistical model; and

assigning a label based on the result of the determining step.

10. The method according to claim 1, wherein the step of spatial/temporal filtering comprises the steps of, for each region of the frame:

performing neighborhood filtering; and

performing spatial filtering, temporal filtering, or a combination thereof.

11. The method according to claim 10, wherein the step of performing spatial filtering, temporal filtering, or a combination thereof comprises the step of performing morphological erosions and dilations, wherein the morphological erosions and dilations are performed in two dimensions or in three dimensions.

12. The method according to claim 11, wherein said step of performing morphological erosions and dilations comprises the steps of:

performing a number, n, of erosions, wherein any region labeled definite foreground is not eroded;

following the step of performing n erosions, performing n dilations; and

following the step of performing n dilations, performing a second number, m, of dilations.

13. The method according to claim 10, wherein the step of performing neighborhood filtering comprises the steps of, for each region of the frame, P_i, not labeled background:

determining a location, P_m, in the scene model corresponding to P_i;

determining a neighborhood of regions, P′

_m around P_m;

determining if any of the P′

_m matches P_i;

if any of the P′

_m matches P_i, determining a neighborhood of regions P′

_i around P_i;

determining if any of the P′

_i matches P_m; and

if any of the P′

_m matches P_i, relabeling P_i background.

14. A system adapted to perform a one-pass method of video segmentation for differentiating between foreground and background portions of video comprising:

circuitry to obtain a frame sequence from a video stream and for each frame in the frame sequence, performing the following steps;

aligning the frame with a scene model;

building a background statistical model;

labeling the regions of the frame; and

performing spatial/temporal filtering.

15. The system according to claim 14, wherein the step of spatial/temporal filtering comprises the steps of, for each region of the frame:

performing neighborhood filtering; and

performing spatial filtering, temporal filtering, or a combination thereof.

16. The system according to claim 15, wherein the step of performing spatial filtering, temporal filtering, or a combination thereof comprises the step of performing morphological erosions and dilations, wherein the morphological erosions and dilations are performed in two dimensions or in three dimensions.

17. The system according to claim 16, wherein said step of performing morphological erosions and dilations comprises the steps of:

performing a number, n, of erosions, wherein any region labeled definite foreground is not eroded;

following the step of performing n erosions, performing n dilations; and

following the step of performing n dilations, performing a second number, m, of dilations.

18. The system according to claim 15, wherein the step of performing neighborhood filtering comprises the steps of, for each region of the frame, P_i, not labeled background:

determining a location, P_m, in the scene model corresponding to P_i;

determining a neighborhood of regions, P′

_m around P_m;

determining if any of the P′

_m matches P_i;

if any of the P′

_m matches P_i, determining a neighborhood of regions P′

_i around P_i;

determining if any of the P′

_i matches P_m; and

if any of the P′

_m matches P_i, relabeling P_i background.

19. The system according to claim 14, wherein said step of building a background statistical model comprises the steps of:

for each region of the frame, performing the steps of;

determining if the background statistical model is mature and stable;

if the background statistical model is mature and stable, determining if a value of the region matches a corresponding value of the background statistical model; and

if the value of the region matches the value of the background statistical model, updating the background statistical model using the value of the region.

20. A non-transitory computer-readable medium comprising instructions, which when executed by a circuitry causes the circuitry to perform a one-pass method of video segmentation for differentiating between foreground and background portions of video, the computer-readable medium comprising instructions for:

obtaining a frame sequence from a video stream; and

for each frame in the frame sequence, performing the following steps;

aligning the frame with a scene model;

building a background statistical model and a secondary statistical model;

labeling the regions of the frame; and

performing spatial/temporal filtering.