Foreground detector for video analytics system

• US 10,303,955 B2
• Filed: 04/28/2017
• Issued: 05/28/2019
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Intellective Ai, Inc.
• Original Assignee: Omni International, LLC (Blue Sail Medical Co., Ltd.)
• Inventors: Saitwal, Kishor Adinath, Risinger, Lon W., Cobb, Wesley Kenneth
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/582,524
• Publication Number:
  US 20180082130A1
• Time in Patent Office: 760 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

Techniques are disclosed for creating a background model of a scene using both a pixel based approach and a context based approach. The combined approach provides an effective technique for segmenting scene foreground from background in frames of a video stream. Further, this approach can scale to process large numbers of camera feeds simultaneously, e.g., using parallel processing architectures, while still generating an accurate background model. Further, using both a pixel based approach and context based approach ensures that the video analytics system can effectively and efficiently respond to changes in a scene, without overly increasing computational complexity. In addition, techniques are disclosed for updating the background model, from frame-to-frame, by absorbing foreground pixels into the background model via an absorption window, and dynamically updating background/foreground thresholds.

Claims

1. A processor-implemented method, comprising:

generating, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including;

classifying each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;

performing at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and

analyzing, via the at least one processor, at least one new video frame based on the background model.

2. The method of claim 1, wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.

3. The method of claim 1, wherein the at least one contextual evaluation includes dilating each pixel classified as a foreground pixel.

4. The method of claim 1, wherein the at least one contextual evaluation includes eroding each pixel classified as a foreground pixel.

5. The method of claim 1, wherein the classifying of each pixel of the plurality of pixels is based on a distance between that pixel and a pixel of the background model.

6. The method of claim 1, wherein the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model.

7. The method of claim 1, wherein:

the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and

as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold.

8. The method of claim 1, wherein:

the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and

as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold,

the method further comprising dynamically updating the threshold.

9. The method of claim 1, wherein:

the classifying of each pixel of the plurality of pixels is based on a Mahalanobis distance between that pixel and a pixel of the background model; and

as a result of the classifying, the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold,

the method further comprising dynamically updating the threshold based on a camera noise model.

10. A system, comprising:

a processor; and

a memory operably coupled to the processor and storing processor-executable instructions to cause the processor to;

generate, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including;

classify each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;

perform at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and

analyze, via the at least one processor, at least one new video frame based on the background model.

11. The system of claim 10, wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.

12. The system of claim 10, wherein the at least one contextual evaluation includes one of dilating or eroding each pixel classified as a foreground pixel.

13. The system of claim 10, wherein the memory stores processor-executable instructions to cause the processor to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model.

14. The system of claim 10, wherein the memory stores processor-executable instructions to cause the processor to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model, such that the pixel is classified as a foreground pixel if the distance exceeds a pre-determined threshold.

15. A non-transitory, processor-readable medium storing processor-executable instructions to:

generate, via at least one processor, a background model of a scene depicted in a sequence of video frames, the generating including;

classify each pixel of a plurality of pixels from an at least one video frame as a foreground pixel or a background pixel, the classification of each pixel based on comparing an at least one appearance value of a pixel each to a background model of the scene;

perform at least one contextual evaluation on at least one pixel classified as depicting foreground, the contextual evaluation providing selective pixel reclassification of at least one pixel as a foreground pixel or background pixel based on the classification of one or more other pixels of the plurality of pixels in the at least one video frame as a foreground pixel or a background pixel; and

analyze, via the at least one processor, at least one new video frame based on the background model.

16. The processor-readable medium of claim 15, wherein the at least one contextual evaluation includes performing a series of morphological operations on each pixel classified as a foreground pixel.

17. The processor-readable medium of claim 15, wherein the at least one contextual evaluation includes one of dilating or eroding each pixel classified as a foreground pixel.

18. The processor-readable medium of claim 15, wherein the medium stores processor-executable instructions to classify each pixel of the plurality of pixels based on a distance between that pixel and a pixel of the background model.

19. The processor-readable medium of claim 15 the medium stores processor-executable instructions to classify each pixel as a foreground pixel if the distance exceeds a pre-determined threshold.