Method and system for flow detection and motion analysis

• US 7,660,439 B1
• Filed: 12/16/2004
• Issued: 02/09/2010
• Est. Priority Date: 12/16/2003
• Status: Expired due to Fees

• Current Assignee: Verification Corporation
• Original Assignee: Verification Corporation
• Inventors: Wolf, Wayne, Ozer, Ibrahim Burak, Lu, Tiehan
• Primary Examiner(s): Ahmed; Samir A.
• Assistant Examiner(s): RASHIDIAN, MOHAMMAD M
• Application Number:
  US11/014,545
• Time in Patent Office: 1,881 Days
• Field of Search: 382/103, 382/107, 382/236, 382168-179, 382/199, 248154-155, 386/28, 386/27, 734/88, 735/51, 348153-169, 356 27- 29
• US Class Current: 382/107
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

A surveillance system receives video data of a monitored scene and identifies flows of motion in the scene. Specifically, a video camera communicates with a video recognition system, which identifies a flow motion region of a monitored scene and determines characteristics of the identified flow motion region.

Claims

1. A method for identifying a flow motion region and one or more flow motion region characteristics, the method comprising the steps of:

receiving a video input comprising a plurality of frames;

identifying motion vectors for each of the plurality of frames;

identifying a predicted flow motion region in at least one of the plurality of frames based at least in part on the motion vectors of the frame;

temporally filtering the predicted flow motion region, wherein said temporally filtering comprising distinguishing between a non-flow motion and a flow motion in the predicted flow motion region, wherein said non-flow motion is a motion of non-interest and flow motion is a motion of interest;

generating a mask based at least in part on the temporal filtered predicted flow motion region;

applying the mask to a frame comprising the motion vectors to obtain an identified flow motion region; and

determining one or more characteristics of the identified flow motion region.

2. The method of claim 1, further comprising the step of refining the predicted flow motion region by reducing an effect of predicted non-flow motion prior to the step of generating the mask.

3. The method of claim 1, further comprising the step of refining the identified flow motion region based at least in part on magnitudes of the motion vectors in the identified flow motion region.

4. The method of claim 1, wherein the motion vectors are identified using a block motion estimation method.

5. The method of claim 1, wherein the step of identifying the predicted flow motion region includes reducing an effect of video-input motion.

6. The method of claim 5, wherein the video-input motion is determined using a camera motion estimation method.

7. The method of claim 2, wherein the step of refining the predicted flow motion region includes eliminating motion vectors having magnitudes less than a predetermined threshold.

8. The method of claim 2, wherein the step of refining the predicted flow motion region reducing includes temporally filtering the motion vectors of two or more frames.

9. The method of claim 2, wherein the step of refining the predicted flow motion region includes labeling the predicted flow motion region with one or more motion labels.

10. The method of claim 1, further comprising the step of refining the motion vectors for each of the plurality of frames prior to the step of applying the mask.

11. The method of claim 10, wherein the step of refining the motion vectors includes temporally filtering the motion vectors of two or more frames.

12. The method of claim 3, wherein the step of refining the identified flow motion region includes calculating an average and a standard deviation of the magnitudes of the motion vectors in the identified flow motion region to produce an adaptive threshold, which is a linear combination of the average and standard deviation.

13. The method of claim 12, wherein the step of refining the identified flow motion region includes eliminating motion vectors in the identified flow motion region having magnitudes less than the adaptive threshold.

14. The method of claim 1, further comprising the step of fitting a shape to the identified flow motion region.

15. The method of claim 14, wherein the shape includes an ellipse.

16. The method of claim 15, further comprising the step of refining the ellipse-fitted identified flow motion region.

17. The method of claim 16, wherein the step of refining the ellipse-fitted identified flow motion region includes temporally filtering motion vectors thereof.

18. The method of claim 17, further comprising the step of analyzing the refined ellipse-fitted identified flow motion region to determine characteristics thereof.

19. The method of claim 1, furthering comprising the step of generating an alarm based at least in part on the identified flow motion region.

20. The method of claim 19, wherein the alarm is generated when an area of the identified flow motion region is greater than or equal to an alarm threshold.

21. The method of claim 1 wherein the determined characteristic(s) includes a direction of the identified flow motion region.

22. The method of claim 21, wherein the direction of the identified flow motion region is determined by calculating an average of the motion vectors of the identified flow motion region.

23. A system for identifying a flow motion region and one or more flow motion region characteristics, comprising:

a video camera; and

a computer communicatively connected to the video camera, wherein the computer is programmed to;

receive a video input comprising a plurality of frames;

identify motion vectors for each of the plurality of frames;

identify a predicted flow motion region in at least one of the plurality of frames based at least in part on the motion vectors of the frame,

temporal filter the predicted flow motion region to distinguish between a non-flow motion and a flow motion, wherein said non-flow motion is a motion of non-interest and flow motion is a motion of interest;

generate a mask based at least in part on the temporal filtered predicted flow motion region;

apply the mask to a frame comprising the motion vectors to produce an identified flow motion region; and

determine one or more characteristics of the identified flow motion region.

24. The system of claim 23, wherein the computer is programmed to refine the predicted flow motion region by reducing an effect of predicted non-flow motion prior to generation of the mask.

25. The system of claim 23, wherein the computer is programmed to refine the identified flow motion region based at least in part on magnitudes of the motion vectors in the identified flow motion region prior to determination of the one or more characteristics.

26. A computer-implemented method for identifying a flow motion region and one or more flow motion region characteristics, the method comprising the steps of:

receiving a video input comprising a plurality of frames;

identifying motion vectors for each of the plurality of frames;

identifying a predicted flow motion region in at least one of the plurality of frames based at least in part on the motion vectors of the frame;

temporally filtering the predicted flow motion region, wherein said temporally filtering comprising distinguishing between a non-flow motion and a flow motion in the predicted flow motion region, wherein said non-flow motion is a motion of non-interest and flow motion is a motion of interest;

generating a mask based at least in part on the temporal filtered predicted flow motion region;

applying the mask to a frame comprising the motion vectors to produce an identified flow motion region; and

determining one or more characteristics of the identified flow motion region.

27. The method of claim 26, further comprising the step of refining the predicted flow motion region by reducing an effect of predicted non-flow motion prior to the step of generating the mask.

28. The method of claim 26, further comprising the step of refining the identified flow motion region based at least in part on magnitudes of the motion vectors in the identified flow motion region prior to the step of determining the one or more characteristics.

29. A computer-readable storage medium storing computer code for implementing a method for identifying a flow motion region and one or more flow motion region characteristics, wherein the method comprises the steps of:

receiving a video input comprising a plurality of frames;

identifying motion vectors for each of the plurality of frames;

identifying a predicted flow motion region in at least one of the plurality of frames based at least in part on the motion vectors of the frame;

temporally filtering the predicted flow motion region, wherein said temporally filtering comprising distinguishing between a non-flow motion and a flow motion in the predicted flow motion region, wherein said non-flow motion is a motion of non-interest and flow motion is a motion of interest;

generating a mask based at least in part on the temporal filtered predicted flow motion region;

applying the mask to a frame comprising the motion vectors to produce an identified flow motion region; and

determining one or more characteristics of the identified flow motion region.

30. The computer-readable storage medium of claim 29, wherein the method further comprises the step of refining the predicted flow motion region by reducing an effect of predicted non-flow motion prior to generation of the mask.

31. The computer-readable storage medium of claim 29, wherein the method further comprises the step of refining the identified flow motion region based at least in part on magnitudes of the motion vectors in the identified flow motion region prior to determination of the one or more characteristics.