Robust Motion Filtering for Real-time Video Surveillance

• US 20190057588A1
• Filed: 08/17/2018
• Published: 02/21/2019
• Est. Priority Date: 08/17/2017
• Status: Active Grant

• Current Assignee: Shanghai Hanshi Information Technology Company Limited (Hanshow Technology Co., Ltd.)
• Original Assignee: Bossa Nova Robotics IP, Inc.
• Inventors: Savvides, Marios, Lin, An Pang, Venugopalan, Shreyas, Thanikkal, Ajmal, Singh, Karanhaar, Adler, Gavriel, Neblett, Kyle
• Granted Patent: US 10,861,302 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19604 involving reference image or background adaptation with time to compensate for changing conditions, e.g. reference image update on detection of light level change
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/1961 Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image

Abstract

A video monitoring method that involves determining motion changes in a set of video frames to find potential objects is described. One or more bounding boxes are defined around the potential objects. These bounding boxes are spatially and temporally filtered to eliminate potential object candidates, with only potential objects in the bounding boxes remaining after filtering being classified or identified.

Claims

1. A video monitoring method, comprising the steps of:

determining motion changes in a set of video frames to find potential objects;

defining one or more bounding boxes around the potential objects;

spatially and temporally filtering the bounding boxes to eliminate potential objects; and

classifying potential objects in the remaining filtered bounding boxes.

2. The video monitoring method of claim 1, wherein the video frames are preprocessed to reduce at least one of color depth and resolution.

3. The video monitoring method of claim 1, wherein the bounding boxes are first spatially filtered and then temporally filtered.

4. The video monitoring method of claim 1, wherein determining motion changes further comprises frame subtraction and morphological processing.

5. The video monitoring method of claim 1, defining one or more bounding boxes further comprises fitting shape contours, fitting minimum bounding rectangles, and merging adjacent boxes.

6. The video monitoring method of claim 1, wherein spatially filtering the bounding boxes further comprises a bounding box shape check, filtering size, filtering aspect ratio, and filtering based on spatial location of the bounding box.

7. The video monitoring method of claim 1, wherein temporally filtering the bounding boxes further comprises object motion analysis and object tracking.

8. The video monitoring method of claim 1, wherein an identified region of interest bounding box has its contained potential object classified or identified using machine learning.

9. The video monitoring method of claim 1, further comprising use of a stationary camera to acquire the set of video frames.

10. The video monitoring method of claim 1, further comprising use of a movable camera to acquire the set of video frames.

12. The video monitoring method of claim 1, further comprising use of a camera mounted on an autonomous robot to acquire the set of video frames.

13. A video monitoring system for analysis of a set of video frames, comprising:

a motion module that determines motion changes in the set of video frames to find potential objects;

a bounding box module that defines one or more bounding boxes around the potential objects;

a filtering module that spatially and temporally filters the bounding boxes to eliminate potential objects; and

an object classifier that classifies potential objects in the remaining filtered bounding boxes.

14. The video monitoring system of claim 13, wherein the video frames are preprocessed to reduce at least one of color depth and resolution.

15. The video monitoring system of claim 13, wherein the bounding boxes are first spatially filtered and then temporally filtered by the filtering module.

16. The video monitoring system of claim 13, wherein the motion module further comprises frame subtraction and morphological processing.

17. The video monitoring system of claim 13, wherein defining one or more bounding boxes in the bounding box module further comprises fitting shape contours, fitting minimum bounding rectangles, and merging adjacent boxes.

18. The video monitoring system of claim 13, wherein spatially filtering the bounding boxes in the filtering module further comprises a bounding box shape check, filtering size, filtering aspect ratio, and filtering based on spatial location of the bounding box.

19. The video monitoring system of claim 13, wherein temporally filtering the bounding boxes in the filtering module further comprises object motion analysis and object tracking.

20. A video monitoring system mountable on an autonomous robot, comprising:

a camera system attached to an autonomous robot to provide a set of video frames,

a motion module that determines motion changes in the set of video frames to find potential objects;

a bounding box module that defines one or more bounding boxes around the potential objects;

a filtering module that spatially and temporally filters the bounding boxes to eliminate potential objects; and

an object classifier that classifies potential objects in the remaining filtered bounding boxes.