Video surveillance method based on object detection and system thereof

• US 10,657,783 B2
• Filed: 01/02/2019
• Issued: 05/19/2020
• Est. Priority Date: 06/29/2018
• Status: Active Grant

• Current Assignee: Hangzhou Eyecloud Technologies Co., Ltd.
• Original Assignee: Hangzhou Eyecloud Technologies Co., Ltd.
• Inventors: Yuan, Po, Pan, Shengjun, Zhao, Junneng, Mariniuc, Daniel
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US16/238,487
• Publication Number:
  US 20200005613A1
• Time in Patent Office: 503 Days
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G08B 13/19606 Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 25/08 using communication transmission lines G08B13/19658, G08B21/0286, G08B25/016 take precedence
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A video surveillance method includes the steps of detecting, by a motion detector, an object motion, in the field of view of a surveillance device when the surveillance device is in a standby mode; generating, responsive to a positive detection to switch the surveillance device from the standby mode to an operation mode, one or more images of the moving object; determining, by processing the one or more images with a deep neural network (DNN) model of an object detector, whether the objects contained in the one or more images belong to a given categories, wherein the DNN model comprises N (N is a positive integer and ranged from 4-12) depthwise separable convolution layers; and video recording, responsive to a positive determination, the moving object in the field of view of the surveillance device.

Claims

1. A video surveillance method based on object detection, comprising the steps of:

detecting, by a motion detector, an object motion, in the field of view of a surveillance device when the surveillance device is in a standby mode, wherein the object detector is integrated in the surveillance device;

generating, responsive to a positive detection to switch the surveillance device from the standby mode to an operation mode, one or more images of the moving object;

determining, by processing the one or more images with a deep neural network (DNN) model of an object detector, whether the objects contained in the one or more images belong to a given categories, wherein the DNN model comprises N (N is a positive integer and ranged from 4-12) depthwise separable convolution layers, wherein each depthwise separable convolution layer comprises a depthwise convolution layer for applying a single filter to each input channel and a pointwise layer for linearly combining the outputs of the depthwise convolution layer to obtain feature maps of the one or more images;

video recording, responsive to a positive determination, the moving object in the field of view of the surveillance device; and

generating an alert responsive to a positive determination;

wherein the step of determining whether the objects contained in the one or more images belong to a given categories;

comprises the steps of;

identifying different image regions between a first and a second image of the one or more images;

grouping the different image regions between the first image and the second image into one or more regions of interest (ROIs);

transforming the one or more ROIs into grayscale;

classifying, by processing the grayscale ROIs with a deep neural network (DNN) model, the objects contained in the one or more ROIs; and

determining whether the objects contained in the one or more ROIs belong to the given categories.

2. The video surveillance method, as recited in claim 1, wherein prior to identifying different image regions between a first and a second image of the one or more images, the method further comprises a step of transforming the second image to compensate for the physical movement of the surveillance device when capturing the first image and the second image.

3. The video surveillance method, as recited in claim 2, further comprising a step of outputting the recorded video.

4. The video surveillance method, as recited in claim 3, further comprising a step of receiving a control signal to selectively switch the surveillance device between the standby mode and the operation mode.

5. A video surveillance system based on object detection, comprising:

a motion detector configured to detect an object motion in the field of view of a surveillance device when the surveillance device is in a standby mode;

a mode switcher configured to switch, responsive to a positive detection, the surveillance device from the standby mode to an operation mode, wherein in the operation mode, one or more images of the moving object are generated by the surveillance device;

an object detector configured to process the one or more images with deep neural network (DNN) model of an object detector to determine whether the objects contained in the one or more images belong to a given categories, wherein the DNN model comprises N (N is a positive integer and ranged from 4-12) depthwise separable convolution layers, wherein each depthwise separable convolution layer comprises a depthwise convolution layer for applying a single filter to each input channel and a pointwise layer for linearly combining the outputs of the depthwise convolution layer to obtain feature maps of the one or more images; and

a video recorder configured to video record, responsive to a positive determination, the moving object in the field of view of the surveillance device, wherein the video recorder is further configured to generate an alert responsive to a positive determination;

wherein the object detector is further configured to;

identify different image regions between a first and a second image of the one or more images;

group the different image regions between the first image and the second image into one or more regions of interest (ROIs);

transform the one or more ROIs into grayscale;

classify, by processing the grayscale ROIs with a deep neural network (DNN) model, the objects contained in the one or more ROIs; and

determine whether the objects contained in the one or more ROIs belong to a given categories.

6. The video surveillance system, as recited in claim 5, wherein the object detector is further configured to, prior to identifying different image regions between a first and a second image of the one or more images, transform the second image to compensate for the physical movement of the surveillance device when capturing the first image and the second image.

7. The video surveillance system, as recited in claim 6, wherein the video recorder is further configured to output the recorded video.

8. The video surveillance system, as recited in claim 7, wherein the mode switcher is further configured to receive a control signal to selectively switch the surveillance device between the standby mode and the operation mode.

9. The video surveillance system, as recited in claim 7, wherein the object detector is further configured to:

determine whether the objects contained in the one or more images belong to particular objects of the given categories, wherein the DNN model of the object detector is trained to recognize particular objects of the given categories.

10. The video surveillance system, as recited in claim 9, wherein the given categories are set as human beings.

11. The video surveillance system, as recited in claim 10, further comprising a client system configured to receive the recorded video.

12. The video surveillance system, as recited in claim 11, wherein the client system is capable of generating a control signal for selectively switching the surveillance device between the standby mode and the operation mode.