Intelligent processing method and system for video data

• US 10,178,430 B2
• Filed: 12/09/2015
• Issued: 01/08/2019
• Est. Priority Date: 01/26/2015
• Status: Active Grant

• Current Assignee: Hangzhou Hikvision Digital Technology Company Limited
• Original Assignee: Hangzhou Hikvision Digital Technology Company Limited
• Inventors: Wang, Wei, Lin, Qiqian, Wang, Weichun
• Primary Examiner(s): Salce, Jason P
• Application Number:
  US15/537,462
• Publication Number:
  US 20180007429A1
• Time in Patent Office: 1,126 Days
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 21/24 Reminder alarms, e.g. anti-loss alarms
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23614 Multiplexing of additional data and video streams arrangements for simultaneous broadcast of plural pieces of information H04H20/28
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4408 involving video stream encryption, e.g. re-encrypting a decrypted video stream for redistribution in a home network
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/681 Motion detection
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/9201 involving the multiplexing of an additional signal and the video signal
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The present application discloses an intelligent processing method and system for video data, wherein, in the method an intelligent camera set a warning rule, the method comprises: the intelligent camera collecting video data and analyzing the collected video data in real time, generating intelligent data if the warning rule is met, which intelligent data contain an encoder identifier and motion trajectory information; the intelligent camera packaging the video data and the intelligent data into a program stream and sending it to a frame analyzing component in a cloud storage system; the frame analyzing component unpacking the received program stream to obtain the video data and the intelligent data, and storing the video data and the intelligent data in storage components respectively; the storage components sending storage address information of the video data and the intelligent data to an index server for recording respectively. The solutions of the present application can perform intelligent processing for the collected video data in real time.

Claims

1. An intelligent processing method for video data, wherein in the case that an intelligent camera sets a warning rule, the method comprises:

the intelligent camera collecting video data and analyzing the collected video data in real time, generating intelligent data if the warning rule is met, wherein intelligent data contains an encoder identifier and motion trajectory information;

the intelligent camera packaging the video data and the intelligent data into a program stream and sending it to a frame analyzing component in a cloud storage system;

the frame analyzing component unpacking the received program stream to obtain the video data and the intelligent data, and storing the video data and the intelligent data in storage components respectively;

the storage components sending storage address information of the video data and the intelligent data to an index server for recording respectively;

wherein the video data and the intelligent data are stored in respective storage components in a cloud storage system separately in the form of data blocks, usage states of respective data blocks in all the storage components are stored in the index server;

when data need to be written in, the storage components firstly request the index server for free data blocks, the index server selects storage components according to distributed storage strategies, chooses suitable free data blocks in the corresponding storage components and then feeds address information of the chosen data blocks back to the frame analyzing component;

the frame analyzing component writes the video data and the intelligent data into the corresponding data blocks respectively according to the address information of the data blocks;

wherein, the storage address information recorded by the index server comprises an encoder identifier, address information, and a time point, the method further comprising;

a platform server receiving a playback request and sending it to the index server, the playback request comprising an encoder identifier, a playback time range;

the index server obtaining, by means of a query, storage address information of corresponding video data according to the playback request, sending an extracting request to a corresponding storage component according to the storage address information;

the storage component reading the corresponding video data according to the extracting request and sending it to the platform server;

the platform server playing the video data, receiving a retrieval task during the playing, sending the retrieval task to the index server, the retrieval task comprising an encoder identifier, a retrieval time range and a retrieval rule;

the index server obtaining, by means of a query, storage address information of corresponding intelligent data according to the encoder identifier and the retrieval time range in the retrieval task and sending a retrieval request to the corresponding storage component according to the storage address information;

the storage component receiving the retrieval request and reading the corresponding intelligent data;

invoking a computing component to determine the time point meeting the retrieval rule according to the read intelligent data;

the storage component converting the extracted time point to a time slice comprising this time point and feeding the time slice back to the platform server;

the platform server playing the video data corresponding to the time slice.

2. The method as claimed in claim 1, wherein, the warning rule comprises at least one of target motion information and target feature information, the target motion information comprising location range information and motion variation information.

3. The method as claimed in claim 1, wherein, the retrieval rule comprises target motion information that comprises location range information and motion variation information;

the platform server receiving a retrieval task during the playing comprises;

during the playing, the platform server receiving location range information drawn on a paused image by a user; and

receiving a motion variation rule input on an interface for setting rules.

4. The method as claimed in claim 1, wherein, the retrieval task further comprises the target feature information;

the platform server receiving a retrieval task during the playing comprises, during the playing, receiving the target feature information input by a user on an interface for setting rules.

5. The method as claimed in claim 1, wherein, the generated intelligent data further comprises relative time that is the difference between the time of generating the intelligent data and the time of collecting the corresponding video data;

when storing the intelligent data, the storage component further stamps the stored intelligent data with a local timestamp of storage;

determining the time point meeting the retrieval rule according to the read intelligent data comprises;

determining the intelligent data meeting the retrieval rule from the read intelligent data;

extracting the relative time and the local timestamp from the determined intelligent data and adding the relative time to the local timestamp to obtain an absolute time point which is the determined time point meeting the retrieval rule.

6. The method as claimed in claim 1, wherein, the method further comprises:

the intelligent camera performing statistics on traffic information periodically to obtain traffic parameters;

the intelligent camera further packaging the traffic parameters into the program stream;

when unpacking the received program stream, the frame analyzing component also obtaining the traffic parameters and storing the traffic parameters in the storage components;

the storage components further sending storage address information of the traffic parameters to the index server for recording.

7. The method as claimed in claim 6, wherein the storage address information recorded by the index server comprises an encoder identifier, address information, and a time point, the method further comprising:

the platform server receiving a traffic parameter request and sending it to the index server, the traffic parameter request comprising an encoder identifier, a time range and a traffic retrieval rule;

the index server obtaining, by means of a query, storage address information of a corresponding traffic parameter according to the encoder identifier and the time range in the traffic parameter request and sending the traffic parameter request to a corresponding storage component according to the storage address information;

the storage component receiving the traffic parameter request and reading the corresponding traffic parameter;

invoking a computing component to determine a time point meeting the traffic retrieval rule according to the read traffic parameter;

the storage component converting the extracted time point to a time slice comprising this time point and feeding the time slice back to the platform server;

the platform server playing the video data corresponding to the time slice.

8. An intelligent processing system for video data, wherein the system comprises an intelligent camera, a frame analyzing component, an index server and a plurality of storage components;

the intelligent camera sets a warning rule;

the intelligent camera is used for collecting video data and analyzes the collected video data in real time, generating intelligent data that contain an encoder identifier and motion trajectory information if the warning rule is met, packaging the video data and the intelligent data into a program stream and sending it to the frame analyzing component;

the frame analyzing component is used for unpacking the received program stream to obtain the video data and the intelligent data, and storing the video data and the intelligent data in the storage components respectively;

the storage components are used for sending storage address information of the video data and the intelligent data to the index server after the video data and the intelligent data are stored;

the index server is used for recording the received storage address information of the video data and the intelligent data respectively;

wherein the video data and the intelligent data are stored in respective storage components in a cloud storage system separately in the form of data blocks, usage states of respective data blocks in all the storage components are stored in the index server;

when data need to be written in, the storage components firstly request the index server for free data blocks, the index server selects storage components according to distributed storage strategies, chooses suitable free data blocks in the corresponding storage components and then feeds address information of the chosen data blocks back to the frame analyzing component;

the frame analyzing component writes the video data and the intelligent data into the corresponding data blocks respectively according to the address information of the data blocks;

wherein the storage address information recorded by the index server comprises an encoder identifier, address information, and a time point;

the system further comprises a platform server which is used for receiving a playback request and sending it to the index server, the playback request comprising an encoder identifier, a playback time range;

the platform server is further used for receiving video data fed back by a corresponding storage component, playing the video data, receiving a retrieval task during the playing, sending the retrieval task to the index server, the retrieval task comprising an encoder identifier, a retrieval time range and a retrieval rule;

the platform server is further used for receiving a time slice fed back by a corresponding storage component and playing the video data corresponding to the time slice;

the index server is used for obtaining, by means of a query, storage address information of corresponding video data according to the received playback request, and sending an extracting request to a corresponding storage component according to the storage address information;

it is further used for receiving the retrieval task from the platform server, obtaining, by means of a query, storage address information of corresponding intelligent data according to the encoder identifier and the retrieval time range in the retrieval task, and sending the retrieval task to a corresponding storage component according to the storage address information;

the storage component is used for receiving the playback request from the index server, reading corresponding video data according to the playback request, and sends it to the platform server;

it is further used for receiving the retrieval task from the index server, reading the intelligent data corresponding to the encoder identifier and the retrieval time range in the retrieval task;

determining a time point meeting the retrieval rule according to the read intelligent data;

converting the extracted time point to a time slice comprising this time point and feeding the time slice back to the platform server.

9. The system as claimed in claim 8, wherein, the platform server is used for, during the playing, receiving location range information drawn on a paused image by a user;

and receiving a motion variation rule input on an interface for setting rules.

10. The system as claimed in claim 8, wherein, the intelligent camera is further used for obtaining relative time and incorporating the relative time into the intelligent data, the relative time being the difference between the time of generating the intelligent data and the time of collecting the corresponding video data;

the storage components are further used for, when storing the intelligent data, stamping the stored intelligent data with a local timestamp of storage;

during the determination of a time point meeting the retrieval rule according to the read intelligent data, determining the intelligent data meeting the retrieval rule from the read intelligent data;

extracting the relative time and the local timestamp from the determined intelligent data, and obtaining an absolute time point by adding the relative time to the local timestamp, the absolute time point being the determined time point meeting the retrieval rule.

11. The system as claimed in claim 8, wherein,

the intelligent camera is further used for performing statistics on traffic information periodically to obtain traffic parameters, and packaging the traffic parameters into the program stream;

the frame analyzing component is further used for, when unpacking the received program stream, obtaining the traffic parameters and storing the traffic parameters in the storage components;

the storage components are further used for sending storage address information of the traffic parameters to the index server;

the index server is used for recording the received storage address information related to the traffic parameters.

12. The system as claimed in claim 11, wherein, the storage address information recorded by the index server comprises an encoder identifier, address information, and a time point,

the system further comprises a platform server which is used for receiving a traffic parameter request and sending it to the index server, the traffic parameter request comprising an encoder identifier, a time range and a traffic retrieval rule;

the platform server is further used for receiving a time slice fed back by a storage component and playing the video data corresponding to the time slice;

the index server is used for obtaining, by means of a query, storage address information of a corresponding traffic parameter according to the encoder identifier and the time range in the traffic parameter request and sending the traffic parameter request to a corresponding storage component according to the storage address information;

the storage component is used for receiving the traffic parameter request, reading the corresponding traffic parameter, invoking a computing component to determine a time point meeting the traffic retrieval rule according to the read traffic parameter;

converting the extracted time point to a time slice comprising this time point and feeding the time slice back to the platform server.