Large-scale processing and querying for real-time surveillance

• US 9,792,334 B2
• Filed: 09/25/2014
• Issued: 10/17/2017
• Est. Priority Date: 09/25/2014
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Wang, Mengjiao, Li, Wen-Syan
• Primary Examiner(s): LE, UYEN T
• Application Number:
  US14/496,990
• Publication Number:
  US 20160092452A1
• Time in Patent Office: 1,118 Days
• Field of Search: None
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G08B 13/19608 Tracking movement of a target, e.g. by detecting an object predefined as a target, using target direction and or velocity to predict its new position
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19671 Addition of non-video data, i.e. metadata, to video stream
• CPC Class Codes: G08G 1/0175 by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/054 photographing overspeeding vehicles

Abstract

A priority request handler may receive a priority request to associate a priority label with a sensed data file, the sensed data file being included within a time series of sensed data files received from a plurality of sensors. A queue manager may then store the sensed data file within a priority queue in association with the priority label, and independently of a temporal order of a time series of sensed data files. The queue manager may thus retrieve the sensed data file from the priority queue, based on the priority label, for processing thereof to obtain a sensed data record.

Claims

1. A system comprising:

at least one processor; and

instructions recorded on a non-transitory computer-readable medium, and executable by the at least one processor, the system including

a surveillance management system configured to receive surveillance tracking data tracking movements of a plurality of sensed items moving individually past at least a subset of a network of a plurality of geographically-spaced motion sensors, to thereby provide a plurality of sensed data files in which each sensed data file corresponds to a motion capture of a sensed item of the sensed items, as captured by a corresponding motion sensor of the plurality of geographically-spaced motion sensors;

a priority request handler configured to receive a priority request from a user to associate a priority label with a sensed data file of the plurality of sensed data files, the sensed data file being included within a time series of sensed data files received from a the plurality of geographically-spaced motion sensors; and

a queue manager configured to store the sensed data file within a priority queue in association with the priority label, and independently of a temporal order of a the time series of sensed data files, and further configured to retrieve the sensed data file from the priority queue, based on the priority label, for processing thereof to thereby provide the user with a sensed data record in which metadata characterizing the motion capture of the sensed item is included.

2. The system of claim 1, wherein the priority request handler is configured to associate the priority label with the sensed data file, based on identification of a the sensed item represented within the sensed data file.

3. The system of claim 1, wherein the priority request handler is configured to associate the priority label with the sensed data file, based on historical sensed data records related to sensed items represented within corresponding sensed data files of the sensed data files.

4. The system of claim 1, wherein the priority request handler is configured to associate the priority label with the sensed data file including associating at least one of the plurality of geographically-spaced motion sensors with the priority label, based on a prediction of a sensing of the sensed item represented within the sensed data file by the at least one sensor.

5. The system of claim 1, wherein the priority queue includes a plurality of priority levels, and the priority label corresponds to one of the priority levels.

6. The system of claim 5, wherein a lowest-priority level of the plurality of priority levels is a default level for sensed data files not associated with any priority level, and wherein sensed data files at the default level are processed after higher-priority levels and in a time-ordered fashion.

7. The system of claim 1, wherein the sensed data file associated with the priority label is later-received than a lower priority sensed data file, but is processed to obtain the sensed data record prior to processing of the lower priority sensed data file.

8. The system of claim 1, wherein the queue manager is configured to distribute the sensed data file, along with a plurality of sensed data files obtained by the plurality of geographically-spaced motion sensors, among a plurality of load-balanced servers for processing thereof.

9. The system of claim 1, wherein the processing of the sensed data file to obtain the sensed data record includes analyzing the sensed data file to determine whether a real-time or near real-time alert is required, based on processing criteria.

10. The system of claim 1, wherein the queue manager is configured to store the sensed data record, along with a plurality of sensed data records, among a plurality of load-balanced storage units.

11. The system of claim 10, wherein the storage units are configured to store sensed data records using a key-value and hash based storage scheme, and wherein a key of the storage scheme is derived from a unique physical characteristic of the sensed item as included within sensed data of the sensed data record.

12. The system of claim 10, further comprising a query handler configured to receive a query and apply the query against sensed data records within the load-balanced storage units, to thereby obtain query results including the sensed data record.

13. A computer-implemented method for executing instructions stored on a non-transitory computer readable storage medium, the method comprising:

receiving surveillance tracking data tracking movements of a plurality of sensed items moving individually past at least a subset of a network of a plurality of geographically-spaced motion sensors, to thereby provide a plurality of sensed data files in which each sensed data file corresponds to a motion capture of a sensed item of the sensed items, as captured by a corresponding motion sensor of the plurality of geographically-spaced motion sensors;

storing the sensed data files in a time-ordered fashion within a priority queue;

receiving a priority request from a user to associate a priority label with a designated sensed data file of the plurality of sensed data files, for a designated item of the sensed items that is expected to be sensed by at least one of the plurality of geographically-spaced motion sensors;

receiving the designated sensed data file and relating the designated sensed data file with the priority label;

storing the designated sensed data file in the priority queue, ahead of the earlier-received, time-ordered sensed data files; and

retrieving the designated sensed data file for processing to thereby provide the user with a designated sensed data record in which metadata characterizing the motion capture of the sensed item is included, before retrieving the earlier-received, time-ordered sensed data files.

14. The method of claim 13, wherein the priority label is associated with the designated sensed data file including associating at least one of the plurality of sensors with the priority label, based on a prediction of a sensing of the designated item by the at least one sensor.

15. The method of claim 13, wherein the priority queue includes a plurality of priority levels, and the priority label corresponds to one of the priority levels.

16. A computer program product, the computer program product being tangibly embodied on a non-transitory computer-readable storage medium and comprising instructions that, when executed, are configured to cause at least one processor to:

receive surveillance tracking data tracking movements of a plurality of sensed items moving individually past at least a subset of a network of a plurality of geographically-spaced motion sensors, to thereby provide a plurality of sensed data files in which each sensed data file corresponds to a motion capture of a sensed item of the sensed items, as captured by a corresponding motion sensor of the plurality of geographically-spaced motion sensors;

receive a priority request from a user to associate a priority label with a sensed data file of the plurality of sensed data files, the sensed data file being included within a time series of sensed data files received from a the plurality of geographically-spaced motion sensors;

store the sensed data file within a priority queue in association with the priority label, and independently of a temporal order of a the time series of sensed data files;

retrieve the sensed data file from the priority queue, based on the priority label; and

process the sensed data file to thereby provide the user with a sensed data record in which metadata characterizing the motion capture of the sensed item is included.

17. The computer program product of claim 16, wherein the instructions, when executed, are configured to cause the at least one processor to:

process of the sensed data file to obtain the sensed data record including analyzing the sensed data file to determine whether a real-time or near real-time alert is required, based on processing criteria.

18. The computer program product of claim 16, wherein the sensed data files correspond to sensed items represented therein, and wherein the priority label is associated with the sensed data file based on expected inclusion of a designated item that is expected to be sensed by at least one of the plurality of sensors.

19. The computer program product of claim 16, wherein the instructions, when executed, are configured to cause the at least one processor to:

distribute the sensed data file, along with a plurality of sensed data files obtained by the plurality of sensors, among a plurality of load-balanced servers for processing thereof; and

store the sensed data record, along with a plurality of sensed data records corresponding to the plurality of sensed data files, among a plurality of load-balanced storage units.

20. The computer program product of claim 16, wherein the plurality of geographically-spaced motion sensors include a plurality of cameras and the sensed data files include image files, and further wherein the priority label is associated with the sensed image file based on inclusion of a designated sensed item therein.