High definition surveillance image storage optimization apparatus and methods of retention triggering

• US 20170048482A1
• Filed: 10/31/2016
• Published: 02/16/2017
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: Eagle Eye Networks, Inc.
• Original Assignee: Eagle Eye Networks, Inc.
• Inventors: Drako, Dean, Roskowski, Steven
• Granted Patent: US 10,341,684 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/625 using discrete cosine transform [DCT]
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/4335 Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

A high definition surveillance video storage optimization system executes purging directives according to policy and retention meta data. Segments of video are purged from storage when thresholds of retention metrics are applied and when policy permits. Retention metrics are determined from acoustic sound energy, radiation energy, access control events, and movement. Policies reflect calendars and elapsed time, percentage of available storage, statutory and judicial mandates, and utility of the stored segments. Extrinsic sensor data and operations are recorded into non-transitory retention meta data stores. Metrics are transformed into retention flags at either time of recordation or time of purgation. Security file storage includes metadata about the environment acquired by sensors include doors, audio noise, object identification and tracking and inputs which influence retention or purging. A circuit filters for motion and stores metadata per segment. Meta data enables content based purging of least consequential files.

Claims

1. An apparatus for high definition surveillance image storage optimization comprises:

a purging module;

a network interface to extrinsic sensors;

an extrinsic retention meta data store coupled to the purging module and further coupled to;

a video stream transformation module; and

a network interface to a plurality of high definition video cameras,

said modules, interfaces, and interconnected by communication circuits and switches.

2. The apparatus of claim 1 further comprises:

a control panel to enter purging directives; and

an interface to a non-transitory store of video segment files.

3. The apparatus of claim 1 further comprises:

a circuit to receive H.264 streams from high definition video cameras;

a circuit to segment H.264 streams into files and embed retention meta data in each header of a file.

4. The apparatus of claim 3 further comprises:

a circuit to determine that an extrinsic sensor measurement or event is of substantial significance to a surveillance system; and

,

a circuit to set a retention flag in an extrinsic retention meta data store.

5. The apparatus of claim 4 further comprises:

a circuit to decode movement vectors and low frequency coefficients in pixel blocks; and

a circuit to set a movement flag into retention meta data of a video segment file.

6. The apparatus of claim 5 further comprises:

a circuit to receive a purging directive; and

a circuit to remove pointers to video file segment files without a retention flag set.

7. The apparatus of claim 1 wherein said purging module comprises:

a retention policy store;

a statutory and judicial requirement store;

a metric threshold store;

a calendar module;

a camera and location ownership store;

an interface to disk-file management; and

a circuit to score retention meta data.

8. The apparatus of claim 1 wherein said extrinsic retention meta data store has, encoded in non-transitory media associated by time with video segments, sensor measurements selected from the groups:

portal transits;

physical access control requests and actuations;

occupancy of rooms, halls, and location space;

vibration and movement from accelerometers;

temperature and heat;

wind speed and humidity;

acoustical sound energy;

electromagnetic signals from motion, smoke, fire, CO detectors; and

perimeter or equipment intrusion alarms.

9. The apparatus of claim 1 wherein said video transformation module comprises:

a circuit to transform H.264 streams into video segments;

a circuit to annotate video segments with metrics of acoustic energy transitions;

a circuit to annotate video segments with metrics of visible and non-visible spectrum energy transitions;

a circuit to transform segments of H.264 video stream into metrics of movement vectors;

a circuit to transform segments of H.264 video stream into JPEG pixel blocks;

a circuit to transform frequency coefficients to movement metrics; and

a circuit to store said metrics into file headers.

10. A method for operation of a high definition video surveillance storage optimization apparatus comprises:

receiving a high definition video stream;

transforming the video stream into segments with retention meta data headers;

receiving extrinsic events and sensor measurements;

receiving retention policies and metric thresholds;

determining retention metrics and flags;

receiving a purging directive;

evaluating calendar and policy constraints on purging;

masking video file segments protected by retention meta data from purging; and

removing pointers to video file segments available to purging.

11. The method of claim 10 wherein receiving a high definition video stream comprises:

connecting to a plurality of high definition cameras;

downloading video streams in at least one of H.264 and JPEG coding;

reversing coding to obtain movement vectors or frequency bin coefficients; and

determining a metric of motion.

12. The method of claim 10 wherein transforming the video stream into segments with retention meta data headers 1420 comprises:

selecting blocks of time to segment a stream;

converting a stream into video segment files;

generating a plurality of low resolution preview frames;

storing locations of segment files and preview frames;

determining motion metrics from contents of segments and previews; and

storing retention meta data into tables and file headers.

13. The method of claim 10 wherein receiving extrinsic events and sensor measurements comprises at least one of the group:

receiving an indication of portal transit;

receiving an indication of area occupancy;

receiving a measurement of acoustic energy change;

receiving a measurement of vibration and movement;

receiving a measurement of radiation energy change;

receiving an access control request and actuation;

receiving a measurement of temperature or thermal change;

receiving a building or equipment intrusion alarm; and

receiving a chemical sensor measurement.

14. The method of claim 10 wherein receiving retention policies and metric thresholds comprises:

receiving a minimum time from recordation to retain video;

receiving a minimum time from most recent use to retain video;

receiving a minimum free space to trigger purging;

receiving a minimum measure of movement to flag retention;

receiving a periodic time to trigger purging;

receiving a statutory or judicial requirement for retention;

receiving a threshold of a sensor measurement for retention; and

receiving an extrinsic event to flag retention.

15. The method of claim 10 wherein determining retention metrics and flags comprises:

comparing low frequency coefficients with a threshold;

comparing changes in heat, light, sound, and vibration with a threshold;

comparing magnitude of motion vectors with a threshold;

comparing traffic and occupancy with norms by calendar; and

comparing extrinsic events with norms by time and day.

16. The method of claim 10 wherein receiving a purging directive comprises:

receiving a command from a user interface; and

receiving parameters on thresholds, dates, and locations.

17. The method of claim 10 wherein evaluating calendar and policy constraints on purging comprises:

determining that time since recordation has passed a retention threshold;

determining that time since last use has passed a retention threshold;

determining that percentage of free space has fallen below a threshold;

determining that location of the cameras is not within a retention zone;

determining that ownership of the cameras is not by a retention client; and

determining that there are no judicial or statutory constraints on purging video segments owned by the owner, during that time period, at the location of the camera.

18. The method of claim 10 wherein masking video file segments protected by retention meta data from purging 1480 comprises:

listing all video file segments older than minimum retention from recordation;

removing video file segments recently used within a retention window;

comparing retention meta data metrics with thresholds;

setting retention flags on video file segments with metrics of significant value;

setting retention flags on video file segments which coincide with extrinsic events;

extracting retention flags and metrics embedded in file headers; and

,

setting retention flags on file segments subject to legal requirements, statutes, warrants, court orders, and subpoenas.

19. The method of claim 10 wherein removing pointers to video file segments available to purging comprises:

determining when a file segment is protected from purging by legal constraints;

determining when a file segment is protected from purging by policy;

determining when a file segment is protected from purging by elapsed time from recordation or use;

determining when a file segment is protected from purging by an extrinsic retention meta data value or event mark;

determining when a file segment is protected from purging by a measure of content above a threshold; and

removing a pointer to unprotected video segments from the file system whereby the non-transitory media is returned to free space.

20. A method for surveillance image storage optimization comprising:

receiving one or more purging control directives;

retrieving a retention meta data array for each of a plurality of image segment identifiers;

masking each retrieved retention meta data array by each received purging control directive to set a retention flag for each image segment identifier;

purging from surveillance image store, the image segments corresponding to image segment identifiers not set for retention;

transforming high definition video into a plurality of image frames;

transforming stored image frames into image segments and retention meta data arrays;

storing image segment identifiers and retention meta data arrays into a retention meta data store;

reversing Huffman encoding of coefficients of pixel blocks;

differencing the low frequency coefficients between corresponding pixel blocks in Kframes and Jframes;

determining a metric of relative movement by normalizing the difference of low frequency coefficients; and

,

storing into retention meta data store an image segment identifier and an associated metric of relative movement;

wherein a retention meta data array includes an extrinsic sensor measurement correlated to the image segment;

wherein a retention meta data array includes a timestamp of file creation and records of file system operations on the image segment;

wherein purging control directives include a threshold of a metric of relative movement within the image segment, a level of extrinsic sensor measurement, and an inequality of at least one of elapsed time since capture, most recent file access, and least recent file access;

wherein purging control directives include ownership, location, and time of capture as specified in judicial orders, legislative statutes, and administrative policies.