Method and apparatus for event detection permitting per event adjustment of false alarm rate

• US 20120271782A1
• Filed: 03/22/2012
• Published: 10/25/2012
• Est. Priority Date: 04/20/2011
• Status: Active Grant

• Current Assignee: United States Air Force
• Original Assignee: United States Air Force
• Inventors: Blowers, Misty, Salisbury, Chad
• Granted Patent: US 8,732,100 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G05B 23/00 Testing or monitoring of control systems or parts thereof monitoring of programme-control systems G05B19/048, G05B19/406
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks

Abstract

Method and apparatus for object or event of interest detection which minimizes the level of false alarms and maximizes the level of detections as defined on a per event or object basis by the analyst. The invention allows for the minimization of false alarms for objects or events of interest which have a close resemblance to all other objects or events mapped to the same multidimensional feature space, and allows for the per event or per object adjustment on false alarms for objects or events of higher interest.

Claims

1. A method for event detection permitting per event adjustment of false alarm rate, comprising the steps of:

loading operational data;

loading background data;

comparing said operational data to said background data;

determining from said step of comparing, whether said operational data contains predictive data;

tagging predictive data at the time when and where it occurs;

extracting and reporting statistical features from said predictive data;

selecting said statistical features of choice so as to form at least one feature vector;

associating into a plurality of clusters, patterns formed by said at least one feature vector according to the character of said predictive data and said time at which it occurred;

optimizing false alarm thresholds and detection rates for each of said plurality of clusters;

detecting events; and

feeding back results.

2. The method of claim 1, wherein said step of associating further comprises the steps of:

mapping each of said at least one feature vector in multidimensional feature space;

forming clusters of each of said at least one feature vector into a class of feature vectors corresponding to time stamps which define conditions of interest;

calculating an optimum number of associated patterns based upon the compactness of said clusters;

initialize bounds around regions defining said associated patterns using multi-objective optimization method, comprising the steps of;

testing threshold values of 0 through 100% for each said cluster;

finding a threshold value for each said cluster that results in highest detection rate to false alarm rate ratio;

WHEN the overall said detection rate is less than a desired minimum;

finding a cluster that would produce the greatest difference between a change in said detection rate and change in said false alarm rate if its threshold was increased by 0.05; and

increasing said threshold by 0.05;

OTHERWISE, setting said threshold of each of said clusters with a detection rate of 0, to 0.

3. Method of claim 2, wherein said step of detecting events further comprises the steps of:

loading validation data; and

optimizing event detection.

4. The method of claim 3, wherein said step of optimizing event detection further comprises the steps of:

determining whether optimal event detection has been achieved;

WHEN optimal event detection has not yet been achieved

associating said operational data with corresponding said clusters;

adjusting threshold values which bound the clusters based upon event attribution associated with each cluster; and

comparing results of false alarm rate and correct detection rate with an acceptable false alarm rate on a per cluster basis;

OTHERWISE, returning to said step of detecting events.

5. The method of claim 1 wherein, when said event detection comprises industrial process event detection,

said operational data comprises operator logs;

said background data comprises training data; and

said predictive data comprises indicators of unstable process events.

6. The method of claim 1 wherein, when said event detection comprises object detection,

said operational data comprises object data;

said background data comprises attribution data; and

said predictive data comprises indicators of objects of interest.

7. Method of claim 1, wherein said step of selecting said statistical features further comprises the steps of:

extracting statistical features;

performing correlation analysis using statistical T-Test so as to compare on a feature-to-feature basis said statistical features which provide the most class discrimination value;

determining quantity of outlying features on a per feature basis;

performing statistical analysis on a per feature basis between the two classes; and

reporting via graphical user interface, the results of above analysis to an analyst.

8. An apparatus for detecting process events, comprising

a computer;

a software program; and

a means for communicating inputs and outputs to and from said computer and said process;

wherein

said software program further comprises a set of computer-implementable instructions stored on a non-transitory media, which, when executed by said computer cause said computer to perform the following steps;

loading operational data;

loading background data;

comparing said operational data to said background data;

determining from said step of comparing, whether said operational data contains predictive data;

tagging predictive data at the time when and where it occurs;

extracting and reporting statistical features from said predictive data;

selecting said statistical features of choice so as to form at least one feature vector;

associating into a plurality of clusters, patterns formed by said at least one feature vector according to the character of said predictive data and said time at which it occurred;

optimizing false alarm thresholds and detection rates for each of said plurality of clusters;

detecting events; and

feeding back results.

9. The apparatus of claim 8, wherein said step of associating further comprises the steps of:

mapping each of said at least one feature vector in multidimensional feature space;

forming clusters of each of said at least one feature vector into a class of feature vectors corresponding to time stamps which define conditions of interest;

calculating an optimum number of associated patterns based upon the compactness of said clusters;

initialize bounds around regions defining said associated patterns using multi-objective optimization method, comprising the steps of;

testing threshold values of 0 through 100% for each said cluster;

finding a threshold value for each said cluster that results in highest detection rate to false alarm rate ratio;

WHEN the overall said detection rate is less than a desired minimum;

finding a cluster that would produce the greatest difference between a change in said detection rate and—

change in said false alarm rate if its threshold was increased by 0.05; and

increasing said threshold by 0.05;

OTHERWISE, setting said threshold of each of said clusters with a detection rate of 0, to 0.

10. The apparatus of claim 9, wherein said step of detecting events further comprises the steps of:

loading validation data; and

optimizing event detection.

11. The apparatus of claim 10, wherein said step of optimizing event detection further comprises the steps of:

determining whether optimal event detection has been achieved;

WHEN optimal event detection has not yet been achieved associating said operational data with corresponding said clusters;

adjusting threshold values which bound the clusters based upon event attribution associated with each cluster; and

comparing results of false alarm rate and correct detection rate with an acceptable false alarm rate on a per cluster basis;

OTHERWISE, returning to said step of detecting events.

12. The apparatus of claim 8 wherein, when said event detection comprises industrial process event detection,

said operational data comprises operator logs;

said background data comprises training data; and

said predictive data comprises indicators of unstable process events.

13. The apparatus of claim 8 wherein, when said event detection comprises object detection,

said operational data comprises object data;

said background data comprises attribution data; and

said predictive data comprises indicators of objects of interest.

14. The apparatus of claim 8, wherein said step of selecting said statistical features further comprises the steps of:

extracting statistical features;

performing correlation analysis using statistical T-Test so as to compare on a feature-to-feature basis said statistical features which provide the most class discrimination value;

determining quantity of outlying features on a per feature basis;

performing statistical analysis on a per feature basis between the two classes; and

reporting via said computer'"'"'s graphical user interface, the results of above analysis to an analyst.