Minimization of surprisal context data through application of a hierarchy of reference artifacts

• US 9,176,998 B2
• Filed: 06/20/2013
• Issued: 11/03/2015
• Est. Priority Date: 05/28/2013
• Status: Active Grant

• Current Assignee: Daedalus Blue LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Friedlander, Robert R., Kraemer, James R., Adams, Samuel S
• Primary Examiner(s): Arjomandi, Noosha
• Application Number:
  US13/922,625
• Publication Number:
  US 20140358941A1
• Time in Patent Office: 866 Days
• Field of Search: 707/745
• US Class Current: 1/1
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management

Abstract

A method, system, and computer program product for minimizing surprisal context data. The method comprising the steps of: identifying characteristics of a data event; receiving an input of rank of at least two identified characteristics of the data event; the computer generating a hierarchy of ranked, identified characteristics based on the rank of the at least two identified characteristics of the data event; and comparing the hierarchy of ranked, identified characteristics to a repository of reference artifacts arranged in characteristic context patterns. If at least one reference artifact arranged in characteristic context patterns from the repository matches the hierarchy of ranked, identified characteristics, the characteristic context pattern is stored as a surprisal context filter and compared to a data input of data events to detect anomalous events.

Claims

1. A method of minimizing surprisal context data, the method comprising the steps of:

a computer identifying characteristics of at least one data event;

the computer receiving an input of rank of at least two identified characteristics of the at least one data event;

the computer generating a hierarchy of ranked, identified characteristics based on the rank of the at least two identified characteristics of the at least one data event;

the computer comparing the hierarchy of ranked, identified characteristics to a repository of reference artifacts arranged in characteristic context patterns;

when at least one reference artifact arranged in characteristic context patterns from the repository matches the hierarchy of ranked, identified characteristics,

the computer storing the characteristic context pattern with the least one matched reference artifact in a repository as a surprisal context filter with context determined to be probabilistically present within a specified degree of certainty in a data input;

the computer comparing the data input to the selected at least one surprisal context filter;

the computer discarding the events from the data input that are the same as the context data for which the at least one surprisal context filter selects; and

the computer storing in a repository the events remaining in the data input as anomalous events.

2. The method of claim 1, wherein if more than one surprisal context filter is present, each of the surprisal context filters have context data determined to be probabilistically present at different specified degrees of certainty in the data input.

3. The method of claim 1, wherein the data input is a data stream.

4. The method of claim 1, wherein the data input is data stored in a database.

5. The method of claim 1, further comprising the step of the computer sending a notification to a user regarding the anomalous events.

6. The computer program product of claim 1, wherein the data input is a data stream.

7. The computer program product of claim 1, wherein the data input is data stored in a database.

8. The computer program product of claim 1, further comprising the step of the computer sending a notification to a user regarding the anomalous events.

9. A computer program product for minimizing surprisal context data, the computer program product comprising:

one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to identify characteristics of at least one data event;

program instructions, stored on at least one of the one or more storage devices, to receive an input of rank of at least two identified characteristics of the at least one data event;

program instructions, stored on at least one of the one or more storage devices, to generate a hierarchy of ranked, identified characteristics based on the rank of the at least two identified characteristics of the at least one data event;

program instructions, stored on at least one of the one or more storage devices, to compare the hierarchy of ranked, identified characteristics to a repository of reference artifacts arranged in characteristic context patterns;

when at least one reference artifact arranged in characteristic context patterns from the repository matches the hierarchy of ranked, identified characteristics,

program instructions, stored on at least one of the one or more storage devices, to store the characteristic context pattern with the least one matched reference artifact in a repository as a surprisal context filter with context determined to be probabilistically present within a specified degree of certainty in a data input;

compare the data input to the selected at least one surprisal context filter;

discard the events from the data input that are the same as the context data for which the at least one surprisal context filter selects; and

store in a repository the events remaining in the data input as anomalous events.

10. The computer program product of claim 9, wherein when more than one surprisal context filter is present, each of the surprisal context filters have context data determined to be probabilistically present at different specified degrees of certainty in the data input.

11. A system for minimizing surprisal context data, the system comprising:

one or more processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to identify characteristics of at least one data event;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to receive an input of rank of at least two identified characteristics of the at least one data event;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to generate a hierarchy of ranked, identified characteristics based on the rank of the at least two identified characteristics of the at least one data event;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to compare the hierarchy of ranked, identified characteristics to a repository of reference artifacts arranged in characteristic context patterns;

when at least one reference artifact arranged in characteristic context patterns from the repository matches the hierarchy of ranked, identified characteristics,

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to store the characteristic context pattern with the least one matched reference artifact in a repository as a surprisal context filter with context determined to be probabilistically present within a specified degree of certainty in a data input;

compare the data input to the selected at least one surprisal context filter;

discard the events from the data input that are the same as the context data for which the at least one surprisal context filter selects; and

store in a repository the events remaining in the data input as anomalous events.

12. The system of claim 11, wherein when more than one surprisal context filter is present, each of the surprisal context filters have context data determined to be probabilistically present at different specified degrees of certainty in the data input.

13. The system of claim 11, wherein the data input is a data stream.

14. The system of claim 11, wherein the data input is data stored in a database.

15. The system of claim 11, further comprising the step of the computer sending a notification to a user regarding the anomalous events.