AUTOMATED DATA EXPLORATION

• US 20140040279A1
• Filed: 08/02/2012
• Published: 02/06/2014
• Est. Priority Date: 08/02/2012
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mastronarde, Nicholas, Parthasarathy, Srinivasan, Beygelzimer, Alina, Riabov, Anton V., Turaga, Deepak, Udrea, Octavian
• Application Number:
  US13/565,257
• Publication Number:
  US 20140040279A1
• Time in Patent Office: Days
• US Class Current: 707/748
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: H04L 41/145 involving simulating, designing, planning or modelling of a network
• CPC Class Codes: H04L 43/026 using flow identification
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00

Abstract

A method for automated data exploration including selecting a plurality of analytic flows from an analytic flow pattern, executing a task, wherein the task is tracked by the plurality of analytic flows, receiving feedback for each of the plurality of analytic flows, determining a performance score for each of the plurality of analytic flows, and adjusting the flow according to the performance score.

Claims

1. A method for automated data exploration comprising:

receiving a data flow via a network of connected computer nodes;

extracting a plurality of attributes of the data flow;

deriving a plurality of features from each of the attributes;

aggregating a plurality of data items of the data flow;

creating a model of the data flow given the attributes, the features, and an aggregation of the data items; and

detecting an event in the data flow according to the model.

2. The method of claim 1, wherein the aggregation is performed over an entirety of the data flow.

3. The method of claim 1, further comprising partitioning the data flow, wherein the aggregation is performed over a partition of the data flow.

4. The method of claim 1, wherein the event inconsistent with the model.

5. The method of claim 4, further comprising receiving feedback corresponding to a measured performance of the model.

6. The method of claim 5, further comprising adjusting the extraction of the plurality of attributes of the data flow according to the feedback.

7. A computer program product for automated data exploration comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code configured to select a plurality of analytic flows from an analytic flow pattern;

computer readable program code configured to execute a task, wherein the task is tracked by the plurality of analytic flows;

computer readable program code configured to receive feedback for each of the plurality of analytic flows;

computer readable program code configured to determine a performance score for each of the plurality of analytic flows; and

computer readable program code configured to adjust the selecting of the plurality of analytic flows from the analytic flow pattern according to the performance score.

8. The computer program product of claim 7, wherein adjusting the flow comprises adding a flow from the pattern.

9. The computer program product of claim 7, wherein adjusting the selection of the plurality of analytics flows comprises removing a flow from an existing selection.

10. The computer program product of claim 7, further comprises requesting the feedback.

11. The computer program product of claim 10, wherein the feedback is provided by an external source.

12. The computer program product of claim 10, wherein the feedback is provided learned from a plurality of subscriptions to an external source.

13. A method for automated data exploration comprising:

selecting a plurality of analytic flows from an analytic flow pattern for detecting an anomaly in computer network traffic between a network of connected computer nodes;

executing a task for detecting the anomaly in the computer network traffic, wherein the task is tracked by the plurality of analytic flows;

receiving feedback for each of the plurality of analytic flows;

determining a performance score for each of the plurality of analytic flows indicative of a respective analytic flow'"'"'s ability to detect malware activity in the computer network traffic; and

adjusting the selection of the plurality of analytic flows according to the performance score.

14. The method of claim 13, wherein adjusting the selection of the plurality of analytic flows comprises adding an analytic flow from the pattern.

15. The method of claim 13, wherein the selection of the plurality of analytic flows comprises removing an analytic flow from the existing selection.

16. The method of claim 13, wherein further comprises requesting the feedback.

17. The method of claim 13, wherein the feedback is provided by an external source.

18. The method of claim 13, wherein the feedback is provided learned from a plurality of subscriptions to an external source.

19. The method of claim 13, wherein the feedback is a confirmation of a performance of at least one analytic flow.

20. The method of claim 13, wherein the feedback is a rejection of a performance of at least one analytic flow.