METHOD, SYSTEM, AND COMPUTER PROGRAM PRODUCT FOR IDENTIFYING COMMON FACTORS ASSOCIATED WITH NETWORK ACTIVITY WITH REDUCED RESOURCE UTILIZATION

• US 20090055420A1
• Filed: 08/21/2007
• Published: 02/26/2009
• Est. Priority Date: 08/21/2007
• Status: Active Grant

• Current Assignee: InMon Corp.
• Original Assignee: InMon Corp.
• Inventors: McKEE, Neil, PHAAL, Peter
• Granted Patent: US 8,838,774 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04L 43/02 Capturing of monitoring data

Abstract

Disclosed are a method, system, and computer program product for determining a common factor contributing to network activity with reduced computational resource utilization. In some embodiments of the present invention, the method or the system determines one or more factors by examining one or more information transmitted across the network. The method or the system stores a number of information attributable to each of the factors and determines whether the number exceeds a threshold requirement. Where the number for a factor exceeds the threshold requirement, the method or the system then determines a summary statistic for the number of information. Thereafter, the method or the System updates the data structure corresponding to the factor being analyzed based upon the summary statistic. Once the data structure is updated based upon the summary statistic, the method or the system determine one or more common factors for the network activities.

Claims

1. A machine implemented method for analyzing network traffic with reduced resource utilization, comprising:

identifying a first factor for a first record among one or more records;

storing a first value associated with the first factor in a data structure on a tangible machine accessible medium;

determining a summary statistic of all of the first values for some of the one or more records; and

compacting the data structure based upon the summary statistic, wherein the data structure is used for analyzing network traffic.

2. The machine implemented method of claim 1, in which the compacting the data structure based upon the summary statistic comprises:

pruning the data structure based upon the summary statistic until a first criterion is satisfied.

3. The machine implemented method of claim 2, in which the pruning the data structure comprises:

pruning a second value from the data structure, in which the second value is smaller than or equal to the summary statistic.

4. The machine implemented method of claim 1, in which the first record comprises information transmitted across a network.

5. The machine implemented method of claim 1, in which the first record comprises a network usage record reflecting the network traffic.

6. The machine implemented method of claim 1, in which the one or more records comprise information transmitted across a network or a network usage record.

7. The machine implemented method of claim 1, further comprising:

identifying a second factor for the first record.

8. The machine implemented method of claim 7, further comprising:

storing, in a tangible machine accessible medium, a second value associated with the second factor in the data structure; and

determining the summary statistic of all the first values and second values of some of the one or more records.

9. The machine implemented method of claim 1, further comprising:

compacting the data structure with an additional record.

10. The machine implemented method of claim 1, in which the first record comprises one or more attributes extracted from the information transmitted across the network.

11. The machine implemented method of claim 1, in which the data structure constitutes a database or an associative array comprising a hash table or a binary tree.

12. The machine implemented method of claim 1, the first value being an amount of the network traffic attributable to the first factor for the first record.

13. The machine implemented method of claim 1, the summary statistic being a mean. a median, a weighted average of the first values for some of the one or more records.

14. The machine implemented method of claim 1, in which the compacting the data structure is performed until a measurement time interval is complete.

15. The machine implemented method of claim 1, in which a predetermined amount of additional records is used to determine the summary statistic.

16. The machine implemented method of claim 1, in which the compacting the data structure with an additional information continues until a probability of mis-ranking the data structure being updated is within a predetermined tolerance.

17. The machine implemented method of claim 9, further comprising:

filtering the additional record.

18. The machine implemented method of claim 17, in which the filtering the additional record comprising eliminating one or more of the additional record.

19. The machine implemented method of claim 17, in which the filtering the additional information comprising eliminating a part of the additional record

20. The machine implemented method of claim 9, in which the additional record is received from one or more storage devices.

21. The machine implemented method of claim 9, in which the additional record is streamed in real time.

22. The machine implemented method of claim 1, in which the storing a first value in a data structure is performed wherein the first value exceeds a first threshold.

23. A system for identifying common factors associated with network activities with reduced resource utilization, comprising:

means for identifying a first factor for a first record among one or more records;

means for storing a first value associated with the first factor in a data structure on a tangible machine accessible medium;

means for determining a summary statistic of all of the first values for some of the one or more records; and

means for compacting the data structure based upon the summary statistic, wherein the data structure is used for analyzing network traffic.

24. The system of claim 23, in which the means for compacting the data structure based upon the summary statistic comprises:

means for pruning the data structure based upon the summary statistic until a first criterion is satisfied.

25. The system of claim 24, in which the means for compacting the data structure based upon the summary statistic comprises:

means for pruning a second value from the data structure, in which the second value is smaller than or equal to the summary statistic.

26. The system of claim 23, further comprising:

means for identifying a second factor for the first record.

27. The system of claim 26, further comprising:

means for storing, in a tangible machine accessible medium, a second value associated with the second factor in the data structure; and

means for determining the summary statistic of all the first values and second values of some of the one or more records.

28. The system of claim 23, further comprising:

means for compacting the data structure with an additional record.

29. The system of claim 28, further comprising:

means for filtering the additional record.

30. A computer program product that includes a medium usable by a processor, the medium having stored thereon a sequence of instructions which, when executed by said processor, causes said processor to execute a process for, the process comprising:

identifying a first factor for a first record among one or more records;

storing a first value associated with the first factor in a data structure on a tangible machine accessible medium;

determining a summary statistic of all of the first values for some of the one or more records; and

compacting the data structure based upon the summary statistic, wherein the data structure is used for analyzing network traffic.

31. The computer program product of claim 30, in which the process further comprises:

pruning the data structure based upon the summary statistic until a first criterion is satisfied.

32. The computer program product of claim 31, in which the process further comprises:

pruning a second value from the data structure, in which the second value is smaller than or equal to the summary statistic.

33. The computer program product of claim 30, in which the process further comprises:

identifying a second factor for the first record.

34. The computer program product of claim 33, in which the process further comprises:

storing, in a tangible machine accessible medium, a second value associated with the second factor in the data structure; and

determining the summary statistic of all the first values and second values of some of the one or more records.

35. The computer program product of claim 30, in which the process further comprises:

compacting the data structure with an additional record.

36. The computer program product of claim 35, in which the process further comprises:

filtering the additional record.