Network usage analysis system and method for updating statistical models

• US 20030028631A1
• Filed: 07/31/2001
• Published: 02/06/2003
• Est. Priority Date: 07/31/2001
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Rhodes, N. Lee
• Granted Patent: US 7,506,046 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 41/5019 Ensuring fulfilment of SLA
• CPC Class Codes: H04L 41/5029 Service quality level-based billing, e.g. dependent on measured service level customer is charged more or less
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/022 by sampling
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

In one embodiment, the present invention provides a system and method for analyzing a stream of network data. The method includes generating a statistical model from a set of record events. A most recent record event is received. The statistical model is updated using the most recent record event by adding the most recent record event to the statistical model.

Claims

1. A method for analyzing a stream of network data comprising:

generating a statistical model from a set of record events;

receiving a most recent record event; and

updating the statistical model using the most recent record event by adding the most recent record event to the statistical model.

2. The method of claim 1, wherein updating the statistical model further comprises removing a least recent record event from the statistical model.

3. The method of claim 1, further comprising the steps of storing the set of record events in a history cache, and wherein if the history cache is full, updating the statistical model includes removing a least recent record event from the statistical model.

4. The method of claim 3, further comprising defining the statistical model to include an aggregation of each record event set stored in the history cache.

5. The method of claim 4, wherein an identifier is associated with each record event, and wherein generating a statistical model from the set of record events includes generating an aggregation table for tracking an aggregation of record events associated with an identifier.

6. The method of claim 5, comprising generating a complex statistical model representative of the network data from the aggregation table.

7. The method of claim 5, comprising generating a histogram statistical model representative of the network data from the aggregation table.

8. The method of claim 5, wherein the most recent record event is associated with an identifier;

and wherein updating the statistical model includes updating only the aggregation of record events in the tracking table for that identifier.

9. The method of claim 8, further comprising generating a complex statistical model from the aggregation table.

10. The method of claim 9, further wherein updating the statistical model includes updating only a portion of the complex statistical model associated with an identifier.

11. The method of claim 8, further comprising generating a histogram statistical model from the aggregation table.

12. The method of claim 5, wherein upon receiving a most recent record event the method further comprising replacing the least recent record event stored in the history cache with the most recent record event.

14. The method of claim 11, wherein if the history cache is full, updating the statistical model further includes removing a least recent record event set associated with a least recent update time interval from the statistical model.

15. The method of claim 11, further comprising defining the statistical model to include an aggregation of each record event set stored in the history cache.

16. The method of claim 11, wherein the history cache is an array of memory segments, wherein the number of memory segments is equal to the number of update time intervals in the rolling time interval.

17. The method of claim 16, further comprising storing each record event in a memory segment in the history cache.

18. The method of claim 17, further comprising defining an index array associated including a set of contiguous index segments, wherein each index segment including a pointer to the memory segment storing in the history cache storing the next consecutive record event.

19. The method of claim 18, further defining a first pointer to the index segment associated with the memory segment storing the least recent record event.

20. The method of claim 19, wherein upon receiving a most recent record event the method further comprising replacing the least recent record event stored in the history cache with the most recent record event.

21. The method of claim 20, further comprising moving the first pointer to the next contiguous index segment.

22. The method of claim 19, further defining a second pointer to the index segment associated with the memory segment storing the most recent record event.

24. The method of claim 21, wherein if the history cache is full, updating the statistical model further includes removing a least recent record event set associated with a least recent update time interval from the statistical model.

25. The method of claim 21, further comprising defining the statistical model to include an aggregation table having an aggregation table entry for tracking an aggregation of each record event set stored in the history cache for each user identifier.

13. A method for analyzing a stream of data over a rolling time interval comprising:

defining a statistical model for analyzing the stream of data over the rolling time interval;

defining the rolling time interval to include a plurality of update time intervals;

receiving a record event from the stream of data for each update time interval;

storing the record event for each update interval in a history cache;

generating the statistical model over the rolling time interval using the statistical model and each record event stored in the history cache; and

updating the statistical model using the statistical model and a most recent record event for a most recent update time interval.

23. A method for analyzing a stream of data over a rolling time interval comprising:

defining a statistical model for analyzing the stream of data over the rolling time interval;

defining the rolling time interval to include a plurality of update time intervals;

receiving a record event set from the stream of data for each update time interval, each record event set including one or more record events, wherein each record event is associated with a user identifier;

storing the record event set for each update interval in a history cache;

generating the statistical model over the rolling time interval using each record event stored in the history cache; and

updating only a portion of the statistical model associated with the most recent record event for a most recent update time interval.

26. The method of claim 23, wherein the step of updating the statistical model includes the step of updating only the aggregation table entry associated with the most recent record event.

27. The method of claim 26, further comprising generating a histogram from the aggregation table.

28. The method of claim 27, further wherein updating the statistical result includes updating only the portion of the histogram associated with the aggregation table entry associated with the most recent record event.

29. A method for analyzing a stream of network data over a rolling time interval comprising:

defining a statistical model for analyzing the stream of network data over the rolling time interval;

defining the rolling time interval to include a plurality of update time intervals;

receiving a record event set from the stream of network data for each update time interval;

storing the record event set for each update interval in a history cache;

generating the statistical model over the rolling time interval using each record event stored in the history cache including generating an aggregation table; and

updating the statistical model using a most recent record event for a most recent update time interval including updating only a portion of the aggregation table associated with the most recent update time interval.

30. The method of claim 29, wherein if the history cache is full, updating the statistical model further includes removing a least recent record event set associated with a least recent update time interval from the statistical model.

31. The method of claim 29, further comprising defining the statistical model to include an aggregation of each record event set stored in the history cache.

32. The method of claim 29, wherein the history cache is an array of memory segments, wherein the number of memory segments is equal to the number of update time intervals in the rolling time interval.

33. The method of claim 32, further comprising storing each record event in a memory segment in the history cache.

34. The method of claim 29, further comprising defining an index array associated including a set of contiguous index segments, wherein each index segment including a pointer to the memory segment storing in the history cache storing the next consecutive record event.

35. The method of claim 34, further defining a first pointer to the index segment associated with the memory segment storing the least recent record event.

36. The method of claim 29, further comprising generating a histogram statistical model from the aggregation table;

and updating only a portion of the histogram statistical model associated with most recent record event set.

37. A network usage analysis system for analyzing a stream of network data comprising:

a data analysis system server which generates a statistical model from a set of record events, and upon receiving a most recent record event, the data analysis system server updates the statistical model using the most recent record event by adding the most recent record event to the statistical model.

38. The system of claim 37, further comprising:

defining a least recent record event; and

wherein the data analysis system server removes a least recent record event from the statistical model.

39. The system of claim 37, further comprising:

defining a history cache, wherein the data analysis system server stores the set of record events in the history cache, and wherein if the history cache is full, updating the statistical model includes removing a least recent record event from the statistical model.

40. The system of claim 39, further comprising:

defines the statistical model to include an aggregation of each record event set stored in the history cache.

41. The system of claim 40, wherein an identifier is associated with each record event, and wherein generating a statistical model from the set of record events includes the data analysis system server generating an aggregation table for tracking an aggregation of record events associated with an identifier.

42. The system of claim 41, wherein the data analysis system server generates a complex statistical model representative of the network data from the aggregation table.

43. The system of claim 41, wherein the most recent record event is associated with an identifier;

and wherein the data analysis system server updating the statistical model includes updating only the aggregation of record events in the tracking table for that identifier.

44. The system of claim 43, further wherein the data analysis system server generates a complex statistical model from the aggregation table.

45. A computer-readable medium having computer executable instructions for performing a method for analyzing a stream of network data, the method comprising:

generating a statistical model from a set of record events;

receiving a most recent record event; and

updating the statistical model using the most recent record event by adding the most recent record event to the statistical model.