Network usage analysis system and method for updating statistical models

• US 7,506,046 B2
• Filed: 07/31/2001
• Issued: 03/17/2009
• Est. Priority Date: 07/31/2001
• Status: Expired due to Fees

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Rhodes, N. Lee
• Primary Examiner(s): Burgess; Glenton B
• Assistant Examiner(s): NASH, LASHANYA RENEE
• Application Number:
  US09/919,527
• Publication Number:
  US 20030028631A1
• Time in Patent Office: 2,786 Days
• Field of Search: 709223-226, 709/231, 705 10- 11, 705/52
• US Class Current: 709/224
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: H04L 41/142 using statistical or mathematical methods
• CPC Class Codes: H04L 41/5003 Managing SLA; Interaction between SLA and QoS
• CPC Class Codes: H04L 41/5019 Ensuring fulfilment of SLA
• CPC Class Codes: H04L 41/5029 Service quality level-based billing, e.g. dependent on measured service level customer is charged more or less
• CPC Class Codes: H04L 43/00 Arrangements for monitoring or testing data switching networks
• CPC Class Codes: H04L 43/022 by sampling
• CPC Class Codes: H04L 43/16 Threshold monitoring

Abstract

In one embodiment, the present invention provides a system and method for analyzing a stream of network data. The method includes generating a statistical model from a set of record events. A most recent record event is received. The statistical model is updated using the most recent record event by adding the most recent record event to the statistical model.

Claims

1. A method for analyzing a stream of network usage data comprising:

generating a statistical model from a set of usage data record events from the stream of network usage data;

receiving a most recent record event;

updating the statistical model using the most recent record event by adding the most recent record event to the statistical model, wherein an identifier is associated with each record event, including updating only a portion of the statistical model associated with the identifier;

storing the set of record events in a history cache, and wherein if the history cache is full, updating the statistical model includes removing a least recent record event from the statistical model; and

defining the statistical model to include an aggregation of each record event set stored in the history cache;

wherein generating a statistical model from the set of record events includes generating an aggregation table for tracking an aggregation of record events associated with an identifier; and

wherein the most recent record event is associated with an identifier; and

wherein updating the statistical model includes updating only the aggregation of record events in the tracking table for that identifier.

2. The method of claim 1, comprising generating a complex statistical model representative of the network data from the aggregation table.

3. The method of claim 1, comprising generating a histogram statistical model representative of the network data from the aggregation table.

4. The method of claim 1, further comprising generating a complex statistical model from the aggregation table.

5. The method of claim 4, further wherein updating the statistical model includes updating only a portion of the complex statistical model associated with an identifier.

6. The method of claim 1, further comprising generating a histogram statistical model from the aggregation table.

7. The method of claim 6, wherein if the history cache is full, updating the statistical model further includes removing a least recent record event set associated with a least recent update time interval from the statistical model.

8. The method of claim 6, further comprising defining the statistical model to include an aggregation of each record event set stored in the history cache.

9. The method of claim 6, further comprising storing each record event in a memory segment in the history cache.

10. The method of claim 9, further comprising defining an index array including a set of contiguous index segments, wherein each index segment including a pointer to the memory segment storing in the history cache storing the next consecutive record event.

11. The method of claim 10, further defining a first pointer to the index segment associated with the memory segment storing the least recent record event.

12. The method of claim 11, wherein upon receiving a most recent record event the method further comprising replacing the least recent record event stored in the history cache with the most recent record event.

13. The method of claim 12, further comprising moving the first pointer to the next contiguous index segment.

14. The method of claim 11, further defining a second pointer to the index segment associated with the memory segment storing the most recent record event.

15. The method of claim 1, wherein upon receiving a most recent record event the method further comprising replacing the least recent record event stored in the history cache with the most recent record event.

16. A method for analyzing a stream of network usage data over a rolling time interval comprising:

defining a statistical model for analyzing the stream of network usage data over the rolling time interval;

defining the rolling time interval to include a plurality of update time intervals;

receiving a record event set from the stream of data for each update time interval, each record event set including one or more record events, wherein each record event is associated with a user identifier;

storing the record event set for each update interval in a history cache;

generating the statistical model over the rolling time interval using each record event stored in the history cache;

updating only a portion of the statistical model associated with the most recent record event for a most recent update time interval;

defining the statistical model to include an aggregation table having an aggregation table entry for tracking an aggregation of each record event set stored in the history cache for each user identifier; and

generating a histogram from the aggregation table;

wherein the step of updating the statistical model includes the step of updating only the aggregation table entry associated with the most recent record event; and

wherein updating the statistical result includes updating only the portion of the histogram associated with the aggregation table entry associated with the most recent record event.

17. The method of claim 16, wherein if the history cache is full, updating the statistical model further includes removing a least recent record event set associated with a least recent update time interval from the statistical model.

18. A network usage analysis system for analyzing a stream of network usage data comprising:

a data analysis system server which generates a statistical model from a set of usage data record events, and upon receiving a most recent record event, the data analysis system server updates the statistical model using the most recent record event by adding the most recent record event to the statistical model, wherein customer usage is associated with each record event, including updating only a portion of the statistical model associated with the customer usage;

defining a history cache, wherein the data analysis system server stores the set of record events in the history cache, and wherein if the history cache is full, updating the statistical model includes removing a least recent record event from the statistical model; and

defines the statistical model to include an aggregation of each record event set stored in the history cache;

wherein an identifier is associated with each record event, and wherein generating a statistical model from the set of record events includes the data analysis system server generating an aggregation table for tracking an aggregation of record events associated with an identifier; and

wherein the most recent record event is associated with an identifier; and

wherein the data analysis system server updating the statistical model includes updating only the aggregation of record events in the tracking table for that identifier.

19. The system of claim 18, wherein the data analysis system server generates a complex statistical model representative of the network data from the aggregation table.

20. The system of claim 18, further wherein the data analysis system server generates a complex statistical model from the aggregation table.

21. A method for analyzing a stream of network usage data comprising:

generating a statistical model from a set of network usage record events;

receiving a most recent record event; and

updating the statistical model using the most recent record event by adding the most recent record event to the statistical model further comprising the steps of;

storing the set of record events in a history cache;

wherein if the history cache is full, updating the statistical model includes removing a least recent record event from the statistical model, further comprising;

defining the statistical model to include an aggregation of each record event set stored in the history cache, wherein an identifier is associated with each record event, and wherein generating a statistical model from the set of record events includes generating an aggregation table for tracking an aggregation of record events associated with an identifier, wherein the most recent record event is associated with an identifier; and

wherein updating the statistical model includes updating only the aggregation of record events in the tracking table for that identifier, further comprising generating a histogram statistical model from the aggregation table, wherein the history cache is an array of memory segments, wherein the number of memory segments is equal to the number of update time intervals in the rolling time interval, and storing each record event in a memory segment in the history cache, and defining an index array associated with the statistical model including a set of contiguous index segments, wherein each index segment includes a pointer to the memory segment in the history cache storing the next consecutive record event, further defining a first pointer to the index segment associated with the memory segment storing the least recent record event, wherein upon receiving a most recent record event the method replacing the least recent record event stored in the history cache with the most recent record event, and further moving the first pointer to the next contiguous index segment.

22. The method of claim 21, wherein the histogram statistical model includes a first statistical model including a histogram having a first axis illustrating total usage defined by a number of bins, each bin having a usage variable range, and a second axis defined by a frequency corresponding to a number of users having a total usage within the usage variable range of each bin.