Method and system for efficient distribution of network event data

• US 7,043,727 B2
• Filed: 06/08/2001
• Issued: 05/09/2006
• Est. Priority Date: 06/08/2001
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: Micromuse Incorporated (International Business Machines Corporation)
• Inventors: Franklin, David Richard, Stewart, Kristian Jon, Bennett, Andrew Jonathan
• Primary Examiner(s): Thomson, William
• Assistant Examiner(s): Wu, Qing-Yuan
• Application Number:
  US09/877,619
• Publication Number:
  US 20030014462A1
• Time in Patent Office: 1,796 Days
• Field of Search: 709/219, 709/247, 709223-224, 707/3, 718/100
• US Class Current: 718/100
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 12/1868 Measures taken after transmission, e.g. acknowledgments

Abstract

Described herein are methods and systems for preparing to efficiently distribute data to be extracted from a data store to clients and for distributing such prepared data to the clients. The methods and system have particular utility in the context of a network monitoring system which captures and stores network event data and makes the event data available to clients according to any desired view or summary. One method for preparing the data involves storing as primary requests one or more client requests for data to be extracted from the event database. Additional client requests for event are each compared to the stored primary requests to determine whether each additional request matches a stored primary request in accordance with a given criterion, such as a filter or summary formula. If the additional request matches a stored primary request, the additional client request is associated with the matching primary request. If the additional request does not match a stored primary request, the additional request is stored as a new primary request. As a result of this method, client requests which match one another can be processed at once and distributed to all clients registering the request.

Claims

1. A computer implemented method for preparing to distribute data to be extracted from a data store to a plurality of clients, the method comprising:

storing in a table as primary requests one or more client requests for periodic updates of data to be extracted from the data store according to one or more first filters;

for an additional client request for periodic updates of data to be extracted from the data store according to one or more second filters, comparing the additional client request to the primary requests stored in the table to determine whether the one or more second filters in the additional client request matches the one or more first filters of one of the stored primary requests;

if the one or more second filters of the additional client request matches the one or more first filters of a particular stored primary request, storing data identifying the additional client request in the table as a secondary request associated with the particular stored primary request; and

if the one or more second filters of the additional client request does not match the one or more first filters of any of the stored primary requests, storing the additional client request in the table as an additional primary request,

periodically extracting data updates from the data store in accordance with the one or more first filters in a stored primary request; and

distributing the extracted data updates to the stored primary request and an associated secondary request.

2. The method of claim 1, wherein the stored primary requests and the additional client request each comprise a request for summary data comprising data to be extracted from the data store and processed in accordance with a metric.

3. The method of claim 2, wherein comparing the additional request to the stored primary requests comprises comparing the metric of the additional request to the metric of the stored primary request.

4. The method of claim 1, wherein the data store comprises event data relating to events occurring on a network.

5. The method of claim 1, wherein the stored primary requests are received from first clients and the additional client request is received from a second client different than the first clients.

6. The method of claim 1, comprising storing in the table a set of client requests received from a plurality of clients, the set comprising a primary client request for periodic updates of data to be extracted from the data store according to one or more first filters to a first client in association with one or more secondary client requests for periodic updates of data to be extracted from the data store according to one or more second filters to one or more second clients different than the first client, the secondary client requests each having one or more second filters for performing the periodic updates matching one or more first filters for performing the periodic updates contained in the primary client request;

periodically extracting data updates from the data store in accordance with the one or more first filters in the primary client request; and

distributing the extracted data updates to the first client and to the one or more second clients.

7. The method of claim 6, comprising storing a plurality of sets of client requests, each set comprising a primary client request for periodic updates of data in association with one or more secondary client requests, the secondary client requests each having one or more second filters that match the one or more first filters of a respective primary client request.

8. The method of claim 7, comprising performing the steps of extracting and distributing data for each of the plurality of sets of client requests.

9. The method of claim 8, comprising pausing a first time interval between extracting and distributing each of the plurality of sets of client requests.

10. The method of claim 9, comprising determining a length of time required to extract and distribute data to all first and second clients and, if the determined time length exceeds a threshold, increasing the first time interval.

11. The method of claim 6, wherein the primary and secondary client requests each comprise the filters for extracting a subset of data from the data store, and wherein storing the set comprises identifying the set through the filters.

12. The method of claim 6, wherein at least one of the client requests in the set comprise a metric for summarizing data extracted from the data store, comprising summarizing the extracted data using the metric and distributing the summarized data to any first or second client whose client request comprises the metric.

13. A system for distribution of network event data, the system comprising:

a data store containing data relating to events occurring on the network;

a library for storing as primary requests one or more client requests for periodic updates of data to be extracted from the data store according to one or more first filters;

for an additional client request for periodic updates of data to be extracted from the data store according to one or more second filters, comparing the additional client request to the primary requests stored in the table to determine whether the one or more second filters in the additional client request matches the one or more first filters of one of the stored primary requests;

if the one or more second filters of the additional client request matches the one or more first filters of a particular stored primary request, storing data identifying the additional client request in the table as a secondary request associated with the particular stored primary request; and

if the one or more second filters of the additional client request does not match the one or more first filters of any of the stored primary requests, storing the additional client request in the table as an additional primary request,

periodically extracting data updates from the data store in accordance with the one or more first filters in a stored primary request; and

a notification system for distributing the extracted data updates to the stored primary request and an associated secondary request.

14. The system of claim 13, wherein the notification system comprises a notifier program running a plurality of program threads.

15. The system of claim 13, comprising a plurality of data stores arranged in a cluster, wherein one data store comprises a master data store and one or more other data stores comprise slave data stores.

16. The system of claim 15, comprising means for synchronizing the network event data stored in the data stores between the master and slave data stores.

17. The system of claim 16, comprising a slave client request library associated with each slave data store and means for synchronizing the library for the master data store with the slave client request libraries.

18. The system of claim 17, comprising a slave notification system associated with each slave data store for, in the event of failure of the master data store or notification system, distributing to the plurality of clients data extracted from the slave data store in accordance with the one or more sets of client requests stored in the slave library.

19. The system of claim 18, wherein the master notification system distributes data extracted from the master data store to the one or more slave notification systems.

20. The system of claim 18, comprising means for maintaining an order of distributed data delivered to a first client to thereby enable the first client to properly order the distributed data in the event of a temporary failure of the first client.

21. The system of claim 20, wherein the first client is a gateway between a second client and the slave notification system.

22. The system of claim 20, wherein the means for maintaining the order comprises a sequence count inserted in the distributed data by the master and slave notification systems.