Method and system for efficient distribution of network event data

• US 20030014462A1
• Filed: 06/08/2001
• Published: 01/16/2003
• Est. Priority Date: 06/08/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Micromuse Incorporated (International Business Machines Corporation)
• Inventors: Bennett, Andrew Jonathan, Franklin, David Richard, Stewart, Kristian Jon
• Granted Patent: US 7,043,727 B2
• Time in Patent Office: Days
• US Class Current: 718/100
• CPC Class Codes: H04L 12/1859 adapted to provide push services, e.g. data channels
• CPC Class Codes: H04L 12/1868 Measures taken after transmission, e.g. acknowledgments

Abstract

Described herein are methods and systems for preparing to efficiently distribute data to be extracted from a data store to clients and for distributing such prepared data to the clients. The methods and system have particular utility in the context of a network monitoring system which captures and stores network event data and makes the event data available to clients according to any desired view or summary. One method for preparing the data involves storing as primary requests one or more client requests for data to be extracted from the event database. Additional client requests for event are each compared to the stored primary requests to determine whether each additional request matches a stored primary request in accordance with a given criterion, such as a filter or summary formula. If the additional request matches a stored primary request, the additional client request is associated with the matching primary request. If the additional request does not match a stored primary request, the additional request is stored as a new primary request. As a result of this method, client requests which match one another can be processed at once and distributed to all clients registering the request.

Claims

1. A method for preparing to efficiently distribute data to be extracted from a data store to a plurality of clients, the method comprising:

storing as primary requests one or more client requests for data to be extracted from the data store;

for an additional client request for data to be extracted from the data store, comparing the additional request to the stored primary requests to determine whether the additional request matches a stored primary request in accordance with a given criterion;

if the additional request matches a stored primary request, storing the additional client request as a secondary request associated with the matching primary request; and

if the additional request does not match a stored primary request, storing the additional request as an additional primary request.

2. The method of claim 1, wherein client requests and the additional client request each comprise a filter for extracting a subset of data from the data store.

3. The method of claim 2, wherein comparing the additional request to the stored primary requests comprises comparing the filter in the additional request to the stored primary request filters to determine whether the additional request filter matches any stored primary request filter.

4. The method of claim 1, wherein one or more client requests and the additional client request each comprise a request for summary data comprising data to be extracted from the data store and processed in accordance with a metric.

5. The method of claim 4, wherein comparing the additional request to the stored primary requests comprises comparing the additional request metric to the stored primary request metrics to determine whether the additional request metric matches any stored primary request metric.

6. The method of claim 4, wherein client requests and the additional client request each comprise a filter for extracting a subset of data from the data store, and wherein comparing the additional request to the stored primary requests comprises comparing the additional request filter to the stored primary request filters to determine whether the additional request filter matches any stored primary request filter.

7. The method of claim 1, wherein the data store comprises event data relating to events occurring on a network.

8. The method of claim 1, wherein the primary requests are received from first clients and the additional client request is received from a second client different than at least one of the first clients.

9. A method for distributing data to be extracted from a data store to a plurality of clients, the method comprising:

storing a set of client requests received from a plurality of clients, the set comprising a primary client request to provide data to a first client in association with one or more secondary client requests to provide data to one or more second clients different than the first client, the secondary client requests each matching the primary client request in accordance with a given criterion;

extracting data from the data store in accordance with the primary client request; and

distributing the extracted data to the first client requesting the primary data request and to the second client or clients requesting the secondary data request.

10. The method of claim 9, comprising storing a plurality of sets of client requests, each set comprising a primary client request for data in association with one or more secondary client requests which each match the primary client request in accordance with a given criterion.

11. The method of claim 10, comprising performing the steps of extracting and distributing data for each of the plurality of sets of client requests.

12. The method of claim 11, comprising pausing a first time interval between extracting and distributing each of the plurality of sets of client requests.

13. The method of claim 12, comprising determining a length of time required to extract and distribute data to all first and second clients and, if the determined time length exceeds a threshold, increasing the first time interval.

14. The method of claim 9, wherein the primary and secondary client requests each comprise a filter for extracting a subset of data from the data store, and wherein storing the set comprises identifying the set through the filter.

15. The method of claim 14, wherein extracting the data comprises extracting the data using the filter.

16. The method of claim 9, wherein at least one of the client requests in the set comprise a metric for summarizing data extracted from the data store, comprising summarizing the extracted data using the metric and distributing the summarized data to any first or second client whose client request comprises the metric.

19. The method of claim 18, wherein at least some of the client requests further contain a metric to be used to summarize data extracted from the data store, comprising:

establishing a metric subset of any set associated with a filter contained in a client request containing the metric, and

associating each of the client requests containing a metric with the metric subset associated with the metric contained in the client request.

17. A method for distributing data to be extracted from a data store to a plurality of clients, the method comprising:

ordering a plurality of client requests received from a plurality of clients into a set, the set comprising a primary client request to provide data to a first client in association with one or more secondary client requests to provide data to one or more second clients different than the first client, the secondary client requests each matching the primary client request in accordance with a given criterion;

extracting data from the data store in accordance with the primary client request; and

distributing the extracted data to the first client requesting the primary data request and to the second client or clients requesting the secondary data request.

18. A method for preparing to efficiently distribute data to be extracted from a data store to a plurality of clients in accordance with a plurality of client requests, each client request containing a filter, the method comprising:

establishing a plurality of sets of client requests, each set being uniquely associated with one of the filters; and

associating each of the client requests with the set associated with the filter contained in the client request.

20. A system for efficient distribution of network event data, the system comprising:

a data store containing data relating to events occurring on the network;

a library for storing client requests for data from the data store, the client requests identifying a plurality of clients and being ordered as one or more sets of client requests, each set containing one or more client requests matching a given criterion; and

a notification system for distributing to the plurality of clients data extracted from the data store in accordance with the one or more sets of client requests stored in the library.

21. The system of claim 20, wherein the notification system comprises a notifier program running a plurality of program threads.

22. The system of claim 20, comprising a plurality of data stores arranged in a cluster, wherein one data store comprises a master data store and one or more other data stores comprise slave data stores.

23. The system of claim 22, comprising means for synchronizing the network event data stored in the data stores between the master and slave data stores.

24. The system of claim 23, comprising a slave client request library associated with each slave data store and means for synchronizing the library for the master data store with the slave client request libraries.

25. The system of claim 24, comprising a slave notification system associated with each slave data store for, in the event of failure of the master data store or notification system, distributing to the plurality of clients data extracted from the slave data store in accordance with the one or more sets of client requests stored in the slave library.

26. The system of claim 25, wherein the master notification system distributes data extracted from the master data store to the one or more slave notification systems.

27. The system of claim 25, comprising means for maintaining an order of distributed data delivered to a first client to thereby enable the first client to properly order the distributed data in the event of a temporary failure of the first client.

28. The system of claim 27, wherein the first client is a gateway between a second client and the notification system.

29. The system of claim 27, wherein the means for maintaining the order comprises a sequence count inserted in the distributed data by the master and slave notification systems.