Internet usage data recording system and method with configurable data collector system

• US 7,024,468 B1
• Filed: 04/27/2000
• Issued: 04/04/2006
• Est. Priority Date: 04/27/2000
• Status: Expired due to Term

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Meyer, Jeffrey D., Ranous, Alexander C., Murray, Thomas A.
• Primary Examiner(s): Wiley, David
• Assistant Examiner(s): DELGADO, MICHAEL A
• Application Number:
  US09/560,509
• Time in Patent Office: 2,168 Days
• Field of Search: 705/40, 713/1, 717/104, 709/224, 709/220, 709/223
• US Class Current: 709/220
• CPC Class Codes: H04L 41/08 Configuration management of networks or network elements address allocation H04L61/50
• CPC Class Codes: H04L 43/02 Capturing of monitoring data
• CPC Class Codes: H04L 43/0876 Network utilisation, e.g. volume of load or congestion level
• CPC Class Codes: H04L 69/08 Protocols for interworking; Protocol conversion

Abstract

An internet usage data recording system and method is described. In one embodiment, the present invention provides a network usage recording system. The network usage recording system includes a configurable collector. The collector includes an encapsulator for reading a plurality of network data records from a network data source and converting the network data records to a plurality of normalized metered events. An aggregator is provided for processing the normalized metered events to create aggregated normalized metered events. A data storage system is provided, wherein the aggregator periodically stores the aggregated normalized metered events in the data storage system.

Claims

1. A network usage recording system comprising:

a collector including;

an encapsulator for reading a plurality of network data records from a network data source and converting the network data records to a plurality of normalized metered events;

an aggregator for processing the normalized metered events to create aggregated normalized metered events; and

a data storage system, wherein the aggregator periodically stores the aggregated normalized metered events in the data storage system; and

a configuration server in communication with the encapsulator, the aggregator and the data storage system, wherein the configuration server stores configuration data for the encapsulator, the aggregator, and the data storage system that determines whether the collector operates as a network data collector or a correlator collector.

2. The system of claim 1, wherein the aggregator includes volatile memory for temporary storage of the aggregated normalized metered events.

3. The system of claim 1, wherein the configuration server communicates the configuration data to the encapsulator, the aggregator and the data storage system at start-up.

4. The system of claim 1, further comprising a collector shell, wherein the configuration server communicates with the encapsulator, the aggregator and the data storage system via a collector shell.

5. The system of claim 1, further comprising a query manager in communication with the data storage system for querying the data storage system.

6. The system of claim 5, wherein the query manager is in communication with the data storage system via the aggregator.

7. The system of claim 1, further comprising a statistics log, wherein the statistics log is in communication with the encapsulator, the aggregator and the data storage system for logging statistical data.

8. The system of claim 7, further comprising a collector operator, wherein the collector operator communicates with the encapsulator, the aggregator, the data storage system and the statistics log to provide administrative access.

9. The system of claim 1, wherein the encapsulator further includes a parser for parsing network data received from the network data souce.

11. The system of claim 1, wherein the collector is configured as a correlator collector.

12. The system of claim 1, wherein the collector is configured to perform additional levels of data reduction and consolidation of data stored in other collectors.

10. The system 1, wherein the collector is configured as a network data collector.

13. A network usage recording system comprising:

a collector system including;

a collector shell;

a query manager;

an encapsulator;

an aggregator;

a data storage system; and

a configuration server in communication with the encapsulator, the aggregator and the data storage system, the configuration server stores configuration data for the collector that determines whether the collector system operates as a network data collector or a correlator collector.

14. The system of claim 13, wherein the configuration server communicates with the encapsulator, the aggregator and the data storage system via the collector shell.

15. A method for recording network usage comprising:

defining a collector including an encapsulator, an aggregator and a data storage system;

operating the encapsulator to read a plurality of network data records from a network data source and convert the network data records to a plurality of normalized metered events;

aggregating the plurality of normalized metered events to create a plurality of aggregated normalized metered events;

storing the aggregated normalized metered events in the data storage system at periodic intervals;

defining a configuration server in communication with the encapsulator, the aggregator and the data storage system; and

storing configuration data for the encapsulator, the aggregator and the data storage system in the configuration server where the configuration data determines whether the collector operates as a network data collector or a correlator collector.

16. The method of claim 15, further comprising defining the aggregator to include volatile memory and storing the aggregated normalized metered events temporarily in the volatile memory.

17. The method of claim 15, further comprising the step of transferring the configuration data to the encapsulator, the aggregator and the data storage system at start-up.

18. The method of claim 15, further comprising the step of defining a query manager in communication with the data storage system for managing queries of the data storage system.

19. The method of claim 15, further comprising the step of defining a statistics log in communication with the encapsulator, the aggregator and the data storage system, and collecting statistics associated with the encapsulator, the aggregator and the data storage system in the statistics log.

20. The method of claim 15, further comprising the step of parsing the network data records from the usage data source read by the encapsulator.

21. A computer readable medium containing instructions for controlling a computer system to perform a method for recording network usage comprising:

defining a collector including an encapsulator, an aggregator and a data storage system;

operating the encapsulator to read a plurality of network data records from a network data source and convert the network data records to a plurality of normalized metered events;

aggregating the plurality of normalized metered events to create a plurality of aggregated normalized metered events;

storing the aggregated normalized metered events in the data storage system at periodic intervals; and

defining a configuration server in communication with the encapsulator, the aggregator and the data storage system; and

storing configuration data for the encapsulator, the aggregator and the data storage system in the configuration server where the configuration data determines whether the collector operates a network data collector or a correlator collector.

22. A network usage recording system comprising:

a plurality of configurable collectors, where each collector configurable to operate as one of a plurality of collector types, the collector types including a data collector and a correlator collector;

a configuration server in communication with each configurable collector, where the configuration server stores configuration data for each configurable collector that determines the collector type for each collector, and once the configuration data is transferred to each configurable collector, each configurable collector becomes the collector type associated with the configuration data.

23. The system of claim 22, wherein the collector types include an aggregator collector.

24. The system of claim 22, where each configurable collector includes three configurable components:

an encapsulator, an aggregator and a data storage system.

25. The system of claim 24, where the configuration data includes encapsulator configuration data, aggregator configuration data and data storage system configuration data.

26. The system of claim 24, wherein the configuration data includes aggregator configuration data, and where the aggregator configuration data includes a flush policy, aggregation scheme and rules.

27. The system of claim 22, where each configurable collector independently queries the configuration server for configuration data, and the configuration server transfers the configuration data for each configurable collector to the corresponding configurable collector.

28. The system of claim 27, where each configurable collector queries the configuration server at start-up of the configurable collector.