INFRASTRUCTURE MANAGEMENT SYSTEM HAVING SCALABLE STORAGE ARCHITECTURE

• US 20170262508A1
• Filed: 03/09/2016
• Published: 09/14/2017
• Est. Priority Date: 03/09/2016
• Status: Abandoned Application

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Inventors: Garcia, Julio Cesar
• Application Number:
  US15/065,677
• Publication Number:
  US 20170262508A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/067 using time frame reporting
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route

Abstract

Techniques for processing infrastructure management data within a service domain are disclosed. The service domain includes service nodes that monitor respective target entities and that generate measurement data for target entities within the service domain. A database engine reads a serial measurement data stream that is generated by at least one of the service nodes. In response to reading the serial measurement data stream, the database engine generates a sequence of measurement data records that each contain target entity measurement data. The generated measurement data records further include metadata summary values and are recorded in a measurement database table in a logical access sequence that corresponds to the order in which the service nodes generated the measurement data.

Claims

1. A method for processing infrastructure management data within a service domain which include service nodes that monitor respective target entities, said method comprising:

generating, by the service nodes, measurement data for target entities within the service domain;

reading a serial measurement data stream that is generated by at least one of the service nodes;

in response to reading the serial measurement data stream, generating a sequence of measurement data records that each contain target entity measurement data, wherein said generating the sequence of measurement data records includes inserting a metadata summary value into at least one of the sequence of measurement data records; and

recording the sequence of measurement data records, including the at least one measurement data record having the inserted metadata summary value, into a database table in a logical access sequence that corresponds to the order in which the service nodes generated the measurement data.

2. The method of claim 1, wherein said generating the sequence of measurement data records further comprises assigning primary access keys to at least one of the measurement data records, including the at least one measurement data record having the inserted metadata summary value, wherein each of the primary access keys associates a performance metric ID with a timestamp.

3. The method of claim 2, wherein said generating the sequence of measurement data records further comprises assigning secondary access keys to respective subsets of the measurement data records, wherein the secondary access keys specify a monitoring collection period.

4. The method of claim 1, further comprising:

determining the metadata summary value for an n^th measurement data record within the sequence of measurement data records based on data values for measurement data records generated prior to generating the n^th measurement data record.

5. The method of claim 4, wherein determining the metadata summary value for the n^th measurement data record comprises:

parsing probe output data objects in the serial measurement data stream to identify measurement data for target entities;

processing the measurement data and statistical relations between the measurement data received in different probe output data objects using an inline data processing component; and

generating the metadata summary value in accordance with the processed measurement data and statistical relations for an m^th probe output data object and a specified number of probe packets preceding the m^th probe output data object in the serial measurement data stream.

6. The method of claim 1, wherein the service nodes each include monitoring probes that generate probe packets and a collection hub that generates the measurement data stream on a messaging bus as a sequence of the probe packets, and wherein at least one of the service nodes includes a local database engine, said reading a serial measurement data stream comprising reading, by the local database engine, a serial measurement data stream generated by a respective one or more of the service nodes.

7. The method of claim 6, wherein the database table is stored in a database storage cluster comprising multiple interconnected storage nodes, said method further comprising:

the local database engine transmitting the measurement data streams generated by the respective service nodes to the database storage cluster;

at least one of the service nodes monitoring input/output (I/O) capacity of the database storage cluster during said transmitting of the measurement data streams; and

adjusting the number of storage nodes in the database storage cluster based on the monitored I/O capacity of the database storage cluster.

8. One or more non-transitory machine-readable media having program code for processing infrastructure management data within a service domain which include service nodes that monitor respective target entities stored therein, the program code to:

generate measurement data for target entities within the service domain;

read a serial measurement data stream that is generated by at least one of the service nodes;

in response to reading the serial measurement data stream, generate a sequence of measurement data records that each contain target entity measurement data, wherein said generating a sequence of measurement data records includes inserting a metadata summary value into at least one of the sequence of measurement data records; and

record the sequence of measurement data records, including the at least one measurement data record having the inserted metadata summary value, into a database table in a logical access sequence that corresponds to the order in which the service nodes generated the measurement data.

9. The non-transitory machine-readable media of claim 8, wherein said generating a sequence of measurement data records further comprises assigning primary access keys to at least one of the measurement data records, including the at least one measurement data record having the inserted metadata summary value, wherein each of the primary access keys associates a performance metric ID with a timestamp.

10. The non-transitory machine-readable media of claim 9, wherein said generating a sequence of measurement data records further comprises assigning secondary access keys to respective subsets of the measurement data records, wherein the secondary access keys specify a monitoring collection period.

11. The non-transitory machine-readable media of claim 8, further comprising program code to:

determine the metadata summary value for an n^th measurement data record within the sequence of measurement data records based on data values for measurement data records generated prior to generating the n^th measurement data record.

12. The non-transitory machine-readable media of claim 11, wherein the program code to determine the metadata summary value for the n^th measurement data record comprises program code to:

parse probe output data objects in the serial measurement data stream to identify measurement data for target entities;

process the measurement data and statistical relations between the measurement data received in different probe output data objects using an inline data processing application; and

generate the metadata summary value in accordance with the processed measurement data and statistical relations for the m^th probe output data object and a specified number of probe packets preceding the m^th probe output data object in the serial measurement data stream.

13. The non-transitory machine-readable media of claim 8, wherein the service nodes each include monitoring probes that generate probe packets and a collection hub that generates the measurement data stream on a messaging bus as a sequence of the probe packets, and wherein at least one of the service nodes includes a local database engine, said program code to read a serial measurement data stream comprising program code to read, by the local database engine, a serial measurement data stream generated by a respective one or more of the service nodes.

14. The non-transitory machine-readable media of claim 13, wherein the database table is stored in a database storage cluster comprising multiple interconnected storage nodes, said non-transitory machine-readable media further comprising program code to:

transmit the measurement data streams generated by the respective service nodes from the local database engine to the database storage cluster;

monitor input/output (I/O) capacity of the database storage cluster during said transmitting of the measurement data streams; and

adjust the number of storage nodes in the database storage cluster based on the monitored I/O capacity of the database storage cluster.

15. An apparatus comprising:

a processor; and

a machine-readable medium having program code executable by the processor to cause the apparatus to,

generate measurement data for target entities within the service domain;

read a serial measurement data stream that is generated by at least one of the service nodes;

in response to reading the serial measurement data stream, generate a sequence of measurement data records that each contain target entity measurement data, wherein said generating a sequence of measurement data records includes inserting a metadata summary value into at least one of the sequence of measurement data records; and

record the sequence of measurement data records, including the at least one measurement data record having the inserted metadata summary value, into a database table in a logical access sequence that corresponds to the order in which the service nodes generated the measurement data.

16. The apparatus of claim 15, wherein the program code executable by the processor to cause the apparatus to generate a sequence of measurement data records comprises program code executable by the processor to cause the apparatus to:

assign primary access keys to at least one of the measurement data records, including the at least one measurement data record having the inserted metadata summary value, wherein each of the primary access keys associates a performance metric ID with a timestamp; and

assign secondary access keys to respective subsets of the measurement data records, wherein the secondary access keys specify a monitoring collection period

17. The apparatus of claim 15, further comprising program code executable by the processor to cause the apparatus to:

determine the metadata summary value for an n^th measurement data record within the sequence of measurement data records based on data values for measurement data records generated prior to generating the n^th measurement data record.

18. The apparatus of claim 17, wherein the program code executable by the processor to cause the apparatus to determine the metadata summary value for the n^th measurement data record comprises program code executable by the processor to cause the apparatus to:

parse probe output data objects in the serial measurement data stream to identify measurement data for target entities;

process the measurement data and statistical relations between the measurement data received in different probe output data objects using an inline data processing application; and

generate the metadata summary value in accordance with the processed measurement data and statistical relations for the m^th probe output data object and a specified number of probe packets preceding the m^th probe output data object in the serial measurement data stream.

19. The apparatus of claim 15, wherein the service nodes each include monitoring probes that generate probe packets and a collection hub that generates the measurement data stream on a messaging bus as a sequence of the probe packets, and wherein at least one of the service nodes includes a local database engine, said program code executable by the processor to cause the apparatus to read a serial measurement data stream comprising program code executable by the processor to cause the apparatus to read, by the local database engine, a serial measurement data stream generated by a respective one or more of the service nodes.

20. The apparatus of claim 19, wherein the database table is stored in a database storage cluster comprising multiple interconnected storage nodes, and further comprising program code executable by the processor to cause the apparatus to:

transmit the measurement data streams generated by the respective service nodes from the local database engine to the database storage cluster;

monitor input/output (I/O) capacity of the database storage cluster during said transmitting of the measurement data streams; and

adjust the number of storage nodes in the database storage cluster based on the monitored I/O capacity of the database storage cluster.