Real-time monitoring of computer system processor and transaction performance during an ongoing performance test

• US 10,158,549 B2
• Filed: 09/18/2015
• Issued: 12/18/2018
• Est. Priority Date: 09/18/2015
• Status: Active Grant

• Current Assignee: FMR LLC
• Original Assignee: FMR LLC
• Inventors: Paul, Kanwar Gaurav
• Primary Examiner(s): Chea, Philip J
• Assistant Examiner(s): MCADAMS, BRAD
• Application Number:
  US14/858,731
• Publication Number:
  US 20170085458A1
• Time in Patent Office: 1,187 Days
• CPC Class Codes: H04L 41/046 comprising network management agents or mobile agents therefor
• CPC Class Codes: H04L 43/04 Processing captured monitoring data, e.g. for logfile generation
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 43/12 Network monitoring probes

Abstract

Methods and apparatuses are described for real-time monitoring of computer system processor and transaction performance during an ongoing performance test. A plurality of monitoring agent processes executing on a processor of a first computing device in a computer system testing environment extracts performance data relating to the ongoing performance test in the computer system testing environment by scraping CPU usage data, analyzing submitted transaction data as stored in a transaction log file and executing search queries against a transaction database in the computing system testing environment to generate a transaction lifecycle dataset. A second computing device retrieves files stored at a predetermined location and generates one or more graphical representations of the performance data contained in the files for display, the graphical representations including at least one interactive chart to indicate a relationship between the CPU usage data, the submitted transaction data, and the transaction lifecycle dataset.

Claims

1. A computerized method for real-time monitoring of computer system processor and transaction performance during an ongoing performance test, the method comprising

extracting, by a plurality of monitoring agent processes executing on a processor of a first computing device in a computer system testing environment, performance data relating to the ongoing performance test in the computer system testing environment comprising:

scraping, by a first monitoring agent process, CPU usage data from an on-screen performance view of a plurality of test systems engaged in the ongoing performance test for storage in a first file at a predetermined location on the computing device, the scraping comprising;

establishing a connection to a third computing device,

automatically submitting authentication credentials to the third computing device using emulated keyboard input,

issuing a print screen command to the third computing device that, when executed by the third computing device, generates a text file containing text displayed on a screen coupled to the third computing device, and

scanning the text file for position-specific characters associated with the CPU usage data, extract the CPU usage data from the text file, and store the CPU usage data in the first file;

analyzing, by a second monitoring agent process, submitted transaction data as stored in a transaction log file in the computer system testing environment and selecting a subset of the submitted transaction data for storage in a second file at the predetermined location, wherein selection of the subset is based upon one or more transaction types; and

executing, by a third monitoring agent process, search queries against a transaction database in the computing system testing environment to generate a transaction lifecycle dataset for each transaction that identifies at least a starting time and an ending time of one or more events that comprise the transaction and storing the transaction lifecycle dataset in a third file at the predetermined location;

retrieving, by a second computing device outside of the computer system testing environment, the files stored at the predetermined location in the computer system testing environment; and

generating, by the second computing device, one or more graphical representations of the performance data contained in the files for display, the graphical representations including at least one interactive chart to indicate a relationship between the CPU usage data, the subset of submitted transaction data, and the transaction lifecycle dataset.

2. The method of claim 1, wherein the step of analyzing submitted transaction data as stored in a transaction log file comprises:

scanning, by the second monitoring agent process, rows of data stored in the transaction log file to identify rows that match the one or more transaction types, wherein the identified rows are the subset of the submitted transaction data;

extracting, by the second monitoring agent process, data elements from the transaction log file for each of the rows in the subset; and

storing, by the second monitoring agent process, the extracted data elements in a character delimited format in the second file.

3. The method of claim 1, wherein the interactive chart comprises a line graph for each of the CPU usage data, the subset of submitted transaction data, and the transaction lifecycle dataset.

4. The method of claim 1, wherein the plurality of monitoring agent processes complete their processing in under five CPU cycles of the processor of the first computing device.

5. The method of claim 1, wherein the step of retrieving the files stored at the predetermined location comprises:

establishing, by the second computing device, a file transfer session with the first computing device;

pulling, by the second computing device, the first file, the second file, and the third file from the predetermined location; and

closing, by the second computing device, the file transfer session.

6. The method of claim 1, further comprising performing the steps of extracting the performance data and retrieving the files at predetermined regular intervals and updating the graphical representations in real time as soon as the steps of extracting the performance data and retrieving the files are performed.

7. The method of claim 1, wherein the ongoing performance test is a stress test operable to initiate a number of transactions that is multiple times greater than an expected number of transactions.

8. The method of claim 1, wherein the plurality of monitoring agent processes comprise cyclic mainframe jobs that define units of work to be performed by the processor of the first computing device against one or more computing devices in the computing system testing environment.

9. A system for real-time monitoring of computer system processor and transaction performance during an ongoing performance test, the system comprising:

a plurality of monitoring agent processes executing on a processor of a first computing device in a computer system testing environment configured to extract performance data relating to the ongoing performance test in the computer system testing environment, wherein

a first monitoring agent process is configured to scrape CPU usage data from an on-screen performance view of a plurality of test systems engaged in the ongoing performance test for storage in a first file at a predetermined location on the computing device, the scraping comprising;

establishing a connection to a third computing device,

automatically submitting authentication credentials to the third computing device using emulated keyboard input,

issuing a print screen command to the third computing device that, when executed by the third computing device, generates a text file containing text displayed on a screen coupled to the third computing device, and

scanning the text file for position-specific characters associated with the CPU usage data, extract the CPU usage data from the text file, and store the CPU usage data in the first file,

a second monitoring agent process is configured to analyze submitted transaction data as stored in a transaction log file in the computer system testing environment and selecting a subset of the submitted transaction data for storage in a second file at the predetermined location, wherein selection of the subset is based upon one or more transaction types, and

a third monitoring agent process is configured to search queries against a transaction database in the computing system testing environment to generate a transaction lifecycle dataset for each transaction that identifies at least a starting time and an ending time of one or more events that comprise the transaction and storing the transaction lifecycle dataset in a third file at the predetermined location;

a second computing device outside of the computer system testing environment configured to retrieve the files stored at the predetermined location in the computer system testing environment and generate one or more graphical representations of the performance data contained in the files for display, the graphical representations including at least one interactive chart to indicate a relationship between the CPU usage data, the subset of submitted transaction data, and the transaction lifecycle dataset.

10. The system of claim 9, wherein the second monitoring agent process is configured to:

scan rows of data stored in the transaction log file to identify rows that match the one or more transaction types, wherein the identified rows are the subset of the submitted transaction data;

extract data elements from the transaction log file for each of the rows in the subset; and

store the extracted data elements in a character delimited format in the second file.

11. The system of claim 9, wherein the interactive chart comprises a line graph for each of the CPU usage data, the subset of submitted transaction data, and the transaction lifecycle dataset.

12. The system of claim 9, wherein the plurality of monitoring agent processes complete their processing in under five CPU cycles of the processor of the first computing device.

13. The system of claim 9, wherein the second computing device is configured to:

establish a file transfer session with the first computing device;

pull the first file, the second file, and the third file from the predetermined location; and

close the file transfer session.

14. The system of claim 9, wherein the steps of extracting the performance data and retrieving the files are performed at predetermined regular intervals and the graphical representations are updated in real time based upon the most-recently extracted performance data as soon as the steps of extracting the performance data and retrieving the files are performed.

15. The system of claim 9, wherein the ongoing performance test is a stress test operable to initiate a number of transactions that is multiple times greater than an expected number of transactions.

16. The system of claim 9, wherein the plurality of monitoring agent processes comprise cyclic mainframe jobs that define units of work to be performed by the processor of the first computing device against one or more computing devices in the computing system testing environment.

17. A computer program product, tangibly embodied in a non-transitory computer readable storage device for real-time monitoring of computer system processor and transaction performance during an ongoing performance test, the computer program product including instructions operable to cause a plurality of monitoring agent processes executing on a processor of a first computing device in a computer system testing environment to extract performance data relating to the ongoing performance test in the computer system testing environment, wherein

a first monitoring agent process is configured to scrape CPU usage data from an on-screen performance view of a plurality of test systems engaged in the ongoing performance test for storage in a first file at a predetermined location on the computing device, the scraping comprising:

establishing a connection to a third computing device,

automatically submitting authentication credentials to the third computing device using emulated keyboard input,

issuing a print screen command to the third computing device that, when executed by the third computing device, generates a text file containing text displayed on a screen coupled to the third computing device, and

scanning the text file for position-specific characters associated with the CPU usage data, extract the CPU usage data from the text file, and store the CPU usage data in the first file,

a second monitoring agent process is configured to analyze submitted transaction data as stored in a transaction log file in the computer system testing environment and selecting a subset of the submitted transaction data for storage in a second file at the predetermined location, wherein selection of the subset is based upon one or more transaction types, and

a third monitoring agent process is configured to search queries against a transaction database in the computing system testing environment to generate a transaction lifecycle dataset for each transaction that identifies at least a starting time and an ending time of one or more events that comprise the transaction and storing the transaction lifecycle dataset in a third file at the predetermined location;

wherein the computer program product includes instructions operable to cause a second computing device outside of the computer system testing environment to retrieve the files stored at the predetermined location in the computer system testing environment and generate one or more graphical representations of the performance data contained in the files for display, the graphical representations including at least one interactive chart to indicate a relationship between the CPU usage data, the subset of submitted transaction data, and the transaction lifecycle dataset.