Monitoring hierarchical container-based software systems

• US 9,535,794 B2
• Filed: 07/26/2013
• Issued: 01/03/2017
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Globalfoundries US Incorporated (GlobalFoundries, Inc.)
• Original Assignee: GlobalFoundries, Inc.
• Inventors: Barr, Arthur J., Hayton, Stuart J., Nicholson, Robert B., Thorne, Colin J.
• Primary Examiner(s): TRUONG, LOAN
• Application Number:
  US13/951,528
• Publication Number:
  US 20150033072A1
• Time in Patent Office: 1,257 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0706 the processing taking place on a specific hardware platform or in a specific software environment
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 11/1484 involving virtual machines

Abstract

A computer receives a heartbeat message that includes both program reset information and at least one segment, of the heartbeat message, that is configured to indicate whether or not a first program is functioning correctly. The computer determines if the heartbeat message includes the indication that the first program is functioning correctly. If the computer determines that the heartbeat message indicates that the first program is functioning correctly, then the computer sends a data stream to the first program.

Claims

1. A method of monitoring computer components of a hierarchical container-based software system executing on computer hardware, the computer components including at least one of, one or more application programs and one or more operating systems, the hierarchical container-based software system including a network of data flows being routed through the hierarchy of computer components, the method comprising:

a computer receiving an aggregation of single reset and network flow heartbeat messages from each level of the hierarchy indicating whether computer components at each respective hierarchy level are functioning correctly, the aggregation of single reset and network heartbeat messages including both computer component reset information and network data flow information;

the computer determining if the aggregation of single reset and network heartbeat messages includes a first single reset and network heartbeat message indicating that a first computer component at a first level of the hierarchy is functioning correctly;

responsive to the determination that the first single reset and network heartbeat message includes the indication that the first computer component is functioning correctly, the computer routing network data flows to the first computer component;

responsive to the determination that the first single reset and network heartbeat message does not include the indication that the first computer component is functioning correctly, the computer determining that the first computer component has failed; and

the computer initiating restart of all computer components in hierarchal levels below the first hierarchical level.

2. The method of claim 1, wherein if the first computer component is functioning correctly, the network data flow information indicates that the network data flows to the first computer component should be sustained.

3. The method of claim 1, the method further comprising:

responsive to the determination that the first computer component has failed;

the computer redirecting the data stream to a second computer component or to an application endpoint, wherein the application endpoint is a point of termination or completion in a computational process.

4. The method of claim 1, the method further comprising:

responsive to the determination that the first computer component has failed;

the computer initiating a restart of the first computer component.

5. The method of claim 1, the method further comprising:

the computer determining if the aggregation of single reset and network heartbeat messages does not include a single reset and network heartbeat message indicating whether a second computer component at a second level of the hierarchy is functioning correctly responsive to the determination that the aggregation of heartbeat messages does not include a heartbeat message indicating whether the second computer component is functioning correctly, the computer determining if a threshold number of heartbeat messages have not been received with the indication whether the second computer component is functioning correctly; and

responsive to the determination that the threshold number of heartbeat messages have not been received with the indication whether the second computer component is functioning correctly, the computer determining that the second computer component has failed.

6. The method of claim 1, wherein if the first computer component is not functioning correctly, the network data flow information indicates that the network data flows to the first computer component should be bypassed.

7. The method of claim 2, wherein the network data flow information indicates that the network data flows to the first computer component should be sustained comprises placing a network data flow identifier in a slot of the network heartbeat message.

8. The method of claim 6, wherein if the first computer component is not functioning correctly, the network data flow information indicating that the network data flows to the first computer component should be bypassed comprises a slot in the network heartbeat message for the first computer component not having a network data flow identifier.

9. The method of claim 6, wherein all network data flows for the first hierarchal level are bypassed and all network data flows in hierarchal levels below the first hierarchical level are bypassed.

10. A non-transitory computer program product for monitoring computer components of a hierarchical container-based software system executing on computer hardware, the computer components including at least one of, one or more application programs and one or more operating systems, the system including a network of data flows being routed through the hierarchy of computer components, the computer program product comprising:

one or more computer-readable storage media and program instructions stored on the one or more computer-readable storage media, the program instructions comprising;

program instructions to receive an aggregation of single reset and network flow heartbeat messages from each level of the hierarchy indicating whether computer components at each respective hierarchy level are functioning correctly, the aggregation of single reset and network heartbeat messages including both computer component reset information and network data flow information;

program instructions to determine if the aggregation of single reset and network heartbeat messages includes a first single reset and network heartbeat message indicating that a first computer component at a first level of the hierarchy is functioning correctly;

program instructions to respond to the determination that the first single reset and network heartbeat message includes the indication that the first computer component is functioning correctly, by routing network data flows to the first computer component;

program instructions to respond to the determination that the first single reset and network heartbeat message does not include the indication that the first computer component is functioning correctly, by determining that the first computer component has failed; and

program instructions to initiate restart of all computer components in hierarchal levels below the first hierarchical level.

11. The non-transitory computer program product claim 10, wherein if the first computer component is functioning correctly, the network data flow information indicates that the network data flows to the first computer component should be sustained.

12. The non-transitory computer program product of claim 10, further comprising:

responsive to the determination that the first single reset and network heartbeat message does not include the indication that the first computer component is functioning correctly, program instructions to determine that the first computer component has failed.

13. The non-transitory computer program product of claim 12, further comprising:

responsive to the determination that the first computer component has failed;

program instructions to redirect the data stream to a second computer component or to an application endpoint, wherein the application endpoint is a point of termination or completion in a computational process.

14. The non-transitory computer program product of claim 12, further comprising:

responsive to the determination that the first computer component has failed;

program instructions to initiate a restart of the first computer component.

15. The non-transitory computer program product of claim 10, further comprising:

program instructions to determine if the aggregation of single reset and network heartbeat messages does not include a single reset and network heartbeat message indicating whether a second computer component at a second level of the hierarchy is functioning correctly responsive to the determination that the aggregation of heartbeat messages does not include a heartbeat message indicating whether the second computer component is functioning correctly, program instructions to determine if a threshold number of heartbeat messages have not been received with the indication whether the second computer component is functioning correctly; and

responsive to the determination that the threshold number of heartbeat messages have not been received with the indication whether the computer component is functioning correctly, program instructions to determine that the second computer component has failed.

16. The non-transitory computer program product of claim 12, wherein if the first computer component is not functioning correctly, the network data flow information indicates that the network data flows to the first computer component should be bypassed.

17. A computer system for monitoring computer components of a hierarchical container-based software system executing on computer hardware, the computer components including at least one of, one or more application programs and one or more operating systems, the system including a network of data flows being routed through the hierarchy of computer components, the computer system comprising:

one or more computer processors;

one or more computer-readable storage media;

program instructions stored on the computer-readable storage media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions to receive an aggregation of single reset and network flow heartbeat messages from each level of the hierarchy indicating whether computer components at each respective hierarchy level are functioning correctly, the aggregation of single reset and network heartbeat messages including both computer component reset information and network data flow information;

program instructions to determine if the aggregation of single reset and network heartbeat messages includes a first single reset and network heartbeat message indicating that a first computer component at a level of the hierarchy is functioning correctly;

program instructions to respond to the determination that the first single reset and network heartbeat message includes the indication that the first computer component is functioning correctly, by routing network data flows to the first computer component;

program instructions to respond to the determination that the first single reset and network heartbeat message does not include the indication that the first computer component is functioning correctly, by determining that the first computer component has failed; and

program instructions to initiate restart of all computer components in hierarchal levels below the first hierarchical level.

18. The computer system of claim 17, wherein if the first computer component is functioning correctly, the network data flow information indicates that the network data flows to the first computer component should be sustained.

19. The computer system of claim 17, further comprising:

responsive to the determination that the first computer component has failed;

program instructions to redirect the data stream to a second computer component or to an application endpoint, wherein the application endpoint is a point of termination or completion in a computational process.

20. The computer system of claim 17, further comprising:

responsive to the determination that the first computer component has failed;

program instructions to initiate a restart of the computer component program.

21. The computer system of claim 17, further comprising:

program instructions to determine if the aggregation of single reset and network heartbeat messages does not include a single reset and network heartbeat message indicating whether a second computer component at a second level of the hierarchy is functioning correctly responsive to the determination that the aggregation of heartbeat messages does not include a heartbeat message indicating whether the second computer component is functioning correctly, program instructions to determine if a threshold number of single reset and network heartbeat messages have not been received with the indication whether the second computer component is functioning correctly; and

responsive to the determination that the threshold number of heartbeat messages have not been received with the indication whether the second computer component is functioning correctly, program instructions to determine that the second computer component has failed.

22. The computer system of claim 17, wherein if the first computer component is not functioning correctly, the network data flows to the first computer component should be bypassed.

23. The computer system of claim 18, wherein the network data flow information indicates that the network data flows to the first computer component should be sustained comprises placing a network data flow identifier in a slot of the network heartbeat message.

24. The computer system of claim 22, wherein if the first computer component is not functioning correctly, the network data flow information indicating that the network data flows to the first computer component should be bypassed comprises a slot in the network heartbeat message for the first computer component not having a network data flow identifier.

25. The computer system of claim 22, wherein all network data flows for the first hierarchal level are bypassed and all network data flows in hierarchal levels below the first hierarchical level are bypassed.