Orchestration of software applications upgrade using automatic hang detection

• US 10,437,578 B2
• Filed: 07/13/2017
• Issued: 10/08/2019
• Est. Priority Date: 10/29/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Hu, Fang, Vong, Chi Kin, Thiagarajan, Bhuvaneswari, Richardson, Tim, Lu, Jinghuan Jay
• Primary Examiner(s): Vo, Ted T.
• Application Number:
  US15/649,440
• Publication Number:
  US 20170308373A1
• Time in Patent Office: 817 Days
• Field of Search: 717127-128, 717168-178
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416

Abstract

In an upgrade infrastructure performing an overall upgrade operation comprising multiple upgrade processes being executed, possibly concurrently, on multiple hosts for upgrading one or more software applications hosted by hosts, automated hang detection mechanisms are disclosed for quickly, efficiently, and automatically detecting when one or more of the upgrade process are in a hang state. Different hang detection techniques are described including a metadata-driven hang detection mechanism and a code-driven hang detection mechanism.

Claims

1. A method comprising:

monitoring execution of an upgrade process on a host machine, the upgrade process upgrading a software application on the host machine, the upgrade process including a plurality of tasks, the plurality of tasks including at least a first task and a second task;

determining, based upon the monitoring, a time of execution for the first task, wherein the time of execution for the first task is different from a time of execution for the second task;

determining a reference execution time specified for the first task, wherein the reference execution time specified for the first task is different from a reference execution time specified for the second task;

determining a latency tolerance time specified for the first task;

determining, based on the time of execution for the first task and the latency tolerance time specified for the first task, that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time;

upon determining that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time, determining that the upgrade process on the host machine is to be indicated as being in a hang state; and

generating an alert message indicating the upgrade process executing on the host machine is in the hang state.

2. The method of claim 1, wherein the upgrade process is associated with an upgrade reference execution time and an upgrade latency tolerance time for the upgrade process.

3. The method of claim 2, wherein generating the alert message is done before a total of the upgrade reference execution time and the upgrade latency tolerance time for the upgrade process.

4. The method of claim 2, wherein the upgrade reference execution time for the upgrade process is different for different host machines.

5. The method of claim 1, wherein the reference execution time for the first task is different for different host machines.

6. The method of claim 1, wherein the monitoring comprises monitoring the execution of the first task.

7. A system comprising:

one or more processors; and

a non-transitory computer-readable medium including instructions that, when executed by the one or more processors, cause the one or more processors to;

monitor execution of an upgrade process on a host machine, the upgrade process upgrading a software application on the host machine, the upgrade process including a plurality of tasks, the plurality of tasks including at least a first task and a second task;

determine, based upon the monitoring, a time of execution for the first task, wherein the time of execution for the first task is different from a time of execution for the second task;

determine a reference execution time specified for the first task, wherein the reference execution time specified for the first task is different a reference execution time specified for the second task;

determine a latency tolerance time specified for the first task;

determine, based on the time of execution for the first task and the latency tolerance time specified for the first task, that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time; and

upon determining that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time, determining that the upgrade process on the host machine is to be indicated as being in a hang state, and generate an alert message indicating the upgrade process executing on the host machine is in the hang state.

8. The system of claim 7, wherein the upgrade process is associated with an upgrade reference execution time and an upgrade latency tolerance time for the upgrade process.

9. The system of claim 8, wherein generating the alert message is done before a total of the upgrade reference execution time and the upgrade latency tolerance time for the upgrade process.

10. The system of claim 7, wherein the reference execution time for the first task is different for different host machines.

11. The system of claim 7, wherein the monitoring comprises monitoring the execution of the first task.

12. A non-transitory computer-readable storage medium storing a plurality of instructions executable by one or more processors, the plurality of instructions when executed by the one or more processors cause the one or more processors to perform processing comprising:

monitoring execution of an upgrade process on a host machine, the upgrade process upgrading a software application on the host machine, the upgrade process including a plurality of tasks, the plurality of tasks including at least a first task and a second task;

determining, based upon the monitoring, a time of execution for the first task, wherein the time of execution for the first task is different from a time of execution for the second task;

determining a reference execution time specified for the first task, wherein the reference execution time specified for the first task is different a reference execution time specified for the second task;

determining a latency tolerance time specified for the first task;

determining, based on the time of execution for the first task and the latency tolerance time specified for the first task, that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time;

upon determining that the first task has not completed execution for a period of time that is greater than a total of the reference execution time and the latency tolerance time, determining that the upgrade process on the host machine is to be indicated as being in a hang state; and

generating an alert message indicating the upgrade process executing on the host machine is in the hang state.

13. The non-transitory computer-readable storage medium of claim 12, wherein the upgrade process is associated with an upgrade reference execution time and an upgrade latency tolerance time for the upgrade process.

14. The non-transitory computer-readable storage medium of claim 13, wherein generating the alert message is done before a total of the upgrade reference execution time and the upgrade latency tolerance time for the upgrade process.

15. The non-transitory computer-readable storage medium of claim 13, wherein the upgrade reference execution time for the upgrade process is different for different host machines.

16. The non-transitory computer-readable storage medium of claim 12, wherein the reference execution time for the first task is different for different host machines.

17. The non-transitory computer-readable storage medium of claim 12, wherein the monitoring comprises monitoring the execution of the first task.