System and method for managing software upgrades in a distributed computing system

• US 20030233648A1
• Filed: 06/12/2002
• Published: 12/18/2003
• Est. Priority Date: 06/12/2002
• Status: Active Grant

• Current Assignee: Agami Systems, Inc.
• Original Assignee: Agami Systems, Inc.
• Inventors: Earl, William J., Ekambaram, Dhanabal, Hahn, Terry G.
• Granted Patent: US 6,966,058 B2
• Time in Patent Office: Days
• US Class Current: 717/176
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57

Abstract

A system and method for managing software upgrades in a distributed computing system. The distributed computing system may include a plurality of nodes which provide one or more fault-tolerant services. The system and method perform software upgrades in a sequential or “rolling” manner (e.g., node by node). The rolling upgrade process allows all services and data of the distributed computing system to remain operable and available throughout the upgrade process.

Claims

1. A system for managing a software upgrade in a distributed computing system having a plurality of nodes that provide at least one fault-tolerant service, the system comprising:

at least one server which is communicatively connected to the plurality of nodes and which is adapted to receive a software release, and to upgrade each of the plurality of nodes with the software release in a sequential manner, whereby the at least one fault-tolerant service remains available while the software upgrade is in progress.

2. The system of claim 1 further comprising:

a service which is adapted to notify the at least one server when a first copy of a fault-tolerant service becomes unavailable; and

wherein the at least one server is adapted to assign a protected status to a node including any portion of a surviving copy of the fault-tolerant service, the protected status being effective to prevent the node from being upgraded.

3. The system of claim 2 wherein service is further adapted to notify the at least one server when the first copy of the fault-tolerant service is restored, and wherein the at least one server is adapted to remove the protected status of the affected nodes once fault-tolerant status is restored.

4. The system of claim 1 wherein the at least one server is further adapted to determine whether the software release is compatible with the current software running on the plurality of nodes and to upgrade the plurality of nodes only if the software release is compatible with the current software.

5. The system of claim 4 further comprising:

a configuration database which is communicatively coupled to the at least one server and which stores information regarding the plurality of nodes.

6. The system of claim 5 wherein the at least one server compares compatibility information contained in the software release to information identifying the current software running on the plurality of nodes, which is stored in the configuration database, to determine whether the software release is compatible with the current software running on the plurality of nodes.

7. The system of claim 4 wherein the at least one server is further adapted to detect when a first upgrade to a compatible software is in progress, and to upgrade the plurality of nodes to the software release once the first upgrade is complete.

8. The system of claim 1 wherein the at least one server is adapted to upgrade a node by loading the software onto the node and rebooting the node.

9. The system of claim 1 wherein the at least one server comprises a primary server and a backup server.

10. The system of claim 1 further comprising a plurality of agents which are respectively disposed on the plurality of nodes and which are adapted to assist in upgrading the nodes with the software under remote control of the at least one server.

11. The system of claim 1 further comprising an interface which is communicatively connected to the at least one server, and which is adapted to allow a user to enter an upgrade request, effective to cause the at least one server to initiate an upgrade to the software release.

12. The system of claim 11 wherein the interface is further adapted to allow a user to enter a downgrade request, effective to cause the at least one server to downgrade the software currently running on the plurality of nodes in a sequential manner, whereby none of the fault-tolerant services becomes unavailable while the downgrade is in progress.

13. A method for managing a software upgrade in a distributed computing system having a plurality of nodes that provide at least one fault-tolerant service, the method comprising the steps of:

receiving a new software release; and

upgrading each of the plurality of nodes with the new software release in a sequential manner, whereby the at least one fault-tolerant service remains available while the software upgrade is in progress.

14. The method of claim 13 further comprising the steps of:

determining whether a first copy of a fault-tolerant service has become unavailable; and

preventing any node having a surviving copy of the fault-tolerant service from being upgraded while the first copy of the fault-tolerant service is unavailable.

15. The method of claim 14 further comprising the steps of:

detecting that a previously unavailable copy of a fault-tolerant service has recovered; and

upgrading any node having the surviving copy of the fault-tolerant service.

16. The method of claim 13 further comprising the step of:

determining whether the new software release is compatible with a current software release that is being run on the distributed computing system; and

performing the software upgrade only if the new software release is compatible with the current software release that is being run on the distributed computing system.

17. The method of claim 16 further comprising the step of:

determining whether a first upgrade to a compatible release is in progress;

waiting for the first upgrade to be completed; and

performing the software upgrade to the new software release.

18. The method of claim 13 wherein the step of upgrading a node includes loading the new software release onto the node and rebooting the node.

19. The method of claim 13 wherein the new software release comprises a patch release.

20. A method for managing a software upgrade in a distributed file system having a plurality of nodes, which provide a plurality of fault-tolerant services, comprising:

receiving a new software release;

determining whether the new software release is compatible with a current release running on the distributed file system;

initiating a rolling upgrade process if the new software release is compatible;

performing the rolling upgrade process by sequentially loading and rebooting each of the plurality of nodes with the new software release; and

ensuring that the plurality of fault-tolerant services remains available throughout the rolling upgrade process.

21. The method of claim 20 wherein the plurality of fault-tolerant services includes services selected from the group consisting of bit file storage services, metadata services and gateway services.

22. The method of claim 21 wherein the step of ensuring that the plurality of fault-tolerant services remains available throughout the rolling upgrade process, includes the steps of:

determining whether a first copy of a fault-tolerant service has become unavailable; and

preventing any node having a surviving copy of the fault-tolerant service from being upgraded while the first copy of the fault-tolerant service is unavailable.