Intelligent rolling upgrade for data storage systems

• US 8,108,734 B2
• Filed: 11/02/2009
• Issued: 01/31/2012
• Est. Priority Date: 11/02/2009
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Andrews, Benjamin L., Khandelwal, Nikhil, Ciaravella, Anthony J., Dain, Joseph W.
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Manoskey, Joseph D
• Application Number:
  US12/610,517
• Publication Number:
  US 20110107135A1
• Time in Patent Office: 820 Days
• Field of Search: 714/4.1, 714/4.3, 714/4.4, 714/38.1, 714/42, 714/47.1, 714/47.2, 714/47.3, 717/168, 717/171
• US Class Current: 714/47.1
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/30 Monitoring
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G06F 8/656 while running
• CPC Class Codes: H04L 41/082 the condition being updates or upgrades of network functionality

Abstract

Various method, system, and computer program product embodiments for facilitating upgrades in a computing storage environment are provided. In one such embodiment, one of an available plurality of rolling upgrade policies registering at least one selectable upgrade parameter for an upgrade window is selected. A node down tolerance factor is set for at least one node in the computing storage environment. The node down tolerance factor specifies a percentage of elements of the at least one node taken offline to apply the selected one of the available plurality of rolling upgrade policies during the upgrade window.

Claims

1. A method for facilitating upgrades in a computing storage environment by a processor in communication with a memory device, comprising:

defining a parallel upgrade confidence factor corresponding to a proportion of node elements of the computing storage environment allowed to concurrently receive an upgrade procedure; and

performing the upgrade procedure on at least one of the node elements, wherein pursuant to successful completion of the upgrade procedure on the at least one of the node elements, the parallel upgrade confidence factor is increased to allow additional node elements of the computing storage environment to concurrently receive the upgrade procedure.

2. The method of claim 1, further including pursuant to performing the upgrade procedure on another one of the at least one of the node elements, determining if the parallel upgrade confidence factor has been exceeded.

3. The method of claim 2, wherein if the parallel upgrade confidence factor is determined to have been exceeded, limiting concurrent performance of the upgrade procedure on the node elements to be consistent with the parallel upgrade confidence factor.

4. A system for facilitating upgrades in a computing storage environment, comprising:

a processor in communication with a memory device, the processor operational on a management node in the computing storage environment, wherein the processor is adapted for;

defining a parallel upgrade confidence factor corresponding to a proportion of node elements of the computing storage environment allowed to concurrently receive an upgrade procedure, and

performing the upgrade procedure on at least one of the node elements, wherein pursuant to successful completion of the upgrade procedure on the at least one of the node elements, the parallel upgrade confidence factor is increased to allow additional node elements of the computing storage environment to concurrently receive the upgrade procedure.

5. The system of claim 4, wherein the processor is further adapted for, pursuant to performing the upgrade procedure on another one of the at least one of the node elements, determining if the parallel upgrade confidence factor has been exceeded.

6. The system of claim 5, wherein the processor is further adapted for, if the parallel upgrade confidence factor is determined to have been exceeded, limiting concurrent performance of the upgrade procedure on the node elements to be consistent with the parallel upgrade confidence factor.

7. A computer program product for facilitating upgrades in a computing storage environment by a processor in communication with a memory device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for defining a parallel upgrade confidence factor corresponding to a proportion of node elements of the computing storage environment allowed to concurrently receive an upgrade procedure; and

a second executable portion for performing the upgrade procedure on at least one of the node elements, wherein pursuant to successful completion of the upgrade procedure on the at least one of the node elements, the parallel upgrade confidence factor is increased to allow additional node elements of the computing storage environment to concurrently receive the upgrade procedure.

8. The computer program product of claim 7, further including a third executable portion for, pursuant to performing the upgrade procedure on another one of the at least one of the node elements, determining if the parallel upgrade confidence factor has been exceeded.

9. The computer program product of claim 8, further including a fourth executable portion for, if the parallel upgrade confidence factor is determined to have been exceeded, limiting concurrent performance of the upgrade procedure on the node elements to be consistent with the parallel upgrade confidence factor.

10. A method for facilitating upgrades in a computing storage environment by a processor in communication with a memory device, comprising:

examining a current number of node elements of the computing storage environment having failed an upgrade procedure as an error propagation count, comparing the error propagation count against a predefined error propagation tolerance value; and

reporting if the predefined error propagation tolerance value is exceeded to provide a health status of the upgrade procedure, wherein if a subsequent performance of the upgrade procedure on an additional node element is unsuccessful, the error propagation count is incremented and a critical error is reported so as to avoid propagation of an upgrade error.

11. The method of claim 10, further including pursuant to the subsequent performance of the upgrade procedure, setting at least one persistent checkpoint timestamp to provide for error recovery.

12. The method of claim 10, further including, previous to the examining the current number of node elements:

estimating a time needed to perform the upgrade procedure as a function of time information obtained from previous performances of the upgrade procedure on additional node elements so as to provide greater accuracy to the estimation of time needed, and

comparing the estimation of the time needed with a predefined upgrade window, wherein if the estimation is within the predefined upgrade window, performing the subsequent upgrade procedure on the additional node element.

13. A system for facilitating upgrades in a computing storage environment, comprising:

a processor, in communication with a memory device, the processor operational on a management node in the computing storage environment, wherein the processor is adapted for;

examining a current number of node elements of the computing storage environment having failed an upgrade procedure as an error propagation count, comparing the error propagation count against a predefined error propagation tolerance value, and

reporting if the predefined error propagation tolerance value is exceeded to provide a health status of the upgrade procedure, wherein if a subsequent performance of the upgrade procedure on an additional node element is unsuccessful, the error propagation count is incremented and a critical error is reported so as to avoid propagation of an upgrade error.

14. The system of claim 13, wherein the processor is further adapted for, pursuant to the subsequent performance of the upgrade procedure, setting at least one persistent checkpoint timestamp to provide for error recovery.

15. The system of claim 13, wherein the processor is further adapted for estimating a time needed to perform the upgrade procedure as a function of time information obtained from previous performances of the upgrade procedure on additional node elements so as to provide greater accuracy to the estimation of time needed, and comparing the estimation of the time needed with a predefined upgrade window, wherein if the estimation is within the predefined upgrade window, the subsequent upgrade procedure is performed on the additional node element.

16. A computer program product for facilitating upgrades in a computing storage environment by a processor in communication with a memory device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for examining a current number of node elements of the computing storage environment having failed an upgrade procedure as an error propagation count, comparing the error propagation count against a predefined error propagation tolerance value; and

a second executable portion for reporting if the predefined error propagation tolerance value is exceeded to provide a health status of the upgrade procedure, wherein if a subsequent performance of the upgrade procedure on an additional node element is unsuccessful, the error propagation count is incremented and a critical error is reported so as to avoid propagation of an upgrade error.

17. The computer program product of claim 16, further including a third executable portion for, pursuant to the subsequent performance of the upgrade procedure, setting at least one persistent checkpoint timestamp to provide for error recovery.

18. The computer program product of claim 16, further including:

a third executable portion for estimating a time needed to perform the upgrade procedure as a function of time information obtained from previous performances of the upgrade procedure on additional node elements so as to provide greater accuracy to the estimation of time needed; and

a fourth executable portion for comparing the estimation of the time needed with a predefined upgrade window, wherein if the estimation is within the predefined upgrade window, the subsequent upgrade procedure is performed on the additional node element.