Multiple Node/Virtual Input/Output (I/O) Server (VIOS) Failure Recovery in Clustered Partition Mobility

• US 20130024718A1
• Filed: 09/15/2012
• Published: 01/24/2013
• Est. Priority Date: 11/05/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mewhinney, Greg R., Nevarez, David, Pafumi, James A., Rosales, Jacob J.
• Granted Patent: US 8,996,477 B2
• Time in Patent Office: Days
• US Class Current: 714/2
• CPC Class Codes: G06F 11/1425 by reconfiguration of node membership
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 2201/815 Virtual

Abstract

A method utilizes cluster-awareness to effectively support a live partition mobility (LPM) event and provide recovery from node failure within a Virtual Input/Output (I/O) Server (VIOS) cluster. An LPM utility creates a monitoring thread on a first VIOS on initiation of a corresponding LPM event. The monitoring thread tracks a status of an LPM and records status information in the mobility table of a database. The LPM utility creates other monitoring threads on other VIOSes running on the (same) source server. If the first VIOS VIOS sustains one of multiple failures, the LPM utility provides notification to other functioning nodes/VIOSes. The LPM utility enables a functioning monitoring thread to update the LPM status. In particular, a last monitoring thread may perform cleanup/update operations within the database based on an indication that there are nodes on the first server that are in failed state.

Claims

1. In a data processing system having a processor, a memory coupled to the processor, at least one input/output (I/O) adapter that enables connection to an external network with a shared storage repository, and a plurality of virtual I/O servers (VIOSes) that form a VIOS cluster with a shared database, where each VIOS is cluster aware, a method comprising:

activating a first monitoring thread on a first VIOS of a first server to track a status of a live partition mobility (LPM) event;

recording information about the LPM event within the shared database by using said first monitoring thread;

in response to the first VIOS sustaining a failure condition, identifying one or more functioning monitoring threads that continue to function on a source server, wherein the failure condition results in a loss of LPM event monitoring by the first monitoring thread;

determining whether said one or more functioning monitoring threads is a single, last monitoring thread; and

in response to a first VIOS on the first server being in failed state, performing, via said last monitoring thread, cleanup and update operations within the shared database, wherein the cleanup and update are performed responsive to receipt of an indication that there are one or more nodes on the first server that are in the failed state.

2. The method of claim 1, wherein said activating a first monitoring thread further comprises:

detecting an initiation of an LPM event; and

in response to detecting the initiation of the LPM event, creating one or more of;

(a) the first monitoring thread on the first VIOS on the first server within said cluster-aware data processing system;

(b) a first set of monitoring threads on other VIOSes on the first server; and

(c) a second set of monitoring threads on VIOSes on a second, target server.

3. The method of claim 1, wherein said recording further comprises:

recording information about the LPM event into a mobility table using said monitoring thread;

wherein the recorded information includes information about one or more of;

(a) a client;

(b) the server; and

(c) the status of a corresponding VIOS participating in the LPM event;

determining when the LPM event is terminated; and

in response to the LPM event being terminated, providing via said first monitoring thread one or more of;

(a) associated updates to storage resources;

(b) removal of a corresponding entry in a mobility table of the database; and

(c) other maintenance operations.

4. The method of claim 1, further comprising:

detecting when a particular VIOS associated with a virtualized client partition corresponding to the LPM has sustained a failure condition;

in response to detecting that the particular VIOS has sustained the failure condition and that a corresponding monitoring thread no longer exists, providing notification of the failed state of the VIOS to other VIOSes on a same server within said cluster-aware-DPS; and

enabling one or more of said other VIOSes to appropriately update LPM event status in the database.

5. The method of claim 1, further comprising:

initiating continual checks about the state of the LPM event by other monitoring threads via communication of said other monitoring threads with a hypervisor;

determining whether said LPM event is one of;

(a) terminated;

(b) completed; and

(c) in a failed state;

in response to said LPM event ending, providing to said other monitoring threads;

(a) information that indicates that the LPM event is ended; and

(b) information about a count of the set of other monitoring threads that currently monitors the LPM event;

in response to the set of other monitoring threads being a single, last monitoring thread, initiating via the last monitoring thread a query of the database to determine whether there exists nodes on an appropriate server in a failed state; and

in response to nodes on the appropriate server being in a failed state, performing, via said last monitoring thread, update operations on a corresponding mobility table including one or more of;

(a) operations needed to keep the database consistent and current;

(b) removal of corresponding row/entry from the corresponding mobility table; and

(c) other maintenance operations.

6. The method of claim 5, further comprising:

in response to the LPM event having ended as a result of a failure condition, returning to a particular stage of a corresponding migration process in order to resume and complete the mobility operation.