Providing transparent failover in a file system

• US 8,631,277 B2
• Filed: 12/10/2010
• Issued: 01/14/2014
• Est. Priority Date: 12/10/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Swan, Paul R., George, Mathew, Kruse, David M., Battepati, Roopesh C., Johnson, Michael C.
• Primary Examiner(s): MASKULINSKI, MICHAEL C
• Application Number:
  US12/964,749
• Publication Number:
  US 20120151249A1
• Time in Patent Office: 1,131 Days
• Field of Search: None
• US Class Current: 714/15
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1443 Transmit or communication errors
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component

Abstract

A connection state system is described herein that allows a client to resume a connection with a server or a different replacement server by remotely storing client state information in association with a resume key. The system provides a resume key filter operating at the server that facilitates the storing of volatile server state information. The state information can include information such as oplocks, leases granted to a client, and in-flight operations on a file handle. The resume key filter driver sits above the file system, which allows multiple file access protocols to use the filter. Upon a failover event, such as a server going down or losing connectivity to a client, the system can bring up another server or the same server and reestablish state for file handles held by various clients using the resume key filter.

Claims

1. A computer-implemented method for capturing file system state information to facilitate resuming connections, the method comprising:

receiving, from a first client, a request to access a remote resource stored on a server;

determining an identifier that identifies a client session related to the request;

creating a resume record, wherein the resume record is searchable by the identifier and wherein the resume record associates state information created by operations requested by the first client with the identifier;

receiving a file operation from the first client, wherein the file operation requests access to a file accessible through the server;

storing resume state information in the resume record, wherein the resume state information provides information to resume the received file operation if the first client loses the connection with the server;

upon the first client losing the connection with the server, loading, at a failover server, the resume record to enable the first client to connect to the failover server and continue the file operation, wherein access to the remote resource is restricted for a period of time to prevent a second client from interacting with the remote resource in a manner that conflicts with the first client establishing a connection to the failover server;

performing the requested file operation;

updating the resume state information in the resume record based on an outcome of the file operation; and

sending a response to the first client that indicates the outcome of the requested file operation.

2. The method of claim 1 wherein the access request includes one or more parameters, including a resume key that identifies the client session across multiple potential connections if a connection fails, and wherein the resume key is at least part of the determined identifier.

3. The method of claim 1 further comprising, upon the first client becoming disconnected from the server, receiving at the failover server a new access request that the failover server can correlate with the original access request to help the failover server respond to the first client after a connection failure by correlating state information maintained by the server between multiple client connections.

4. The method of claim 1 wherein a Network File System (NFS) server determines the identifier automatically without receiving a resume key from the first client.

5. The method of claim 1 wherein the identifier is a Server Message Block (SMB) resume key that the first client provides for durable handles that enable disconnected sessions to be resumed.

6. The method of claim 1 wherein creating the resume record comprises storing the resume record at a location external to the server handling the request to access the remote resource stored on the server.

7. The method of claim 1 where receiving the file operation comprises a request to perform an operation selected from the group consisting of opening a file, closing a file, reading a file, writing a file, obtaining a lease on a file, and obtaining a lock on a file.

8. The method of claim 1 wherein performing the requested file operation modifies a state stored by the server, and wherein updating the resume state information captures the modified state.

9. The method of claim 1 wherein updating the resume state information comprises keeping an up to date view of the server state in the resume record that allows another server to reestablish the state and handle client requests in place of the original server without requiring the first client to reestablish at least some of the state information.

10. A computer system for providing transparent failover for clients in a file system, the system comprising:

a processor and memory configured to execute software instructions embodied within the following components;

a state collection component that creates a state record for each file handle and collects state information as a first client requests operations using the file handle;

a state storing component that stores collected state information in association with a session identifier provided by the first client;

a state data store that persistently stores file system state information that a resuming server uses to recreate state information stored by a failing server;

a resume detection component that detects a condition that makes a failing server unavailable and informs a resuming server to act in the failing server'"'"'s place;

a state retrieval component that retrieves stored state information from a location accessible to the resuming server, wherein the state information allows the resuming server to resume any previously requested file system operations that were interrupted by the detected failure condition;

a blackout enforcement component that enforces a blackout period on access to one or more files or other resources to prevent a second client from interfering with the one or more files or other resources in a manner that conflicts with the first client establishing a connection to the resuming server; and

a state restoration component that loads the retrieved state information into the resuming server so that the resuming server can continue operations previously requested by the first client.

11. The system of claim 10 wherein the state collection component is further configured to operate at a server and store state information externally from the server so that the state information can be accessed if the server is unavailable.

12. The system of claim 10 wherein the state collection component is further configured to receive a resume key from the first client when the first client connects to the server, and associate collected state information with the resume key in the state data store.

13. The system of claim 10 wherein the state data store stores and provides information for the resuming server, wherein the resuming server is the same server as the failing server using a different connection to the first client.

14. The system of claim 10 wherein the state data store receives state information as the failing server is performing operations, and, upon a failure, provides access to the previously received state information to the resuming server to resume the state and continue carrying out any operations that were not completed.

15. The system of claim 10 wherein the condition detected by the resume detection component is the first client connecting with a session identifier previously stored in the state data store.

16. The system of claim 10 wherein the state restoration component provides at least one data blob to an external component that manages a file system or protocol, wherein the data blob is stored in the state data store on behalf of the external component for use during a resume operation.

17. The system of claim 10 further comprising a resource suspension component that allows a currently active resource to be suspended and resumed without a failover event to allow a cluster to failover to another node in planned manner.

18. A computer-readable storage medium encoding computer executable instructions which when executed by a processor performs a method, the method comprising:

receiving, from a first client, a request to access a remote resource stored on a server;

determining an identifier that identifies a client session related to the request;

creating a resume record, wherein the resume record is searchable by the identifier and wherein the resume record associates state information created by operations requested by the first client with the identifier;

receiving a file operation from the first client, wherein the file operation requests access to a file accessible through the server;

storing resume state information in the resume record, wherein the resume state information provides information to resume the received file operation if the first client loses the connection with the server;

upon the first client losing the connection with the server, loading, at a failover server, the resume record to enable the first client to connect to the failover server and continue the file operation, wherein access to the remote resource is restricted for a period of time to prevent a second client from interacting with the remote resource in a manner that conflicts with the first client establishing a connection to the failover server.

19. The computer-readable storage medium of claim 18 further comprising instructions for:

upon establishing a connection to the failover server;

performing the requested file operation;

updating the resume state information in the resume record based on an outcome of the file operation; and

sending a response to the first client that indicates the outcome of the requested file operation.

20. The computer-readable storage medium of claim 18 wherein the identifier is a Server Message Block (SMB) resume key.