Method for rapid recovery from a network file server failure including method for operating co-standby servers

• US 5,978,565 A
• Filed: 04/28/1997
• Issued: 11/02/1999
• Est. Priority Date: 07/20/1993
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: Vinca Corp. (Dell Technologies Inc.)
• Inventors: Green, David, Ohran, Richard S., Ohran, Michael R., Winger, John M.
• Primary Examiner(s): An, Meng-Al T.
• Assistant Examiner(s): Davis, Jr., Walter D.
• Application Number:
  US08/848,139
• Time in Patent Office: 918 Days
• Field of Search: 395/181, 395/182.02, 395/182.04, 395/182.05, 395/182.08, 395/182.09, 395/182.11, 395/500
• US Class Current: 714/13
• CPC Class Codes: G06F 11/1482 by means of middleware or OS functionality
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 11/2087 with a common controller
• CPC Class Codes: G11C 29/74 using duplex memories, i.e. using dual copies
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06
• CPC Class Codes: Y10S 707/959 Network
• CPC Class Codes: Y10S 707/967 Peer-to-peer
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method for providing rapid recovery from a network file server failure through the use of a backup computer system. The backup computer system runs a special mass storage access program that communicates with a mass storage emulator program on the network file server, making the disks (or other mass storage devices) on the backup computer system appear like they were disks on the file server computer. By mirroring data by writing to both the mass storage of the file server and through the mass storage emulator and mass storage access program to the disks on the backup computer, a copy of the data on the file server computer is made. Optionally, selected portions of the data read through the mass storage emulator program can be altered before being returned as the result of the read operation on the file server. In the event of failure of the file server computer, the backup computer can replace the file server, using the copy of the file server'"'"'s data stored on its disks. A single backup computer can support a plurality of file server computers. Unlike other redundant file server configurations, this method does not require the backup computer system to be running the file server operating system.

Claims

1. A method for recovery from a file server failure in a network configuration comprising a first network server and a second network server interconnected by means for communicating between said first network server and said second network server, each of said first network server and said second network server comprising a file server operating system and at least one attached mass storage device, said method comprising the steps of:

executing on said first network server said file server operating system so that said first network server becomes operative to process file server requests that result in data being written to or read from the at least one mass storage device attached to said first network server;

executing on said first network server mass storage emulator program means for emulating a mass storage device so that said first network server appears to have, in addition to the at least one mass storage device attached to said first network server, at least one extra attached mass storage device that is accessed in the same manner as the at least one mass storage device attached to said first network server;

executing on said second network server mass storage access program means for communicating with said mass storage emulation program on said first network server over said means for communicating and for reading data from and writing data to said at least one mass storage device attached to said second network server;

said mass storage emulator program means receiving copies of mass storage requests sent to said at least one mass storage device attached to said first network server and said mass storage emulator program means sending said mass storage requests to said mass storage access program means via said means for communicating so that whenever said first network server writes data to the mass storage attached thereto, identical data is written to said mass storage emulator program;

said mass storage emulator program means sending received mass storage requests as commands over said means for communicating to said mass storage access program means; and

said mass storage access program means receiving over said means for communicating said commands and processing said commands to write data to or read data from said at least one mass storage device attached to said second network server.

2. A method for recovery from failure of a network server as recited in claim 1 further comprising the steps of:

executing on said second network server said file server operating system so that said second network server becomes operative to process file server requests that result in data being written to or read from the at least one mass storage device attached to said second network server; and

configuring said second network server to process different file server requests than said first network server so that a given file server request is only processed by one of either said first network server or said second network server.

3. A method for recovery from failure of a network server as recited in claim 2 further comprising the step of initiating data mirroring from said second network server to said first network server so that when a data write request is received by said second network server, data contained in the received data write request is written both to the at least one mass storage device attached to said second network server and to the at least one mass storage device attached to said first network server.

4. A method for recovery from failure of a network server as recited in claim 1 further comprising the steps of:

detecting failure of said first network server;

stopping said mass storage access program on said second network server; and

executing said file server operating system on said second network server and transferring responsibility for processing file server requests that were previously processed by said first network server to said second network server so that said network configuration continues to operate and does not fail even though said first network server has failed.

5. A method for recovery from failure of a network server as recited in claim 1 wherein said mass storage access program comprises a cache of data so that when said mass storage access program receives, from said first network server via said means for communicating, a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the at least one mass storage device attached to said second network server if the requested data does not reside in said cache.

6. A method for recovery from a file server failure in a network configuration comprising a first network server and a second network server interconnected by means for communicating between said first network server and said second network server, each of said first network server and said second network server comprising a mass storage device directly connected thereto said method comprising the steps of:

selecting at least one configuration of two system configurations such that (1) said first network server processes received file server requests and said second network server does not process received file server requests, and (2) said first network server and said second network server each processes different file server requests; and

initiating data mirroring from said first network server to said second network server via said means for communicating so that when a data write request is received by said first network server, data contained in said write request is written both to the mass storage device of said first network server and to the mass storage device of said second network server without said second network server independently receiving a copy of said data write request from the source that initiated said data write request, and wherein said data mirroring step comprises the steps of;

executing on said first network server a mass storage emulation program means for emulating a mass storage device so that said first network server appears to have at least one extra mass storage device in addition to the mass storage device directly connected to said first network server; and

executing on said second network server a mass storage access program means for communicating with said mass storage emulation program means on said first network server over said means for communicating and for writing data received over said means for communicating to the mass storage device connected to said second network server so that when data is written to said mass storage emulation program means on said first network server, the data is transferred from said mass storage emulation program means on said first network server to said second network server and written to the mass storage device connected thereto, and wherein said mass storage access program means is further adapted to return data to said mass storage emulation program means in response to a read command and said mass storage access program means comprises a cache of data so that when said mass storage access program means receives, from said first network server via said means for communicating, a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache.

7. A method for recovery from failure of a network server as recited in claim 6 wherein said first network server and said second network server arc configured to process different file server requests and the method further comprises the step of initiating data mirroring from said second network server to said first network server via said means for communicating so that when a data write request is received by said second network server, data contained in said write request is written both to the mass storage device attached to said second network server and to the mass storage device attached to said first network server.

8. A method for recovery from failure of a network server as recited in claim 7 further comprising the steps of:

detecting the failure of either the first network server or the second network server; and

if said first network server has failed, then transferring responsibility for processing file server requests that were previously processed by said first network server to said second network server and exiting the method; and

if said second network server has failed, then transferring responsibility for processing file server requests that were previously processed by said second network server to said first network server and exiting the method.

9. A method for recovery from failure of a network server as recited in claim 6 wherein if a read request is received by said first network server, the read request is filled by said first network server by performing at least the following steps:

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server; and

returning the retrieved data as the result of the received read request.

10. A method for recovery from failure of a network server as recited in claim 9 wherein if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, the second network server returns alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location.

11. A method for recovery from failure of a network server in a network configuration comprising a first network server and a second network server interconnected by means for communicating between said first network server and said second network server, each of said first network server and said second network server being adapted to execute a file server operating system in order to process file server requests and each of said first network server and said second network server comprising at least one attached mass storage device, said method comprising the steps of:

initiating operation of said file server operating system on said first network server and said second network server so that said first network server and said second network server become operative to process different file server requests such that a given file server request is processed by only one of either said first network server or said second network server; and

initiating data mirroring from said first network server to said second network server via said means for communicating so that when a data write request is received by said first network server data contained in said write request is written both to the at least one mass storage device attached to said first network server and to the at least one mass storage device attached to said second network server without said second network server independently receiving a copy of said data write request from the source that initiated said data write request, and wherein said data mirroring step comprises the steps of;

executing on said first network server a mass storage emulation program means for emulating a mass storage device so that said first network server appears to have at least one extra mass storage device in addition to the mass storage device directly connected to said first network server;

executing on said second network server a mass storage access program means for communicating with said mass storage emulation program means on said first network server over said means for communicating and for writing data received over said means for communicating to the mass storage device connected to said second network server so that when data is written to said mass storage emulation program means on said first network server, the data is transferred from said mass storage emulation program means on said first network server to said second network server and written to the mass storage device connected thereto, and wherein said mass storage access program means is further adapted to return data to said mass storage emulation program means in response to a read command and said mass storage access program means comprises a cache of data so that when said mass storage access program means receives, from said first network server via said means for communicating a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache; and

if a read request is received by said first network server, filling the read request by said first network server by performing at least the following steps;

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server;

returning the retrieved data as the result of the received read request; and

if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, returning, by the second network server alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location.

12. A method for recovery from failure of a network server as recited in claim 11 further comprising the step of initiating data mirroring from said second network server to said first network server via said means for communicating so that when a data write request is received by said second network server, data contained in said write request is written both to the at least one mass storage device attached to said second network server and to the at least one mass storage device attached to said first network server.

13. A method for recovery from failure of a network server as recited in claim 11 further comprising the steps of:

detecting the failure of either the first network server or the second network server; and

if said first network server has failed, then transferring responsibility for processing file server requests that were previously processed by said first network server to said second network server and exiting the method; and

if said second network server has failed, then transferring responsibility for processing file server requests that were previously processed by said second network server to said first network server and exiting the method.

14. A system for recovery from failure of a network server comprising:

a first network server comprising an attached mass storage device and a file server operating system, said first network server being adapted to process file server requests;

a second network server comprising an attached mass storage device and a file server operating system, said second network being adapted to function in at least one of two system configurations such that (1) said second network server does not process file server requests, and (2) said second network server processes different file server requests from said first network server so that a given file server request is processed by only one of either said first network server or said second network server;

means for communicating between said first network server and said second network server so that data can be exchanged over said means for communicating;

a mass storage emulation program executing on said first network server so that said first network server appears to have another mass storage device attached thereto in addition to said mass storage device attached to said first network server; and

a mass storage access program executing on said second network server, said mass storage access program adapted to communicate with said mass storage emulation program on said first network server via said means for communicating and adapted to write data received over said means for communicating to the mass storage device attached to said second network server so that when data is written to said mass storage emulation program on said first network server in response to a write request, the data is transferred from said mass storage emulation program on said first network server to said second network server and written to the mass storage device attached thereto without said second network server independently receiving a copy of said write request from another source, wherein said mass storage access program being further adapted to return data to said mass storage emulation program in response to a read command, said mass storage access program means comprising a cache of data so that when said mass storage access program receives, from said first network server via said means for communicating a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache;

wherein the first network server carries computer-executable instructions that, when executed upon a read request being received by said first network server, perform the steps of;

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server;

returning the retrieved data as the result of the received read request; and

if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, returning, by the second network server, alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location.

15. A system for recovery from failure of a network server as recited in claim 14 wherein said first network server and said second network server are connected to a common network.

16. A system for recovery from failure of a network server as recited in claim 14 further comprising means for automatically detecting failure of said first network server so that responsibility for processing file server requests can be transferred from said first network server to said second network server.

17. A system for recovery from failure of a network server as recited in claim 14 wherein said first network server is further adapted to receive mirroring data over said means for communicating and write said mirroring data to said at least one mass storage device attached to said first network server, thereby creating a system where file server requests processed by said first network server are mirrored from said first network server to said network server and where file server requests processed by said second network server are mirrored from said second network server to said first network server.

18. A system for recovery from failure of a network server as recited in claim 17 wherein each of said first network server and sad second network server further comprises means for automatically detecting the failure of a network server.

19. A system for recovery from failure of a network server comprising:

a first network server comprising an attached mass storage device and a file server operating system adapted to process a first set of file server requests;

a second network server comprising an attached mass storage device and a file server operating system adapted to process a second set of file server requests so that a given file server request is processed by only one of either said first network server or said second network server;

means for communicating between said first network server and said second network server so that data can be exchanged over said means for communicating;

a mass storage emulation program executing on said first network server so that said first network server appears to have another mass storage device attached thereto in addition to said mass storage device attached to said first network server; and

a mass storage access program executing on said second network server, said mass storage access program adapted to communicate with said mass storage emulation program on said first network server via said means for communicating and adapted to write data received over said means for communicating to the mass storage device attached to said second network server so that when data is written to said mass storage emulation program on said first network server in response to a write request, the data is transferred from said mass storage emulation program on said first network server to said second network server and written to the mass storage device attached thereto without said second network server independently receiving a copy of said write request from another source, said mass storage access program being further adapted to return data to said mass storage emulation program in response to a read command, said mass storage access program comprising a cache of data so that when said mass storage access program receives from said first network server via said means for communicating a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache;

wherein the first network server carries computer-executable instructions that, when executed upon a read request being received by said first network server, perform the steps of;

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server;

returning the retrieved data as the result of the received read request; and

if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, returning, by the second network server, alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location.

20. A system for recovery from failure of a network server as recited in claim 19 wherein said first network server and said second network server comprise different hardware configurations.

21. A computer-readable medium having executable instructions encoded thereon for use by a network configuration comprising a first network server and a second network server interconnected by a means for communicating between said first network server and said second network server, each of said first network server and said second network server comprising an attached mass storage device, said executable instructions comprising:

a mass storage emulation program means for emulating a mass storage device when loaded onto said first network server, said mass storage emulation program means being adapted to communicate with said second network server over said means for communicating in order to read data from or write data to the mass storage device attached to said second network server without said second network server independently receiving write requests from another source, said first network server appearing to have at least one extra mass storage device in addition to the mass storage device directly attached to said first network server;

a mass storage emulator program means for emulating a mass storage device so that said first network server appears to have at least one extra mass storage device in addition to the mass storage device directly connected to said first network server; and

a mass storage access program means for communicating with said mass storage emulation program means on said first network server over said means for communicating when said mass storage access program means, is loaded onto said second network server said mass storage access program means being adapted to write data received over said means for communicating to the mass storage device connected to said second network server so that when data is written to said mass storage emulation program means on said first network server, the data is transferred from said mass storage emulation program means on said first network server to said second network server and written to the mass storage device connected thereto, and wherein said mass storage access program means is further adapted to return data to said mass storage emulation program means in response to a read command and said mass storage access program means being associated with a cache of data so that when said mass storage access program means receives, from said first network server via said means for communicating, a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache;

read request program means for filling a read request received by said first network server when said read request program means is loaded onto said first network server, said read request program means being adapted to perform at least the following steps;

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server;

returning the retrieved data as the result of the received read request; and

if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, returning by the second network server, alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location; and

means for automatically detecting failure of said first network server in order to transfer responsibility from processing server requests from said first network server to said second network server.

22. A method for recovery from a file server failure in a network configuration comprising a first network server and a second network server interconnected by means for communicating between said first network server and said second network server, each of said first network server and said second network server comprising a mass storage device directly connected thereto, said method comprising the steps of:

selecting at least one configuration of two system configurations such that (1) said first network server processes received file server requests and said second network server does not process received file server requests, and (2) said first network server and said second network server each process different file server requests; and

initiating data mirroring from said first network server to said second network server via said means for communicating so that when a data write request is received by said first network server, data contained in said write request is written both to the mass storage device of said first network server and to the mass storage device of said second network server, wherein said initiating data mirroring step is accomplished by performing at least the steps of;

executing on said first network server a mass storage emulator program means for emulating a mass storage device so that said first network server appears to have at least one extra mass storage device in addition to the mass storage device directly connected to said first network server; and

executing on said second network server a mass storage access program means for communicating with said mass storage emulation program means on said first network server over said means for communicating and for writing data received over said means for communicating to the mass storage device connected to said second network server so that when data is written to said mass storage emulation program means on said first network server, the data is transferred from said mass storage initiation program means on said first network server to said second network server and written to the mass storage device connected thereto, wherein said mass storage access program means is further adapted to return data to said mass storage emulation program means in response to a read command and wherein said mass storage access program means comprises a cache of data so that when said mass storage access program means receives, from said first network server via said means for communicating, a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache.

23. A method for recovery from failure of a network server as recited in claim 22 wherein if a read request is received by said first network server, the read request is filled by said first network server by performing at least the following steps:

initiating at least one lower level read command to retrieve data both from the mass storage device attached to said first network server and from the mass storage device attached to said second network server;

waiting for the first response to said at least one read command from one of either the mass storage device attached to said first network server or the mass storage device attached to said second network server; and

returning the retrieved data as the result of the received read request.

24. A method for recovery from failure of a network server as recited in claim 23, wherein if the at least one lower level read command specifies that data is to be read from at least one special mass storage location, then when the second network server receives the at least one lower level read command, the second network server returns alternative data in response to said at least one read command and not the data stored in said at least one special mass storage location.

25. A method for recovery from failure of a network server in a network configuration comprising a first network server and a second network server interconnected by means for communicating between said first network server and said second network server, each of said first network server and said second network server being adapted to execute a file server operating system in order to process file server requests and each of said first network server and said second network server comprising at least one attached mass storage device, said method comprising the steps of

initiating operation of said file server operating system on said first network server and said second network server so that said first network server and said second network server become operative to process different file server requests such that a given file server request is processed by only one of either said first network server or said second network server;

and

initiating data mirroring from said first network server to said second network server via said means for communicating so that when a data write request is received by said first network server, data contained in said write request is written both to the at least one mass storage device attached to said first network server and to the at least one mass storage device attached to said second network server, wherein said initiating data mirroring step is accomplished by performing at least the steps of;

executing on said first network server a mass storage emulator program means for emulating a mass storage device so that said first network server appears to have at least one extra mass storage device in addition to the at least one mass storage device attached to said first network server; and

executing on said second network server a mass storage access program means for communicating with said mass storage emulation program means on said first network server over said means for communicating and for writing data received over said means for communicating to the at least one mass storage device attached to said second network server so that when data is written to said mass storage emulation program means on said first network server, the data is transferred from said mass storage emulation program means on said first network server to said second network server and written to the at least one mass storage device attached thereto, wherein said mass storage access program means is further adapted to return data to said mass storage emulation program means in response to a read command and wherein said mass storage access program means comprises a cache of data so that when said mass storage access program means receives, from said first network server via said means for communicating, a read command comprising a request for data, the requested data is retrieved from said cache if the requested data resides therein and from the mass storage device attached to said second network server if the requested data does not reside in said cache.