Recovery from failure of a data processor in a network server

• US 6,275,953 B1
• Filed: 09/26/1997
• Issued: 08/14/2001
• Est. Priority Date: 09/26/1997
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Vahalia, Uresh K., Gupta, Uday, Ting, Dennis P. J.
• Primary Examiner(s): Lee, Thomas
• Assistant Examiner(s): DU, THUAN N
• Application Number:
  US08/938,859
• Time in Patent Office: 1,418 Days
• Field of Search: 714/11, 714/4, 714/10, 714/13
• US Class Current: 714/11
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40
• CPC Class Codes: G06F 11/2007 using redundant communication media
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2035 without idle spare hardware
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2046 where the redundant components share persistent storage G06F11/2043 takes precedence
• CPC Class Codes: H04L 41/0654 using network fault recovery ring fault isolation or reconfiguration in loop networks without recovery actions by a network management system H04L12/437
• CPC Class Codes: H04L 43/0817 by checking functioning

Abstract

A network file server includes a first set of data processors for receiving requests from clients, and a second set of data processors for accessing read-write file systems. A respective data processor in the second set is assigned to each file system for exclusive management of locks on the file system. The file server can detect failure of a failed data processor and automatically recover from the failure. When a failure of a data processor in the first set is detected, a spare data processor is programmed with the logical and physical network addresses of the failed data processor so that the spare data processor assumes the network identity of the failed data processor. When a failure of a data processor in the second set is detected, responsibility for management of the locks on each file system managed by the failed data processor is transferred to an operational data processor. Preferably the responsibility is transferred to the operational data processors in such a way as to balance loading on the operational data processors. The data processors can be commodity digital computers for low cost, and a cached disk storage subsystem or file system caches and remote dual copy techniques can be used to ensure high performance and high data availability.

Claims

1. A method of operating data processors for servicing clients in a network, each of the data processors having a respective network interface for interfacing to the network, each network interface having a respective network address, and each network interlace being programmable for setting the network address of said each network interface, said method comprising the steps of:

responding to a failure of a failed data processor by setting the network address of the network interface of an operational data processor to the network address of the network interface of the failed data processor; and

then

said operational data processor servicing client requests received by the network interface of the operational data processor wherein the network interface of the operational data processor is programmed with a physical network address of the network interface of the failed data processor, and the method further includes the operational data processor using a logical network address corresponding to the physical network address of the network interface of the failed data processor for the servicing of the client requests received by the network interface of the operational data processor.

2. The method as claimed in claim 1, wherein the physical address is a machine address code (MAC), and the logical network address is an internet protocol (IP) address used in a network communication protocol for communication over the network between the operational data processor and the clients having originated the client request received by the network interface of the operational data processor.

3. The method as claimed in claim 1, wherein one of the data processors is programmed to monitor status of other data processors, and said one of the data processors responds to a failure of the failed data processor by sending a command to the operational data processor to set the network address of the network interface of the operational data processor to the network address of the network interface of the failed data processor.

4. The method as claimed in claim 3, wherein said one of the data processors periodically polls the other data processors for status, and detects the failure of the failed data processor upon failing to receive a response from the failed data processor.

5. The method as claimed in claim 3, wherein said one of the data processors periodically polls the other data processors for status, and detects the failure of the failed data processor upon receiving from the failed data processor a response indicating the failure of the failed data processor.

6. The method as claimed in claim 1, wherein the operational data processor is substituted in the network for the failed data processor in a fashion that is transparent to a client originally in communication with the failed data processor except for a period of time when there appears to be a data link failure.

7. The method as claimed in claim 1, wherein the failed data processor had received an uncommitted result for a transaction from a client, and the operational data processor aborts the transaction in response to the failure of the failed processor.

8. The method as claimed in claim 1, wherein the failed data processor had received a request for an asynchronous write to a file from a client, and the operational data processor responds to a first request received by the operational data processor from the client for committing a result of the asynchronous writes to the file by sending to the client a reply indicating a failure to commit.

9. The method as claimed in claim 1, wherein the failed data processor had been assigned a set of files for which the failed data processor had exclusive responsibility for management of locks on the set of files, and in response to the failure of the failed data processor, the operational data processor is assigned exclusive responsibility for management of the locks on the set of files.

10. The method as claimed in claim 9, wherein, in response to the failure of the failed data processor, the operational data processor fetches locking information for the set of files to which the operational data processor is assigned.

11. The file server as claimed in claim 10, wherein the data processors receive asynchronous write requests from the clients, and the locking information for the set of files to which the operational data processor is assigned includes indications of write locks on files in the set of files, and the operational data processor signals to clients having write locks on the files in the set of files that any asynchronous write to a write-locked file in the set of files may have failed.

12. The file server as claimed in claim 11, wherein the operational data processor signals to clients having write locks on the files in the set of files that any asynchronous write to a write-locked file in the set of files may have failed by responding to a first request received by the operational data processor from a client for committing results to a write-locked file in the set of files by sending to the client a reply indicating a failure to commit.

13. The file server as claimed in claim 11, wherein the failed data processor and the operational data processor each have respective disk storage, each file in the set of files was stored in the disk storage of the failed data processor, and in response to the failure of the failed data processor, each file in the set of files is written to the disk storage of the operational data processor.

14. The file server as claimed in claim 13, wherein the failed data processor had responsibility for maintaining local copies of files in the set of files in the disk storage of the failed data processor and sending remote copy data to another data processor which maintained in disk storage of said another data processor remote copies corresponding to the local copies of the files in the set of files, and in response to the failure of the failed data processor, the remote copies are read from the disk storage of said another data processor and written to the disk storage of the operational data processor.