Methods and systems for distributed failure detection and recovery using leasing techniques

• US 6,760,736 B2
• Filed: 12/20/2000
• Issued: 07/06/2004
• Est. Priority Date: 10/11/1996
• Status: Expired due to Term

• Current Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Scheifler, Robert, Waldo, James H., Wollrath, Ann M., Arnold, Kenneth C. R. C.
• Primary Examiner(s): HOMERE, JEAN RAYMOND
• Application Number:
  US09/739,744
• Publication Number:
  US 20010000812A1
• Time in Patent Office: 1,294 Days
• Field of Search: 707/10, 707/3, 707/206, 709/203, 709/219, 709/229
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 12/0261 using reference counting
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 9/4411 Configuring for operating with peripheral devices; Loading of device drivers
• CPC Class Codes: G06F 9/465 Distributed object oriented systems remote method invocation [RMI] G06F9/548
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/5016 the resource being the memory
• CPC Class Codes: G06F 9/5022 Mechanisms to release resources
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services
• CPC Class Codes: H04L 12/417 with deterministic access, e.g. token passing
• CPC Class Codes: H04L 12/433 with asynchronous transmission, e.g. token ring, register insertion
• CPC Class Codes: H04L 69/24 Negotiation of communication capabilities
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 : Recoverability
• CPC Class Codes: Y10S 707/99957 : Garbage collection

Abstract

A system for using a lease to detect a failure and to perform failure recovery is provided in using this system, a client requests a lease from a server to utilize a resource managed by the server for a period of time. Responsive to the request, the server grants the lease, and the client continually requests renewal of the lease. If the client fails to renew the lease, the server detects that an error has occurred to the client. Similarly, if the server fails to respond to a renew request, the client detects that an error has occurred to the server. As part of the lease establishment, the client and server exchange failure-recovery routines that each invokes if the other experiences a failure.

Claims

1. A method for recovering from failures in a distributed system that includes a client and a server, the method, performed by the server, comprising:

receiving a request from the client for a lease that allows the client to access a resource managed by the server for a period of time

storing a request object associated with the request in a predetermined memory space;

granting the lease by returning to the client a lease object including methods for managing the lease; and

invoking a recovery method on the request when it is determined that the lease has expired and the client has not sent a request to renew or terminate the lease.

2. The method of claim 1, further comprising:

deleting the request object from the predetermined memory space after the recovery method has been invoked.

3. A system for recovering from failures, comprising:

a server configured for granting leases for using resources managed by the server for a period of time and for sending lease objects to a client when a lease is granted; and

a client configured for sending lease requests to the server for using the resources for a period of time, and for receiving the lease objects from the server, wherein the lease objects each include a first recovery method and the lease requests each include a second recovery method,

wherein the client is further configured to invoke a the first recovery method when the client determines that a lease renew request sent to the server has not been acknowledged by the server, and the server is further configured to invoke a the second recovery method when the server determines that a lease held by the client has expired and the client has not sent a request to renew or terminate the lease.

4. A method for recovering from failures in a distributed system that includes a client and a server, comprising:

invoking, by the client, a first recovery method on the server when the client determines the server failed to process a lease management request sent by the client; and

invoking, by the server, a second recovery method on the client, when the server determines that the client failed to perform lease management functions before a lease held by the client expired, wherein the lease allows the client to access a resource managed by the server for a period of time.

8. The method of claim 4, wherein the first recovery method invoked by the client was sent to client by the server with a lease object.

9. The method of claim 4, wherein the second recovery method invoked by the server was sent to the server by the client in a request for the lease.

5. The method of 4, wherein the lease management functions includes one of sending a request to renew the lease and sending a request to cancel the lease.

10. The method of claim 5, wherein the first recovery method invoked by the client and the lease management functions were sent with a lease object to client by the server, in response to a request for the lease held by the client.

6. The method of 4, wherein the lease management request includes a request to renew the lease.

7. The method of claim 6, wherein the request to renew the lease is sent by the client upon determination that the lease is about to expire.

11. A method for recovering from failures in a distributed system that includes a client and a server, the method performed by the client, comprising:

sending a request for a lease to the server for using a managed resource for a period of time;

receiving a lease object from the server, wherein the lease object includes a plurality of methods for managing the requested lease and a recovery method for performing a recovery process associated with the managed resource;

determining that the lease is about to expire;

sending a lease renewal request to the server, in response to the determination that the lease is about to expire;

determining that the lease renewal request did not complete successfully; and

invoking a the recovery method to perform the recovery process.

15. The method of claim 11, wherein the lease request includes a second recovery method that enables the server to invoke recovery operations on the client.

12. The method of 11, wherein the lease object includes the recovery method invoked by the client.

13. The method of claim 12, wherein the lease object includes a lease renewal method that enables the client to invoke the lease renewal request.

14. The method of 11, wherein the recovery process includes restarting the server.

16. A computer-readable medium including instructions for performing a method, when executed by a processor, for recovering from failures in a distributed system that includes a client and a server, the method, performed by the server, comprising:

receiving a request from the client for a lease that allows the client to access a resource managed by the server for a period of time;

storing a request object associated with the request in a predetermined memory space;

granting the lease by returning to the client, a lease object including methods for managing the lease; and

invoking a recovery method on the request when it is determined that the lease has expired and the client has not sent a request to renew or terminate the lease.

17. The computer-readable medium of claim 16, wherein the method further comprises:

deleting the request object from the predetermined memory space after the recovery method has been invoked.

18. A computer-readable medium, including instructions, for performing a method, when executed by a processor, for recovering from failures in a distributed system that includes a client and a server, comprising:

invoking, by the client, a first recovery method on the server when the client determines the server failed to process a lease management request sent by the client; and

invoking, by the server, a second recovery method on the client, when the server determines that the client failed to perform lease management functions before a lease held by the client expired, wherein the lease allows the client to access a resource managed by the server for a period of time.

22. The computer-readable medium of claim 18, wherein the first recovery method invoked by the client was sent to client by the server with a lease object.

23. The computer-readable medium of claim 18, wherein the second recovery method invoked by the server was sent to the server by the client in a request for the lease.

19. The computer-readable medium of 18, wherein the lease management functions includes one of sending a request to renew the lease and sending a request to cancel the lease.

24. The computer-readable medium of claim 19, wherein the first recovery method invoked by the client and the lease management functions were sent with a lease object to client by the server, in response to a request for the lease held by the client.

20. The computer-readable medium of 18, wherein the lease management request includes a request to renew the lease.

21. The computer-readable medium of claim 20, wherein the request to renew the lease is sent by the client upon determination that the lease is about to expire.

25. A computer-readable medium, including instructions, for performing a method, when executed by a processor, for recovering from failures in a distributed system that includes a client and a server, the method performed by the client, comprising:

sending a request for a lease to the server for using a managed resource for a period of time;

receiving a lease object from the server, wherein the lease object includes a plurality of methods for managing the requested lease and a recovery method for performing a recovery process associated with the managed resource;

determining that the lease is about to expire;

sending a lease renewal request to the server, in response to the determination that the lease is about to expire;

determining that the lease renewal request did not complete successfully; and

invoking a the recovery method to perform the recovery process.

26. The computer-readable medium of 25, wherein the lease object includes the recovery method invoked by the client.

27. The computer-readable medium of claim 26, wherein the lease object includes a lease renewal method that enables the client to invoke the lease renewal request.

29. The computer-readable medium of claim 26, wherein the lease request includes a second recovery method that enables the server to invoke recovery operations on the client.

28. The computer-readable medium of 25, wherein the recovery process includes restarting the server.