Hybrid locking using network and on-disk based schemes

• US 9,130,821 B2
• Filed: 09/23/2013
• Issued: 09/08/2015
• Est. Priority Date: 02/06/2004
• Status: Active Grant

• Current Assignee: VMware, Inc. (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Rawat, Mayank, Li, Jinyuan, Vilayannur, Murali, Scales, Daniel J.
• Primary Examiner(s): SONG, HUA JASMINE
• Application Number:
  US14/034,171
• Publication Number:
  US 20140025820A1
• Time in Patent Office: 715 Days
• Field of Search: 711/156, 709/225
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: H04L 41/00 Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks

Abstract

A method of acquiring a lock by a node, on a shared resource in a system of a plurality of interconnected nodes, is disclosed. Each node that competes for a lock on the shared resource maintains a list of locks currently owned by the node. A lock metadata is maintained on a shared storage that is accessible to all nodes that may compete for locks on shared resources. A heartbeat region is maintained on a shared resource corresponding to each node so nodes can register their liveness. A lock state is maintained in the lock metadata in the shared storage. A lock state may indicate lock held exclusively, lock free or lock in managed mode. If the lock is held in the managed mode, the ownership of the lock can be transferred to another node without a use of a mutual exclusion primitive such as the SCSI reservation.

Claims

1. A method of breaking and acquiring a lock on a shared resource on a network, wherein the lock is held by a node, the method comprising:

reading lock metadata for the lock;

based on the lock metadata, determining the lock is in a managed mode and is held by the node and is not being used by the node;

sending a request to the node to release the lock;

determining that the node has not renewed a lock lease time and that the lock lease time has expired; and

breaking and acquiring the lock by performing an operation on the shared resource using a mutual exclusion primitive, updating a local lock metadata to include information that a new lock owner node owns the lock, and updating the lock metadata in a shared storage to add a pointer to point to a heartbeat region for the new lock owner node.

2. The method as recited in claim 1, wherein the step of determining that the node has not renewed a lock lease time and that the lock lease time has expired comprises:

probing, at one or more selected intervals, a heartbeat region of the node to ascertain if the node is alive.

3. The method as recited in claim 1, wherein the step of updating the lock metadata including updating lock timestamp.

4. The method as recited in claim 1, wherein the step of updating the lock metadata includes updating a status of the lock.

5. The method as recited in claim 1, wherein the step of determining that the lock lease time has expired comprises:

maintaining in storage at least one heartbeat data structure, the heartbeat data structure comprising;

owner data identifying a current owner of the heartbeat;

state data identifying a current state of the heartbeat;

pulse data identifying a time of last renewal of the heartbeat; and

allowing the heartbeat to be modified if the pulse data is not changed within a selected period of time.

6. The method as recited in claim 5, the heartbeat data structure further comprising generation data identifying a version of the heartbeat.

7. The method as recited in claim 6, further comprising:

maintaining a lock data structure associated with one of a plurality of shared resources, the lock data structure including data identifying a heartbeat.

8. The method as recited in claim 7, wherein the data identifying the heartbeat includes at least one of:

owner data identifying a current owner of the heartbeat and address data identifying a location of the heartbeat.

9. A method of acquiring a lock by a first node, the method comprising:

changing a lock state in a local lock metadata for the lock attempting to be acquired by the first node to indicate that the lock is requested by the first node;

reading lock information from a heartbeat region;

responsive to the lock information indicating that the lock is being held by a second node, sending a request to the second node to release the lock;

responsive to the lock information indicating that the lock is free or that the first node has the lock or that the lock points to a stale heartbeat region, attempting to acquire the lock; and

prior to the step of changing the lock state in a local lock metadata for the lock attempting to be acquired by the first node to indicate that the lock is requested by the first node;

checking the local lock metadata to determine whether the lock is in a managed mode in which the lock is owned by a second node but the lock is not being used by the second node and a lock lease has not timed out;

responsive to the lock being in a managed mode and the lock lease not being timed out, updating the lock state in a shared storage of lock metadata to indicate that the lock is held by the first node, thereby allowing the first node to acquire the lock without going through a process of a mutual exclusion primitive; and

responsive to the lock not being m a managed mode, proceeding to the step of changing the lock state in the local lock metadata to indicate that the lock is requested by the first node.

10. The method of claim 9 wherein the lock information indicates that the lock is free by not pointing to a heartbeat region of a node.

11. The method of claim 9 further comprising:

receiving a message from the second node indicating that the lock is in use if local lock metadata of the second node includes information indicating that the lock is being owned in an exclusive mode by the second node.

12. The method of claim 9 further comprising:

receiving a message from the second node indicating that the second node is not an owner of the lock responsive to a local lock metadata of the second node not including information regarding the lock.

13. The method of claim 12 further comprising:

after receipt of the message, the first node re-reads lock metadata in a heartbeat region to determine any change in a status of the lock.

14. The method of claim 13 further comprising:

responsive to the status of the lock not being changed between a first read and the reread, attempting to break the lock.

15. The method of claim 13 further comprising:

responsive to the status of the lock being changed between a first read and the re-read, determining if the lock is free.

16. A method of managing a lock on a shared resource on a network, the method comprising:

maintaining a heartbeat region for a node on a shared storage on the network, the heartbeat region includes memory to store heartbeats of the node and memory to store a lock metadata of a corresponding lock;

maintaining a local lock metadata in the node to store status of a lock when the lock is owned by the node; and

responsive to a node attempting to acquire a lock, performing the steps comprising;

checking the local lock metadata to determine whether the lock is already owned by the node;

responsive to the lock is not being owned by the node, probing the lock metadata in the shared storage to retrieve the status of the lock and current owner of the lock; and

responsive to the status indicating that the lock is being held by another node in a managed mode in which the lock is being owned by that node but the shared resource is not being used by that node, requesting the another node to grant the lock to the node.

17. The method of claim 16, wherein if the lock is already owned by the node, updating the status in the lock metadata and the local lock metadata in the node to indicate that the lock is being held exclusively by the node.

18. The method of claim 16, wherein when the node no longer needs the shared resource, updating the status of the lock in the lock metadata and the local lock metadata to indicate that the shared resource is now being owned by the node in the managed mode.