Managing read/write locks for multiple CPU cores for efficient access to storage array resources

• US 9,201,802 B1
• Filed: 12/31/2012
• Issued: 12/01/2015
• Est. Priority Date: 12/31/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Armangau, Philippe, Seibel, Chris, Hassey, John A., Wang, Yubing
• Primary Examiner(s): Tseng, Cheng-Yuan
• Assistant Examiner(s): Talukdar, Arvind
• Application Number:
  US13/731,197
• Time in Patent Office: 1,065 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0261 using reference counting
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 9/526 Mutual exclusion algorithms

Abstract

A technique for managing resources on a storage array having multiple CPU cores releases share locks on a resource by decrementing share lock counters associated with the CPU cores that release the share locks, regardless of whether these are the same as the CPU cores that acquired the share locks. As each CPU core changes its own share lock counter, cache trashing caused by changing share lock counters associated with other CPU cores can be avoided.

Claims

1. A method, operable on a computerized system having multiple CPU cores, for managing access to resources of a storage array, comprising:

creating an instance of a mutex object that includes a mutex sub-object for each of the CPU cores, each mutex sub-object having a share lock counter;

receiving a request to obtain non-exclusive access to a resource of the storage array;

executing a first set of instructions on a first CPU core of the computerized system, the first CPU core having its own cache, to acquire a share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core in the cache of the first CPU core; and

executing a second set of instructions on a second CPU core of the computerized system, the second CPU core having its own cache, to release the share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for the second CPU core in the cache of the second CPU core,

wherein instantiating the mutex object includes instantiating the mutex object within a memory shared among the multiple CPU cores.

2. The method of claim 1, wherein the first set of instructions is part of a thread running on the first CPU core, and wherein the method further comprises:

prior to executing the first set of instructions, invoking a spin lock mutex in the thread running on the first CPU core, to serialize a set of requests for accessing the resource that arrive while the first set of instructions is being executed; and

releasing the spin lock mutex after executing the first set of instructions.

3. The method of claim 2, wherein the second set of instructions is part of a thread running on the second CPU core, and wherein the method further comprises:

prior to executing the second set of instructions, invoking a spin lock mutex in the thread running on the second CPU core, to serialize a set of requests for accessing the resource that arrive while the second set of instructions is being executed; and

releasing the spin lock mutex after executing the second set of instructions.

4. The method of claim 2, wherein the spin lock mutex applies to a particular sub-object of the mutex object for the resource and not to other sub-objects of the mutex object for the resource, and wherein the method further comprises modifying another sub-object of the mutex object for the resource after the spin lock mutex has been invoked and before the spin lock mutex has been released.

5. The method of claim 3, wherein the first CPU core and the second CPU core each have a respective cache, wherein the first sub-object and the second sub-object are cache-aligned in the respective caches.

6. The method of claim 3, further comprising:

receiving a second request to obtain non-exclusive access to the resource of the storage array;

executing, on the first CPU core, the first set of instructions to acquire a second share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core;

executing the second set of instructions on the second CPU core to release the second share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for a CPU core other than the second CPU core in response to the share lock counter of the mutex sub-object for the second CPU core indicating zero share locks.

7. The method of claim 6, wherein the mutex sub-object for the CPU core other than the second CPU is not the first CPU core.

8. The method of claim 6, wherein decrementing the share lock counter of the mutex sub-object for the CPU core other than the second CPU core includes:

checking the share lock counter of the sub-objects for each of the other CPU cores to find a sub-object whose share lock counter has a value that indicates a non-zero number of share locks; and

decrementing the share lock counter of the found sub-object.

9. The method of claim 6, wherein the second CPU core shares a common cache with at least one other CPU core of the computerized system, and wherein decrementing the share lock counter of the mutex sub-object for the CPU core other than the second CPU core includes:

checking the share lock counter of the sub-object for the other CPU core(s) that share the common cache with the second CPU core to find a sub-object whose share lock counter has a value that indicates a non-zero number of share locks; and

decrementing the share lock counter of the found sub-object.

10. The method of claim 9, wherein the second CPU core and at least one of the other CPU core(s) that share the common cache with the second CPU core share a common socket.

11. The method of claim 1, wherein the sub-object for each CPU core itself includes a read/write mutex that manages both share locks and exclusive locks for the resource.

12. The method of claim 1, wherein instantiating the object includes accessing a class definition of the object from a memory of the computerized system, the memory being shared by the CPU cores, the class definition including functions for acquiring and releasing share locks and exclusive locks for specified resources.

22. The method of claim 1, wherein the multiple CPU cores are disposed on a single computerized apparatus.

23. The method of claim 22, further comprising:

mirroring the mutex sub-object of the first CPU core from the memory to the cache of the first CPU core; and

mirroring the mutex sub-object of the second CPU core from the memory to the cache of the second CPU core.

13. A computerized apparatus, comprising:

multiple CPU cores; and

memory, coupled to the CPU cores, the memory storing executable instructions, which when executed by the CPU cores cause the CPU cores to perform a method for managing access to resources of a storage array, the method comprising;

creating an instance of a mutex object that includes a mutex sub-object for each of the CPU cores, each mutex sub-object having a share lock counter;

receiving a request to obtain non-exclusive access to a resource of the storage array;

executing a first set of instructions on a first CPU core of the computerized system, the first CPU core having its own cache, to acquire a share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core in the cache of the first CPU core; and

executing a second set of instructions on a second CPU core of the computerized system, the second CPU core having its own cache, to release the share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for the second CPU core in the cache of the second CPU core,

wherein instantiating the mutex object includes instantiating the mutex object within a memory shared among the multiple CPU cores.

14. The computerized apparatus of claim 13, wherein the first CPU core and the second CPU core each have a respective cache, wherein the first sub-object and the second sub-object are cache-aligned in the respective caches.

15. The computerized apparatus of claim 14, wherein the method further comprises:

receiving a second request to obtain non-exclusive access to the resource of the storage array;

executing, on the first CPU core, the first set of instructions to acquire a second share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core;

executing the second set of instructions on the second CPU core to release the second share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for a CPU core other than the second CPU core in response to the share lock counter of the mutex sub-object for the second CPU core indicating zero share locks.

16. The computerized apparatus of claim 15, wherein the second CPU core shares a common cache with at least one other CPU core of the computerized system, and wherein decrementing the share lock counter of the mutex sub-object for the CPU core other than the second CPU core includes:

checking the share lock counter of the sub-object for the other CPU core(s) that share the common cache with the second CPU core to find a sub-object whose share lock counter has a value that indicates a non-zero number of share locks; and

decrementing the share lock counter of the found sub-object.

17. The computerized apparatus of claim 16, wherein the second CPU core and at least one of the other CPU core(s) that share the common cache with the second CPU core share a common socket.

18. A non-transitory computer readable medium including instructions which, when executed by multiple CPU cores of a computerized apparatus, cause the CPU cores to perform a method of managing access to resources of a storage array, the method comprising:

creating an instance of a mutex object that includes a mutex sub-object for each of the CPU cores, each mutex sub-object having a share lock counter;

receiving a request to obtain non-exclusive access to a resource of the storage array;

executing a first set of instructions on a first CPU core of the computerized system, the first CPU core having its own cache, to acquire a share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core in the cache of the first CPU core; and

executing a second set of instructions on a second CPU core of the computerized system, the second CPU core having its own cache, to release the share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for the second CPU core in the cache of the second CPU core,

wherein instantiating the mutex object includes instantiating the mutex object within a memory shared among the multiple CPU cores.

19. The non-transitory computer readable medium of claim 18, wherein the first set of instructions is part of a thread running on the first CPU core, and wherein the method further comprises:

prior to executing the first set of instructions, invoking a spin lock mutex in the thread running on the first CPU core, to serialize a set of requests for accessing the resource that arrive while the first set of instructions is being executed; and

releasing the spin lock mutex after executing the first set of instructions.

20. The non-transitory computer readable medium of claim 18, wherein the method further comprises:

receiving a second request to obtain non-exclusive access to the resource of the storage array;

executing, on the first CPU core, the first set of instructions to acquire a second share lock on the resource, including incrementing the share lock counter of the mutex sub-object for the first CPU core;

executing the second set of instructions on the second CPU core to release the second share lock on the resource acquired by the first set of instructions executing on the first CPU core, including decrementing the share lock counter of the mutex sub-object for a CPU core other than the second CPU core in response to the share lock counter of the mutex sub-object for the second CPU core indicating zero share locks.

21. The non-transitory computer readable medium of claim 20, wherein the second CPU core shares a common cache with at least one other CPU core of the computerized system, and wherein decrementing the share lock counter of the mutex sub-object for the CPU core other than the second CPU core includes:

checking the share lock counter of the sub-object for the other CPU core(s) that share the common cache with the second CPU core to find a sub-object whose share lock counter has a value that indicates a non-zero number of share locks; and

decrementing the share lock counter of the found sub-object.