Distributed symmetric multiprocessing computing architecture

• US 8,607,004 B2
• Filed: 11/15/2010
• Issued: 12/10/2013
• Est. Priority Date: 11/13/2009
• Status: Active Grant

• Current Assignee: Richard S. Anderson
• Original Assignee: Richard S. Anderson
• Inventors: Anderson, Richard S.
• Primary Examiner(s): Tran, Denise
• Application Number:
  US12/946,626
• Publication Number:
  US 20110125974A1
• Time in Patent Office: 1,121 Days
• Field of Search: None
• US Class Current: 711/153
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]

Abstract

Example embodiments of the present invention includes systems and methods for implementing a scalable symmetric multiprocessing (shared memory) computer architecture using a network of homogeneous multi-core servers. The level of processor and memory performance achieved is suitable for running applications that currently require cache coherent shared memory mainframes and supercomputers. The architecture combines new operating system extensions with a high-speed network that supports remote direct memory access to achieve an effective global distributed shared memory. A distributed thread model allows a process running in a head node to fork threads in other (worker) nodes that run in the same global address space. Thread synchronization is supported by a distributed mutex implementation. A transactional memory model allows a multi-threaded program to maintain global memory page consistency across the distributed architecture. A distributed file access implementation supports non-contentious file I/O for threads. These and other functions provide a symmetric multiprocessing programming model consistent with standards such as Portable Operating System Interface for Unix (POSIX).

Claims

1. A method of performing symmetric multiprocessing (SMP) computing, the method comprising:

forming a cluster of SMP compute nodes (servers), with the memory in each node similarly divided into a local and global partition, where the global partitions of each node combine together to form a single global memory;

starting and executing an application program in the local memory partition of the head SMP node in the cluster;

allowing additional threads of the program to be launched and executed by processor cores in the head node or in worker nodes;

identify and copy a given memory page from the global memory to the local memory partition of a given SMP node when needed by the executing program thread; and

enabling a thread to lock for exclusive access a given memory page in the global memory.

2. The method of claim 1 wherein identifying and copying the given memory page occurs with a latency of less than 4 microseconds.

3. The method of claim 1 wherein identifying and copying the given memory page includes transmitting data between a given pair of SMP nodes in the cluster via a directly coupled communications link between the given pair of SMP nodes.

4. The method of claim 1 wherein identifying and copying the given memory page includes transmitting data between a given pair of SMP nodes in the cluster via a switch.

5. The method of claim 1 further including:

maintaining a fixed logical address in the global memory for a given memory page, wherein the fixed logical address;

(a) translates to a physical page address in the global memory partition of a particular SMP node in the cluster;

(b) eliminates any need to search for memory pages in the global memory; and

(c) enables use of one-sided remote direct memory access operations in moving the given memory page between the local memory and the global memory.

6. The method of claim 1 further including:

copying a modified memory page from the local memory partition to global memory to maintain a consistent representation of an application program logical address space.

7. The method of claim 1 wherein enabling a program thread to lock a given global memory page for exclusive access by the thread, by issuing a lock request via an operating system primitive, and further including:

handling the lock request with the SMP node whose global memory partition stores a reference copy of the given memory page.

8. The method of claim 7 further including:

broadcasting a message invalidating and locking the targeted memory page from the SMP node whose global memory partition hosts the reference copy of the targeted memory page, to other SMP nodes in the cluster if the lock request is granted.

9. The method of claim 7 further including:

broadcasting a message to SMP nodes in the cluster unlocking the given memory page when the lock is released.

10. The method of claim 1, further including:

allowing a thread to force immediate availability of a modified memory page to global memory by using a memory page synchronization operating system primitive.

11. The method of claim 1, wherein enabling the programmer to avoid false sharing of independent variables by placing these variables on different memory pages using an aligned allocation operating system primitive that forces a new memory block to begin on a new memory page.

12. The method of claim 1, wherein distributed thread management is implemented on a network of symmetric multiprocessing (SMP) nodes that support distributed shared memory, the method comprising:

reproducing an SMP program execution model across multiple nodes;

starting all application programs on a head node;

launching additional execution threads that run in the head node or in a worker node; and

maintaining a global thread table of threads executing on worker nodes in the head node.

13. A method as in claim 12 further including:

duplicating the process environment of an application program running on the head, including page tables, on to a worker node, to allow the launching and execution of threads on the work node.

14. A method as in claim 12 further including:

launching a thread on the worker node from the head node; and

receiving a notification when execution of the thread is complete.

15. A method of claim 1, wherein distributed mutual exclusion lock (mutex) management is implemented on an underlying network of symmetric multiprocessing servers, the method comprising:

maintaining a global mutex table configured to track remote mutex activity at each node; and

filtering all locks or waits on a mutex through the global mutex table to handle those cases where the mutex is remotely addressed.