Data coherency model and protocol at cluster level

• US 9,383,932 B2
• Filed: 12/27/2013
• Issued: 07/05/2016
• Est. Priority Date: 12/27/2013
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Fleischer, Balint T., Das Sharma, Debendra, Kumar, Mohan J.
• Primary Examiner(s): BUTLER, SARAI E
• Application Number:
  US14/142,733
• Publication Number:
  US 20150186057A1
• Time in Patent Office: 921 Days
• Field of Search: 714/6.1
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/1081 for peripheral access to main memory, e.g. direct memory access [DMA]
• CPC Class Codes: G06F 13/1663 Access to shared memory
• CPC Class Codes: G06F 13/32 using combination of interrupt and burst mode transfer
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 2212/1056 Simplification
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 9/467 Transactional memory G06F9/528 takes precedence
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G11C 14/009 and the nonvolatile element is a resistive RAM element, i.e. programmable resistors, e.g. formed of phase change or chalcogenide material

Abstract

An apparatus for providing data coherency is described herein. The apparatus includes a global persistent memory. The global persistent memory is accessed using a protocol that includes input/output (I/O) semantics and memory semantics. The apparatus also includes a reflected memory region. The reflected memory region is a portion of the global persistent memory, and each node of a plurality of nodes maps the reflected memory region into a space that is not cacheable. Further, the apparatus includes a semaphore memory. The semaphore memory provides a hardware assist for enforced data coherency.

Claims

1. An apparatus for providing data coherency, comprising:

a global persistent memory, wherein the global persistent memory is accessed using a protocol that includes input/output (I/O) semantics and memory semantics;

a reflected memory region, wherein the reflected memory region is a portion of the global persistent memory, and each node of a plurality of nodes maps the reflected memory region into a space that is not cacheable; and

a semaphore memory, wherein the semaphore memory provides a hardware assist for enforced data coherency.

2. The apparatus of claim 1, wherein the input/output (I/O) semantics and the memory semantics are implemented using a Plattsmouth Link.

3. The apparatus of claim 1, wherein a portion of the global persistent memory is used for inter-node communication.

4. The apparatus of claim 1, wherein the global memory includes a message region, and a write to the message region of the global memory results in a write to the destination node'"'"'s private memory region.

5. The apparatus of claim 4, wherein the message region is accessed using memory semantics.

6. The apparatus of claim 1, wherein a directory is maintained within the semaphore region for each shared block of memory of the global memory.

7. The apparatus of claim 1, wherein the global memory is implemented using a DIMM, non-volatile memory, volatile memory, any storage device, and memory device, or any combination thereof.

8. The apparatus of claim 1, wherein the reflected memory region and the semaphore memory region are backed by physical memory.

9. The apparatus of claim 1, wherein each node maps the reflected memory region into its uncacheable space.

10. A system for providing a data coherency at a cluster level, comprising:

a plurality of nodes, wherein the plurality of nodes include a software coherency mechanism;

a global shared memory, wherein the global shared memory includes a message region, a semaphore region, a shared memory region; and

one or more fabric memory controllers, wherein the one or more fabric memory controllers connects the plurality of nodes and provides a hardware assist for data coherency at the cluster level.

11. The system of claim 10, wherein the fabric memory controller is a low latency fabric that supports memory load store semantics and an Input/Output protocol with load and store functionality.

12. The system of claim 10, wherein the fabric memory controller is a low latency fabric that supports a Plattsmouth protocol.

13. The system of claim 10, wherein the fabric memory controller is integrated as part of host fabric interface of the CPU.

14. The system of claim 10, wherein the fabric memory controller is integrated as part of Intel®

high performance computing host fabric interface.

15. The system of claim 10, wherein the fabric memory controller is integrated in to the CPU socket.

16. The system of claim 10, wherein the global memory is implemented using a DIMM, non-volatile memory, volatile memory, any storage device, and memory device, or any combination thereof.

17. The system of claim 10, wherein the global memory is implemented using non-volatile memory or storage, wherein the non-volatile technology is next generation non-volatile technology.

18. The system of claim 10, wherein the global memory is implemented using non-volatile memory or storage where the non-volatile technology is Phase Change Memory and switch (PCMS) devices.

19. The system of claim 10, wherein a local memory of each of the plurality of nodes includes a local message region, and the global memory indirectly accesses the local message region through DMA messages passed by each of the plurality of nodes.

20. The system of claim 10, wherein the local message regions pass messages to access the global memory using a software based handshake that is a direct memory access and does not use an I/O stack to access the data.

21. The system of claim 10, wherein the semaphore memory region is used to implement semaphore semantics for inter-node communication.

22. The system of claim 10, wherein the reflected memory region is used for message passing between nodes.

23. The system of claim 10, wherein the reflected memory region includes an interrupt region that is used for inter node communications.

24. A method of data coherency at the cluster level, the method comprising:

enabling a cacheable global memory, wherein the global memory is stored across a plurality of nodes;

maintaining data coherency across the plurality of nodes; and

implementing independent fault domains for each node of the plurality of nodes.

25. The method of claim 24, wherein data coherency is maintained using software stored on each node of the plurality of nodes and a hardware assist mechanism on a fabric memory controller that connects each node the plurality of nodes.

26. The method of claim 24, wherein the independent fault domains are implemented such that the shared memory on a first node of the plurality of nodes is accessible by other nodes even when the first node has failed.