Memory switching protocol when switching optically-connected memory

• US 9,390,047 B2
• Filed: 02/03/2016
• Issued: 07/12/2016
• Est. Priority Date: 04/13/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Schenfeld, Eugen, Chakraborty, Abhirup
• Primary Examiner(s): Bragdon, Reginald
• Assistant Examiner(s): VO, THANH DUC
• Application Number:
  US15/014,638
• Publication Number:
  US 20160154755A1
• Time in Patent Office: 160 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/0284 Multiple user address space allocation, e.g. using different base addresses interprocessor communication G06F15/163
• CPC Class Codes: G06F 12/0292 using tables or multilevel address translation means G06F12/023 takes precedence; address translation in virtual memory systems G06F12/10
• CPC Class Codes: G06F 12/0891 using clearing, invalidating or resetting means
• CPC Class Codes: G06F 12/10 Address translation
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 12/1027 using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
• CPC Class Codes: G06F 13/404 with address mapping
• CPC Class Codes: G06F 13/4068 Electrical coupling
• CPC Class Codes: G06F 13/42 Bus transfer protocol, e.g. handshake; Synchronisation
• CPC Class Codes: G06F 2212/50 Control mechanisms for virtual memory, cache or TLB
• CPC Class Codes: G06F 2212/60 Details of cache memory
• CPC Class Codes: G06F 2212/683 Invalidation
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Data is collected by an active node from passive nodes. A source node extracts the data format, and a remote memory blade identification (ID), a remote memory blade address, and ranges of the RMMA space, and composes and sends metadata to receiving nodes and receiving racks.

Claims

1. In an optically-connected memory (OCM) system, a method for a memory switching protocol, comprising:

collecting data by an active node from passive nodes and storing data according to receiving nodes,

extracting, by a source node, a data format and a remote memory blade identification (ID), a remote memory blade address, and ranges of a remote machine memory address (RMMA) space, and composing and sending metadata to the receiving nodes and receiving racks,

unmapping the RMMA space by the source node thereby removing all entries in page tables that map a linear address to the remote RMMA space, wherein system memory address (SMA) space that is associated with alternative RMMA space is free,

allocating memory from one of memory blades upon receiving a memory request from the one of a plurality of processors, wherein a circuit is established with the memory for the one of the plurality of processors, an address space of the memory sent to the one of the plurality of processors,

mapping the address space of the memory to the SMA space upon the one of the plurality of processors receiving the address space of the memory, wherein entries of the page table corresponding to the address space are created,

retaining a remote memory superpage of the memory in a memory blade when reading the remote memory super page of the memory into the one of the plurality of processors, and

transferring a physical memory address space from the one of the plurality of processors to an alternative one of the plurality of processors.

2. The method of claim 1, further including performing the collecting, the extracting, and the unmapping on a side of the source node.

3. The method of claim 1, wherein the optically-connected memory (OCM) system includes the plurality of processors arranged in processor blades and memory in the memory blades, with each of the processor blades and the memory blades organized in racks communicatively coupled through an optical-switching fabric, the processor blades and the memory blades remotely separated.

4. The method of claim 3, further including performing one of the collecting, the extracting, and the unmapping for dynamically switching the memory through the optical-switching fabric using a selected one of a plurality of available communication patterns to transfer the RMMA space in the memory blades from one of the plurality of processors to an alternative one of the plurality of processors in the processor blades without physically copying data in the memory to the plurality of processors.

5. The method of claim 3, further including, at a side of the receiving nodes, performing each of:

receiving the metadata sent by the source node,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors into mapping tables,

if a circuit with a remote memory blade does not already exist, setting up a circuit with the remote memory blade,

reading the data by the active node and sending the data to passive processor nodes via an intra-rack edge switch, wherein the data at the RMMA space retains routing information for different portions of the data, and

upon transmission of all the data in the RMMA space, performing, by the active node, one of relinquishing the memory to the memory manager at the memory blades and reuses the memory with new data arriving for a subsequent operation for dynamically switching the memory.

6. The method of claim 3, further including retaining the data in the memory while sending the metadata to the processors for regenerating mapping tables and page tables on the processor blades with information included in the metadata.

7. The method of claim 3, wherein the processor blades and the memory blades are configured with a plurality of channels, and further including accessing the memory through the plurality of channels.

8. The method of claim 3, wherein the memory blades are communicatively coupled with the plurality of processors in the optical-switching fabric via an optical circuit network comprising a plurality of optical external links and at least one optical circuit switch, and further including allowing access of the plurality of processors to the memory in the memory blades through at least one independent circuit established between the processor blades and the memory blades within the optical circuit network.

9. The method of claim 8, further including sharing the memory in the memory blades between more than one of the plurality of processors by occupying a non-overlapping portion of the memory.

10. The method of claim 3, further including disconnecting a circuit to one of the memory blades while signaling another one of the memory blades to establish a circuit with another one of the memory blades.

11. The method of claim 3, further including performing at least one of:

sending the metadata and a switching request to the one of the plurality of processors,

adjusting a remote machine memory address (RMMA) space and flushing a translation look-aside buffer (TLB),

performing an unmapping operation for removing entries in the page tables that map a linear address to the RMMA space,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors and regenerating the mapping table,

dynamically switching the memory from an active node of the one of the plurality of processors to a remote active node of the alternative one of the plurality of processors, and

using the page tables by a receiving one of the plurality of processors for instantaneous access to the data.

12. An optically-connected memory (OCM) system for a memory switching protocol, comprising:

a plurality of nodes including at least a source node, passive nodes, and receiving nodes,

at least one processor device in communication with each of the plurality of nodes and operable in the computing storage environment, wherein the at least one processor device performs each of;

collects data by an active node from passive nodes and stores data according to receiving nodes,

extracts, by the source node, a data format and a remote memory blade identification (ID), a remote memory blade address, and ranges of a remote machine memory address (RMMA) space, and composes and sends metadata to the receiving nodes and receiving racks,

unmaps the RMMA space by the source node thereby removing all entries in page tables that map a linear address to a remote RMMA space, wherein system memory address (SMA) space that is associated with alternative RMMA space is free,

allocating memory from one of memory blades upon receiving a memory request from the one of a plurality of processors, wherein a circuit is established with the memory for the one of the plurality of processors, the address space of the memory sent to the one of the plurality of processors,

mapping the address space of the memory to the SMA space upon the one of the plurality of processors receiving the address space of the memory, wherein entries of the page table corresponding to the address space are created,

retaining a remote memory superpage of the memory in the memory blade when reading the remote memory super page of the memory into the one of the plurality of processors, and

transferring a physical memory address space from the one of the plurality of processors to an alternative one of the plurality of processors.

13. The system of claim 12, wherein the at least one processor device performs the collecting, the extracting, and the unmapping on a side of the source node.

14. The system of claim 12, further including:

at least one processor blade,

at least one memory blade, remotely separated from the at least one processor blade,

an optical plane,

the least one processor device arranged in one of the at least one processor blade and the at least one memory blade and in communication with the optical plane,

a translation look-aside buffer (TLB) in communication with the at least one processor device and the optical plane,

at least one memory in the at least one memory blade, and

an optical switching fabric communicatively coupled between the at least one processor blade and the at least one memory blade and in communication with the at least one processor device.

15. The system of claim 14, wherein the at least one processor device performs one of collecting, extracting, and unmapping for dynamically switching the memory through the optical-switching fabric using a selected one of a plurality of available communication patterns to transfer the RMMA space in the memory blades from one of the plurality of processors to an alternative one of the plurality of processors in the processor blades without physically copying data in the memory to the plurality of processors.

16. The system of claim 14, wherein the at least one processor device, at a side of the receiving nodes, performs each of:

receiving the metadata sent by the source node,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors into mapping tables,

if a circuit with a remote memory blade does not already exist, setting up a circuit with the remote memory blade,

reading the data by the active node and sending the data to passive processor nodes via an intra-rack edge switch, wherein the data at the RMMA space retains routing information for different portions of the data, and

upon transmission of all the data in the RMMA space, performing, by the active node, one of relinquishing the memory to the memory manager at the memory blades and reuses the memory with new data arriving for a subsequent operation for dynamically switching the memory.

17. The system of claim 14, wherein the at least one processor device retains the data in the memory while sending the metadata to the processors for regenerating mapping tables and page tables on the processor blades with information included in the metadata.

18. The system of claim 14, wherein the processor blades and the memory blades are configured with a plurality of channels, and further including accessing the memory through the plurality of channels.

19. The system of claim 14, wherein the memory blades are communicatively coupled with the plurality of processors in the optical-switching fabric via an optical circuit network comprising a plurality of optical external links and at least one optical circuit switch, and further including allowing access of the plurality of processors to the memory in the memory blades through at least one independent circuit established between the processor blades and the memory blades within the optical circuit network.

20. The system of claim 19, wherein the at least one processor device shares the memory in the memory blades between more than one of the plurality of processors by occupying a non-overlapping portion of the memory.

21. The system of claim 14, wherein the at least one processor device disconnects a circuit to one of the memory blades while signaling another one of the memory blades to establish a circuit with another one of the memory blades.

22. The system of claim 14, wherein the at least one processor device performs at least one of:

sending the metadata and a switching request to the one of the plurality of processors,

adjusting a remote machine memory address (RMMA) space and flushing the translation look-aside buffer (TLB),

performing an unmapping operation for removing entries in the page tables that map a linear address to the RMMA space,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors and regenerating the mapping table,

dynamically switching the memory from an active node of the one of the plurality of processors to a remote active node of the alternative one of the plurality of processors, and

using the page tables by a receiving one of the plurality of processors for instantaneous access to the data.

23. In an optically-connected memory (OCM) system, for a memory switching protocol, a computer program product in a computing environment using a processor device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion that collects data by an active node from passive nodes and stores data according to receiving nodes; and

a second executable portion that extracts, by a source node, a data format and a remote memory blade identification (ID), a remote memory blade address, and ranges of the RMMA space, and composes and sends metadata to the receiving nodes and receiving racks,

unmaps the RMMA space by the source node thereby removing all entries in page tables that map a linear address to a remote RMMA space, wherein system memory address (SMA) space that is associated with alternative RMMA space is free,

allocating memory from one of memory blades upon receiving a memory request from the one of a plurality of processors, wherein a circuit is established with the memory for the one of the plurality of processors, the address space of the memory sent to the one of the plurality of processors,

mapping the address space of the memory to the SMA space upon the one of the plurality of processors receiving the address space of the memory, wherein entries of the page table corresponding to the address space are created,

retaining a remote memory superpage of the memory in the memory blade when reading the remote memory super page of the memory into the one of the plurality of processors, and

transferring a physical memory address space from the one of the plurality of processors to an alternative one of the plurality of processors.

24. The computer program product of claim 23, wherein the optically-connected memory (OCM) system includes the plurality of processors arranged in processor blades and memory in the memory blades, with each of the processor blades and the memory blades organized in racks communicatively coupled through an optical-switching fabric, the processor blades and the memory blades remotely separated.

25. The computer program product of claim 23, further including a fourth executable portion that performs one of the collecting, the extracting, and the unmapping on a side of the source node.

26. The computer program product of claim 24, further including a third executable portion that performs one of the collecting, the extracting, and the unmapping for dynamically switching the memory through the optical-switching fabric using a selected one of a plurality of available communication patterns to transfer the RMMA space in the memory blades from one of the plurality of processors to an alternative one of the plurality of processors in the processor blades without physically copying data in the memory to the plurality of processors.

27. The computer program product of claim 24, further including, at a side of the receiving nodes, a third executable portion that performs each of:

receiving the metadata sent by the source node,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors into mapping tables,

if a circuit with a remote memory blade does not already exist, setting up a circuit with the remote memory blade,

reading the data by the active node and sending the data to passive processor nodes via an intra-rack edge switch, wherein the data at the RMMA space retains routing information for different portions of the data, and

upon transmission of all the data in the RMMA space, performing, by the active node, one of relinquishing the memory to the memory manager at the memory blades and reuses the memory with new data arriving for a subsequent operation for dynamically switching the memory.

28. The computer program product of claim 24, further including a third executable portion that retains the data in the memory while sending the metadata to the processors for regenerating mapping tables and page tables on the processor blades with information included in the metadata.

29. The computer program product of claim 24, wherein the processor blades and the memory blades are configured with a plurality of channels, and further including accessing the memory through the plurality of channels.

30. The computer program product of claim 24, wherein the memory blades are communicatively coupled with the plurality of processors in the optical-switching fabric via an optical circuit network comprising a plurality of optical external links and at least one optical circuit switch, and further including allowing access of the plurality of processors to the memory in the memory blades through at least one independent circuit established between the processor blades and the memory blades within the optical circuit network.

31. The computer program product of claim 30, further including a third executable portion that shares the memory in the memory blades between more than one of the plurality of processors by occupying a non-overlapping portion of the memory.

32. The computer program product of claim 24, further including a third executable portion that disconnects a circuit to one of the memory blades while signaling another one of the memory blades to establish a circuit with the another one of the memory blades.

33. The computer program product of claim 24, further including a third executable portion that performs at least one of:

sending the metadata and a switching request to the one of the plurality of processors,

adjusting a remote machine memory address (RMMA) space and flushing a translation look-aside buffer (TLB),

performing an unmapping operation for removing entries in the page tables that map a linear address to the RMMA space,

grafting the RMMA space onto an available system memory address (SMA) space of the one of the plurality of processors and regenerating the mapping table,

dynamically switching the memory from an active node of the one of the plurality of processors to a remote active node of the alternative one of the plurality of processors, and

using the page tables by a receiving one of the plurality of processors for instantaneous access to the data.