Reducing Remote Reads Of Memory In A Hybrid Computing Environment By Maintaining Remote Memory Values Locally

• US 20110035556A1
• Filed: 08/07/2009
• Published: 02/10/2011
• Est. Priority Date: 08/07/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aho, Michael E., Carey, James E., Sanders, Philip J., Markland, Matthew W., Archer, Charles J.
• Granted Patent: US 8,180,972 B2
• Time in Patent Office: Days
• US Class Current: 711/147
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]

Abstract

Reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the hybrid computing environment including a host computer and a plurality of accelerators, the host computer and the accelerators each having local memory shared remotely with the other, including writing to the shared memory of the host computer packets of data representing changes in accelerator memory values, incrementing, in local memory and in remote shared memory on the host computer, a counter value representing the total number of packets written to the host computer, reading by the host computer from the shared memory in the host computer the written data packets, moving the read data to application memory, and incrementing, in both local memory and in remote shared memory on the accelerator, a counter value representing the total number of packets read by the host computer.

Claims

1. A method of reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the hybrid computing environment comprising a host computer having a host computer architecture, a plurality of accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computer and the accelerators adapted to one another for data communications by a system level message passing module, the host computer having local memory shared remotely with the accelerators, each accelerator having local memory shared remotely with the host computer, the shared memory on both the host and the accelerators each configured as a predefined number of individual memory segments, the method comprising:

writing by an accelerator when one or more memory values in accelerator application memory change, through the system level message passing module to the shared memory of the host computer beginning at a next available memory segment, one or more packets of data representing the changes in accelerator memory values, including incrementing, in both local shadow memory on the accelerator and in remote shared memory on the host computer, a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer beginning with a next unread memory segment, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing, in both local shadow memory on the host computer and in remote shared memory on the accelerator, a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

2. The method of claim 1 further comprising notifying, by memory access functions of the accelerator, the system level message passing module of changes in accelerator application memory values.

3. The method of claim 1 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when the number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

4. The method of claim 1 further comprising:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon the predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is the predetermined number of individual memory segments.

5. The method of claim 1 further comprising:

determining by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory memory segments beginning with the identified next unread memory segment.

6. A method of reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the hybrid computing environment comprising a host computer having a host computer architecture, a plurality of accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computer and the accelerators adapted to one another for data communications by a system level message passing module, the host computer having local memory shared remotely with the accelerators, each accelerator having local memory shared remotely with the host computer, the shared memory on both the host and the accelerators each configured as a predefined number of individual memory segments, the method comprising:

writing by the host computer when one or more memory values in host computer application memory change, through the system level message passing module to the shared memory of an accelerator beginning at a next available memory segment, one or more packets of data representing the changes in host computer memory values, including incrementing, in both local shadow memory on the host computer and in remote shared memory on the accelerator, a counter value representing the total number of packets written by the host computer to the shared memory in the accelerator; and

reading by the accelerator asynchronously with respect to the writing, from the shared memory in the accelerator beginning with a next unread memory segment, the packets of data written by the host computer to the shared memory in the accelerator, including moving the read data to application memory of the accelerator and incrementing, in both local shadow memory on the accelerator and in remote shared memory on the host computer, a counter value representing the total number of packets read by the accelerator from the shared memory in the accelerator.

7. The method of claim 6 further comprising notifying, by memory access functions of the host computer, the system level message passing module of changes in host computer application memory values.

8. The method of claim 6 wherein incrementing a counter value representing the total number of packets read by the accelerator from the shared memory in the accelerator further comprises:

incrementing the counter value in local shadow memory in the accelerator every time the accelerator reads packets of data from the shared memory in the accelerator; and

incrementing the counter value in remote shared memory on the host computer only when the number of unread memory segments in the shared memory in the accelerator reaches a predetermined unread memory threshold.

9. The method of claim 6 further comprising:

determining by the host computer, using only counter values stored in local memory on the host computer, a number of unread memory segments in the shared memory of the accelerator in dependence upon the total number of packets written by the host computer to the shared memory in the accelerator and the total number of packets read by the accelerator from the shared memory in the accelerator;

determining by the host computer, using only counter values stored in local memory on the host computer, a number of memory segments in the shared memory of the accelerator available to receive data packets in dependence upon the predetermined number of individual memory segments in the shared memory of the accelerator and the number of unread memory segments in the shared memory of the accelerator; and

identifying by the host computer the next available memory segment in the shared memory of the accelerator according to (L+1) modulo N, where L is the total number of packets written by the host computer to the shared memory of the accelerator and N is the predetermined number of individual memory segments.

10. The method of claim 6 further comprising:

determining by the accelerator, using only counter values stored in local memory on the accelerator, the number of unread memory segments in the shared memory of the accelerator in dependence upon the total number of packets written by the host computer to the shared memory in the accelerator and the total number of packets read by the accelerator from the shared memory in the accelerator; and

calculating by the accelerator, using only counter values stored in local memory on the accelerator, the number of unread memory segments in the shared memory of the accelerator by subtracting from the total number of packets written by the host computer to the shared memory in the accelerator the total number of packets read by the accelerator from the shared memory in the accelerator; and

identifying by the accelerator the next unread memory segment according to (M+1) modulo N, where M is the total number of packets read by the accelerator from the shared memory in the accelerator and N is the predetermined number of individual memory segments in the shared memory of the accelerator;

wherein reading from the shared memory in the host computer further comprises reading unread data from the determined number of unread memory memory segments beginning with the identified next unread memory segment.

11. A hybrid computing environment for reducing remote reads of memory by maintaining remote memory values locally, the hybrid computing environment comprising a host computer having a host computer architecture, a plurality of accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computer and the accelerators adapted to one another for data communications by a system level message passing module, the host computer having local memory shared remotely with the accelerators, each accelerator having local memory shared remotely with the host computer, the shared memory on both the host and the accelerators each configured as a predefined number of individual memory segments, the host computer and the accelerators having disposed within their memories computer program instruction capable, when executed, of causing the hybrid computing environment to function by:

writing by an accelerator when one or more memory values in accelerator application memory change, through the system level message passing module to the shared memory of the host computer beginning at a next available memory segment, one or more packets of data representing the changes in accelerator memory values, including incrementing, in both local shadow memory on the accelerator and in remote shared memory on the host computer, a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer beginning with a next unread memory segment, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing, in both local shadow memory on the host computer and in remote shared memory on the accelerator, a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

12. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by notifying, by memory access functions of the accelerator, the system level message passing module of changes in accelerator application memory values.

13. The hybrid computing environment of claim 11 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when the number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

14. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon the predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is the predetermined number of individual memory segments.

15. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by:

determining by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory memory segments beginning with the identified next unread memory segment.

16. A computer program product for reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the hybrid computing environment comprising a host computer having a host computer architecture, a plurality of accelerators having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions, the host computer and the accelerators adapted to one another for data communications by a system level message passing module, the host computer having local memory shared remotely with the accelerators, each accelerator having local memory shared remotely with the host computer, the shared memory on both the host and the accelerators each configured as a predefined number of individual memory segments, the computer program product disposed in a recordable, computer readable medium, the computer program product comprising computer program instructions capable, when executed, of causing the hybrid computing environment to function by:

writing by an accelerator when one or more memory values in accelerator application memory change, through the system level message passing module to the shared memory of the host computer beginning at a next available memory segment, one or more packets of data representing the changes in accelerator memory values, including incrementing, in both local shadow memory on the accelerator and in remote shared memory on the host computer, a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer beginning with a next unread memory segment, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing, in both local shadow memory on the host computer and in remote shared memory on the accelerator, a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

17. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by notifying, by memory access functions of the accelerator, the system level message passing module of changes in accelerator application memory values.

18. The computer program product of claim 16 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when the number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

19. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon the predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is the predetermined number of individual memory segments.

20. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by:

determining by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory memory segments beginning with the identified next unread memory segment.