Reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally

• US 8,539,166 B2
• Filed: 03/09/2012
• Issued: 09/17/2013
• Est. Priority Date: 08/07/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aho, Michael E., Archer, Charles J., Carey, James E., Markland, Matthew W., Sanders, Philip J.
• Primary Examiner(s): Verbrugge, Kevin
• Application Number:
  US13/416,636
• Publication Number:
  US 20120191920A1
• Time in Patent Office: 557 Days
• Field of Search: None
• US Class Current: 711/147
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]

Abstract

Reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the hybrid computing environment including a host computer and a plurality of accelerators, the host computer and the accelerators each having local memory shared remotely with the other, including writing to the shared memory of the host computer packets of data representing changes in accelerator memory values, incrementing, in local memory and in remote shared memory on the host computer, a counter value representing the total number of packets written to the host computer, reading by the host computer from the shared memory in the host computer the written data packets, moving the read data to application memory, and incrementing, in both local memory and in remote shared memory on the accelerator, a counter value representing the total number of packets read by the host computer.

Claims

1. A method of reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the method comprising:

writing by an accelerator when one or more memory values in accelerator application memory change, to shared memory of the host computer, one or more packets of data representing the changes in accelerator memory values, including incrementing a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

2. The method of claim 1 further comprising notifying, by memory access functions of the accelerator, a system level message passing module of changes in accelerator application memory values.

3. The method of claim 1 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when a number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

4. The method of claim 1 further comprising:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon a predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is the predetermined number of individual memory segments.

5. The method of claim 1 further comprising:

determining by the host computer, using only counter values stored in local memory on the host computer, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory segments beginning with an identified next unread memory segment.

6. A method of reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the method comprising:

writing by the host computer when one or more memory values in host computer application memory change, to shared memory of an accelerator, one or more packets of data representing the changes in host computer memory values, including incrementing a counter value representing the total number of packets written by the host computer to the shared memory in the accelerator; and

reading by the accelerator asynchronously with respect to the writing, from the shared memory in the accelerator, the packets of data written by the host computer to the shared memory in the accelerator, including moving the read data to application memory of the accelerator and incrementing a counter value representing the total number of packets read by the accelerator from the shared memory in the accelerator.

7. The method of claim 6 further comprising notifying, by memory access functions of the host computer, a system level message passing module of changes in host computer application memory values.

8. The method of claim 6 wherein incrementing a counter value representing the total number of packets read by the accelerator from the shared memory in the accelerator further comprises:

incrementing the counter value in local shadow memory in the accelerator every time the accelerator reads packets of data from the shared memory in the accelerator; and

incrementing the counter value in remote shared memory on the host computer only when a number of unread memory segments in the shared memory in the accelerator reaches a predetermined unread memory threshold.

9. The method of claim 6 further comprising:

determining by the host computer, using only counter values stored in local memory on the host computer, a number of unread memory segments in the shared memory of the accelerator in dependence upon the total number of packets written by the host computer to the shared memory in the accelerator and the total number of packets read by the accelerator from the shared memory in the accelerator;

determining by the host computer, using only counter values stored in local memory on the host computer, a number of memory segments in the shared memory of the accelerator available to receive data packets in dependence upon a predetermined number of individual memory segments in the shared memory of the accelerator and the number of unread memory segments in the shared memory of the accelerator; and

identifying by the host computer the next available memory segment in the shared memory of the accelerator according to (L+1) modulo N, where L is the total number of packets written by the host computer to the shared memory of the accelerator and N is the predetermined number of individual memory segments.

10. The method of claim 6 further comprising:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the accelerator in dependence upon the total number of packets written by the host computer to the shared memory in the accelerator and the total number of packets read by the accelerator from the shared memory in the accelerator; and

calculating by the accelerator, using only counter values stored in local memory on the accelerator, the number of unread memory segments in the shared memory of the accelerator by subtracting from the total number of packets written by the host computer to the shared memory in the accelerator the total number of packets read by the accelerator from the shared memory in the accelerator; and

identifying by the accelerator the next unread memory segment according to (M+1) modulo N, where M is the total number of packets read by the accelerator from the shared memory in the accelerator and N is the predetermined number of individual memory segments in the shared memory of the accelerator;

wherein reading from the shared memory in the host computer further comprises reading unread data from the determined number of unread memory segments beginning with an identified next unread memory segment.

11. A hybrid computing environment for reducing remote reads of memory by maintaining remote memory values locally, the hybrid computing environment including computer memory and a computer processor, the computer memory including computer program instructions that, when executed by the computer processor, cause the hybrid computing environment to carry out the steps of:

writing by an accelerator when one or more memory values in accelerator application memory change, to shared memory of the host computer, one or more packets of data representing the changes in accelerator memory values, including incrementing a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

12. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by notifying, by memory access functions of the accelerator, a system level message passing module of changes in accelerator application memory values.

13. The hybrid computing environment of claim 11 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when the number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

14. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon the predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is a predetermined number of individual memory segments.

15. The hybrid computing environment of claim 11 further comprising computer program instruction capable of causing the hybrid computing environment to function by:

determining by the host computer, using only counter values stored in local memory on the host computer, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory segments beginning with an identified next unread memory segment.

16. A non-transitory computer program product for reducing remote reads of memory in a hybrid computing environment by maintaining remote memory values locally, the computer program product disposed in a recordable, computer readable medium, the computer program product comprising computer program instructions capable, when executed, of causing the hybrid computing environment to function by:

writing by an accelerator when one or more memory values in accelerator application memory change, to the shared memory of the host computer, one or more packets of data representing the changes in accelerator memory values, including incrementing a counter value representing the total number of packets written by the accelerator to the shared memory in the host computer; and

reading by the host computer asynchronously with respect to the writing, from the shared memory in the host computer, the packets of data written by the accelerator to the shared memory in the host computer, including moving the read data to application memory of the host computer and incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer.

17. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by notifying, by memory access functions of the accelerator, a system level message passing module of changes in accelerator application memory values.

18. The computer program product of claim 16 wherein incrementing a counter value representing the total number of packets read by the host computer from the shared memory in the host computer further comprises:

incrementing the counter value in local shadow memory in the host computer every time the host computer reads packets of data from the shared memory in the host computer; and

incrementing the counter value in remote shared memory on the accelerator only when a number of unread memory segments in the shared memory in the host computer reaches a predetermined unread memory threshold.

19. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by:

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer;

determining by the accelerator, using only counter values stored in local memory on the accelerator, a number of memory segments in the shared memory of the host computer available to receive data packets in dependence upon a predetermined number of individual memory segments in the shared memory of the host computer and the number of unread memory segments in the shared memory of the host computer; and

identifying by the accelerator the next available memory segment in the shared memory of the host computer according to (J+1) modulo N, where J is the total number of packets written by the accelerator to the shared memory of the host computer and N is the predetermined number of individual memory segments.

20. The computer program product of claim 16 further comprising computer program instructions capable of causing the hybrid computing environment to function by:

determining by the host computer, using only counter values stored in local memory on the host computer, a number of unread memory segments in the shared memory of the host computer in dependence upon the total number of packets written by the accelerator to the shared memory in the host computer and the total number of packets read by the host computer from the shared memory in the host computer; and

calculating by the host computer, using only counter values stored in local memory on the host computer, the number of unread memory segments in the shared memory of the host computer by subtracting from the total number of packets written by the accelerator to the shared memory in the host computer the total number of packets read by the host computer from the shared memory in the host computer; and

identifying by the host computer the next unread memory segment according to (K+1) modulo N, where K is the total number of packets read by the host computer from the shared memory in the host computer and N is the predetermined number of individual memory segments in the shared memory of the host computer;

wherein reading from the shared memory in the accelerator further comprises reading unread data from the determined number of unread memory segments beginning with an identified next unread memory segment.