Data processing in a hybrid computing environment

• US 8,145,749 B2
• Filed: 08/11/2008
• Issued: 03/27/2012
• Est. Priority Date: 08/11/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Carey, James E., Markland, Matthew W.
• Primary Examiner(s): Follansbee, John
• Assistant Examiner(s): Mejia, Anthony
• Application Number:
  US12/189,342
• Publication Number:
  US 20100036940A1
• Time in Patent Office: 1,324 Days
• Field of Search: 709/200, 709/224
• US Class Current: 709/224
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake

Abstract

Data processing in a hybrid computing environment that includes a host computer and an accelerator, the host and the accelerator adapted to one another for data communications by a system level message passing module and a plurality data communications fabrics of at least two different fabric types, the data processing including: monitoring data communications performance for a plurality of data communications modes; receiving, from an application program on the host computer, a request to transmit data according to a data communications mode from the host computer to the accelerator; determining, in dependence upon the monitored performance, whether to transmit the data according to the requested data communications mode; and if the data is not to be transmitted according to the requested data communications mode: selecting, in dependence upon the monitored performance, another data communications mode for transmitting the data and transmitting the data according to the selected data communications mode.

Claims

1. A method of data processing in a hybrid computing environment, the hybrid computing environment comprising:

a host computer having a host computer architecture;

an accelerator having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions;

the host computer and the accelerator adapted to one another for data communications by a system level message passing module;

the host computer and the accelerator adapted to one another for data communications by two or more data communications fabrics of at least two different fabric types;

the method comprising;

monitoring, by the system level message passing module, data communications performance for a plurality of data communications modes between the host computer and the accelerator;

receiving, by the system level message passing module from an application program on the host computer, a request to transmit data according to a data communications mode from the host computer to the accelerator;

determining, by the system level message passing module, in dependence upon the monitored performance whether to transmit the data according to the requested data communications mode; and

if the data is not to be transmitted according to the requested data communications mode;

selecting, by the system level message passing module, in dependence upon the monitored performance another data communications mode for transmitting the data and transmitting the data by the system level message passing module according to the selected data communications mode;

wherein selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance a data communications link through which to transmit the data.

2. The method of claim 1 wherein each data communications mode comprises a data communications fabric type, a data communications link, and a data communications protocol.

3. The method of claim 1 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data by a specified fabric type;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit data by the requested fabric type;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications fabric type by which to transmit the data; and

transmitting the data according to the selected data communications mode further comprises transmitting the data by the selected data communications fabric type.

4. The method of claim 1 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data through a specified data communications link from the host computer to the accelerator;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data through the requested data communications link; and

transmitting the data according to the selected data communications mode further comprises transmitting the data through the selected data communications link.

5. The method of claim 1 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data from the host computer to the accelerator according to a protocol;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data according to the requested protocol;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications protocol; and

transmitting the data according to the selected data communications mode further comprises transmitting the data according to the selected protocol.

6. The method of claim 1 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring a number of requests in a message transmit request queue for a data communications link; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the number of requests in the message transmit request queue exceeds a predetermined threshold.

7. The method of claim 1 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring utilization of a shared memory space; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the utilization of the shared memory space exceeds a predetermined threshold.

8. The method of claim 1 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises aggregating data communications performance information across a plurality of instances of the application program executing on a plurality of host computers in a parallel computing environment.

9. The method of claim 1 wherein selecting another data communications mode for transmitting the data further comprises selecting a data communications protocol in dependence upon data communications message size.

10. A hybrid computing environment comprising:

a host computer having a host computer architecture;

an accelerator having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions;

the host computer and the accelerator adapted to one another for data communications by a system level message passing module;

the host computer and the accelerator adapted to one another for data communications by two or more data communications fabrics of at least two different fabric types;

the system level message passing module comprising computer program instructions that, when executed, cause the hybrid computing environment to carry out the steps of;

monitoring data communications performance for a plurality of data communications modes between the host computer and the accelerator;

receiving, from an application program on the host computer, a request to transmit data according to a data communications mode from the host computer to the accelerator;

determining, in dependence upon the monitored performance, whether to transmit the data according to the requested data communications mode; and

if the data is not to be transmitted according to the requested data communications mode;

selecting, in dependence upon the monitored performance, another data communications mode for transmitting the data and transmitting the data according to the selected data communications mode;

wherein selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance a data communications link through which to transmit the data.

11. The hybrid computing environment of claim 10 wherein each data communications mode comprises a data communications fabric type, a data communications link, and a data communications protocol.

12. The hybrid computing environment of claim 10 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data by a specified fabric type;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit data by the requested fabric type;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications fabric type by which to transmit the data; and

transmitting the data according to the selected data communications mode further comprises transmitting the data by the selected data communications fabric type.

13. The hybrid computing environment of claim 10 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data through a specified data communications link from the host computer to the accelerator;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data through the requested data communications link; and

transmitting the data according to the selected data communications mode further comprises transmitting the data through the selected data communications link.

14. The hybrid computing environment of claim 10 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data from the host computer to the accelerator according to a protocol;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data according to the requested protocol;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications protocol; and

transmitting the data according to the selected data communications mode further comprises transmitting the data according to the selected protocol.

15. The hybrid computing environment of claim 10 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring a number of requests in a message transmit request queue for a data communications link; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the number of requests in the message transmit request queue exceeds a predetermined threshold.

16. The hybrid computing environment of claim 10 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring utilization of a shared memory space; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the utilization of the shared memory space exceeds a predetermined threshold.

17. The hybrid computing environment of claim 10 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises aggregating data communications performance information across a plurality of instances of the application program executing on a plurality of host computers in a parallel computing environment.

18. The hybrid computing environment of claim 10 wherein selecting another data communications mode for transmitting the data further comprises selecting a data communications protocol in dependence upon data communications message size.

19. A computer program product for data processing in a hybrid computing environment, the hybrid computing environment comprising:

a host computer having a host computer architecture;

an accelerator having an accelerator architecture, the accelerator architecture optimized, with respect to the host computer architecture, for speed of execution of a particular class of computing functions;

the host computer and the accelerator adapted to one another for data communications by a system level message passing module;

the host computer and the accelerator adapted to one another for data communications by two or more data communications fabrics of at least two different fabric types;

the computer program product disposed in a computer readable, non-transmission medium, the computer program product comprising computer program instructions that, when executed, cause the hybrid computing environment to carry out the steps;

monitoring, by the system level message passing module, data communications performance for a plurality of data communications modes between the host computer and the accelerator;

receiving, by the system level message passing module from an application program on the host computer, a request to transmit data according to a data communications mode from the host computer to the accelerator;

determining, by the system level message passing module, in dependence upon the monitored performance whether to transmit the data according to the requested data communications mode; and

if the data is not to be transmitted according to the requested data communications mode;

selecting, by the system level message passing module, in dependence upon the monitored performance another data communications mode for transmitting the data and transmitting the data by the system level message passing module according to the selected data communications mode;

wherein selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance a data communications link through which to transmit the data.

20. The computer program product of claim 19 wherein each data communications mode comprises a data communications fabric type, a data communications link, and a data communications protocol.

21. The computer program product of claim 19 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data by a specified fabric type;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit data by the requested fabric type;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications fabric type by which to transmit the data; and

transmitting the data according to the selected data communications mode further comprises transmitting the data by the selected data communications fabric type.

22. The computer program product of claim 19 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data through a specified data communications link from the host computer to the accelerator;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data through the requested data communications link; and

transmitting the data according to the selected data communications mode further comprises transmitting the data through the selected data communications link.

23. The computer program product of claim 19 wherein:

receiving a request to transmit data according to a data communications mode further comprises receiving a request to transmit data from the host computer to the accelerator according to a protocol;

determining whether to transmit the data according to the requested data communications mode further comprises determining whether to transmit the data according to the requested protocol;

selecting another data communications mode for transmitting the data further comprises selecting in dependence upon the monitored performance another data communications protocol; and

transmitting the data according to the selected data communications mode further comprises transmitting the data according to the selected protocol.

24. The computer program product of claim 19 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring a number of requests in a message transmit request queue for a data communications link; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the number of requests in the message transmit request queue exceeds a predetermined threshold.

25. The computer program product of claim 19 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises monitoring utilization of a shared memory space; and

determining whether to transmit the data according to the requested data communications mode further comprises determining whether the utilization of the shared memory space exceeds a predetermined threshold.

26. The computer program product of claim 19 wherein:

monitoring data communications performance for a plurality of data communications modes further comprises aggregating data communications performance information across a plurality of instances of the application program executing on a plurality of host computers in a parallel computing environment.

27. The computer program product of claim 19 wherein selecting another data communications mode for transmitting the data further comprises selecting a data communications protocol in dependence upon data communications message size.