MECHANISMS TO IMPROVE DATA LOCALITY FOR DISTRIBUTED GPUS

• US 20180115496A1
• Filed: 10/21/2016
• Published: 04/26/2018
• Est. Priority Date: 10/21/2016
• Status: Active Application

• Current Assignee: Advanced Micro Devices, Inc.
• Original Assignee: Advanced Micro Devices, Inc.
• Inventors: Eckert, Yasuko, Kayiran, Onur, Jayasena, Nuwan S., Loh, Gabriel H., Zhang, Dong Ping
• Application Number:
  US15/331,002
• Publication Number:
  US 20180115496A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Systems, apparatuses, and methods for implementing mechanisms to improve data locality for distributed processing units are disclosed. A system includes a plurality of distributed processing units (e.g., GPUs) and memory devices. Each processing unit is coupled to one or more local memory devices. The system determines how to partition a workload into a plurality of workgroups based on maximizing data locality and data sharing. The system determines which subset of the plurality of workgroups to dispatch to each processing unit of the plurality of processing units based on maximizing local memory accesses and minimizing remote memory accesses. The system also determines how to partition data buffer(s) based on data sharing patterns of the workgroups. The system maps to each processing unit a separate portion of the data buffer(s) so as to maximize local memory accesses and minimize remote memory accesses.

Claims

1. A system comprising:

a plurality of memory devices; and

a plurality of processing units, wherein each processing unit of the plurality of processing units is coupled to one or more local memory devices of the plurality of memory devices;

wherein the system is configured to;

partition a workload into a plurality of workgroups;

partition one or more data buffers into a plurality of data partitions; and

determine how to dispatch workgroups to the plurality of processing units and map data partitions to the plurality of memory devices based on minimizing accesses to non-local memory devices.

2. The system as recited in claim 1, wherein the system is further configured to:

partition the workload into a plurality of workgroups based on a dimensionality of the workload; and

dispatch M consecutive workgroups to each processing unit, wherein M is equal to a total number of workgroups divided by a number of processing units.

3. The system as recited in claim 2, wherein the system is further configured to partition the one or more data buffers along a same dimensionality as the workload and map data partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

4. The system as recited in claim 3, wherein the one or more data buffers are partitioned at a finer granularity than the workload.

5. The system as recited in claim 1, wherein the system is further configured to:

determine data sharing patterns of the plurality of workgroups to identify workgroups that share a threshold amount of data;

determine which subset of the plurality of workgroups to dispatch to each processing unit of the plurality of processing units based on an analysis of the data sharing patterns;

determine how to partition the one or more data buffers based on the data sharing patterns of the plurality of workgroups; and

map partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

6. The system as recited in claim 1, wherein the system comprises a dispatch table to specify which workgroup identifier (ID) maps to which processing unit on a kernel basis.

7. The system as recited in claim 1, wherein the system is configured to:

identify two or more workgroups that share a threshold amount of data; and

dispatch the two or more workgroups to a first processing unit.

8. A method comprising:

partitioning a workload into a plurality of workgroups;

partitioning one or more data buffers into a plurality of data partitions; and

determining how to dispatch workgroups to a plurality of processing units and map data partitions to the local memory devices of the plurality of processing units based on minimizing non-local memory accesses.

9. The method as recited in claim 8, further comprising:

partitioning the workload into a plurality of workgroups based on a dimensionality of the workload; and

dispatching M consecutive workgroups to each processing unit, wherein M is equal to a total number of workgroups divided by a number of processing units.

10. The method as recited in claim 9, further comprising partitioning the one or more data buffers along a same dimensionality as the workload and mapping data partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

11. The method as recited in claim 10, further comprising partitioning the one or more data buffers at a finer granularity than the workload.

12. The method as recited in claim 8, further comprising:

determining data sharing patterns of the plurality of workgroups to identify workgroups that share a threshold amount of data;

determining which subset of the plurality of workgroups to dispatch to each processing unit of the plurality of processing units based on an analysis of the data sharing patterns;

determining how to partition the one or more data buffers based on the data sharing patterns of the plurality of workgroups; and

mapping partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

13. The method as recited in claim 8, further comprising utilizing a dispatch table to specify which workgroup identifier (ID) maps to which processing unit on a kernel basis.

14. The method as recited in claim 8, further comprising:

identifying two or more workgroups that share a threshold amount of data; and

dispatching the two or more workgroups to a first processing unit.

15. A non-transitory computer readable storage medium storing program instructions, wherein the program instructions are executable by a processor to:

partition a workload into a plurality of workgroups;

partition one or more data buffers into a plurality of data partitions; and

determine how to dispatch workgroups to a plurality of processing units and map data partitions to the local memory devices of the plurality of processing units based on minimizing non-local memory accesses.

16. The non-transitory computer readable storage medium as recited in claim 15, wherein the program instructions are further executable by a processor to:

partition the workload into a plurality of workgroups based on a dimensionality of the workload; and

dispatch M consecutive workgroups to each processing unit, wherein M is equal to a total number of workgroups divided by a number of processing units.

17. The non-transitory computer readable storage medium as recited in claim 16, wherein the program instructions are further executable by a processor to partition the one or more data buffers along a same dimensionality as the workload and map data partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

18. The non-transitory computer readable storage medium as recited in claim 15, wherein the program instructions are further executable by a processor to partition the one or more data buffers at a finer granularity than the workload.

19. The non-transitory computer readable storage medium as recited in claim 15, wherein the program instructions are further executable by a processor to:

determine data sharing patterns of the plurality of workgroups to identify workgroups that share a threshold amount of data;

determine which subset of the plurality of workgroups to dispatch to each processing unit of the plurality of processing units based on an analysis of the data sharing patterns;

determine how to partition the one or more data buffers based on the data sharing patterns of the plurality of workgroups; and

map partitions to memory devices in a manner such that workgroups perform an increased number of local memory accesses as compared to non-local memory accesses.

20. The non-transitory computer readable storage medium as recited in claim 15, wherein the program instructions are further executable by a processor to:

identify two or more workgroups that share a threshold amount of data; and

dispatch the two or more workgroups to a first processing unit.