DEVICE AND METHOD FOR ACCELERATING MATRIX MULTIPLY OPERATIONS

• US 20200133992A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Advanced Micro Devices, Inc.
• Original Assignee: Advanced Micro Devices, Inc.
• Inventors: Aga, Shaizeen, Jayasena, Nuwan, Rush, Allen H., Ignatowski, Michael
• Granted Patent: US 10,956,536 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 15/8007 single instruction multiple data [SIMD] multiprocessors
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 7/5324 partitioned, i.e. using repetitively a smaller parallel parallel multiplier or using an array of such smaller multipliers
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00

Abstract

A processing device is provided which comprises memory configured to store data and a plurality of processor cores in communication with each other via first and second hierarchical communication links. Processor cores of a first hierarchical processor core group are in communication with each other via the first hierarchical communication links and are configured to store, in the memory, a sub-portion of data of a first matrix and a sub-portion of data of a second matrix. The processor cores are also configured to determine a product of the sub-portion of data of the first matrix and the sub-portion of data of the second matrix, receive, from another processor core, another sub-portion of data of the second matrix and determine a product of the sub-portion of data of the first matrix and the other sub-portion of data of the second matrix.

Claims

1. A processing device comprising:

memory configured to store data; and

a plurality of processor cores in communication with each other via first hierarchical communication links and second hierarchical communication links, each processor core in a group of the plurality of processor cores being in communication with each other via the first hierarchical communication links and configured to;

store, in the memory, one of a plurality of sub-portions of data of a first matrix;

store, in the memory, one of a plurality of sub-portions of data of a second matrix;

determine a product of the one sub-portion of data of the first matrix and the one sub-portion of data of the second matrix;

receive, from another processor core of the group of processor cores, another of the sub-portions of data of the second matrix; and

determine a product of the one sub-portion of data of the first matrix and the other sub-portion of data of the second matrix.

2. The processing device according to claim 1, wherein each processor core of the group of processor cores is further configured to receive the other sub-portion of data of the second matrix from the other processor core without accessing the memory.

3. The processing device according to claim 1, wherein

the group of processor cores is one of a plurality of first hierarchical groups of processor cores;

and

the plurality of processor cores comprises a second hierarchical group of processor cores comprising the plurality of first hierarchical groups, the first hierarchical groups being in communication with each other via a plurality of second communication links,

each one of the first communication links extends a first distance, each one of the second communication links extends a second distance and the first distance is less than the second distance, and

the other sub-portion of data of the second matrix is received from the other processor core via one of the first communication links.

4. The processing device according to claim 3, wherein each processor core of the first hierarchical group of processor cores is further configured to determine dot products for the one sub-portion of data of the first matrix and each remaining sub-portion of data of the second matrix by receiving each remaining sub-portion of data of the second matrix from another processor core of the group of processor cores while the one sub-portion of data of the first matrix remains in the memory.

5. The processing device according to claim 1, wherein

the sub-portions of data of the first matrix are sub-portions of a row of the first matrix, and

the sub-portions of data of the second matrix are sub-portions of a column of the second matrix.

6. The processing device according to claim 1, wherein each processor core in the group of processor cores is further configured to communicate the other sub-portion of data of the second matrix to another one of the processor cores.

7. The processing device according to claim 1, wherein each of the plurality of processor cores is a compute unit comprising one or more single instruction, multiple data (SIMD) units.

8. A processing device comprising:

memory configured to store data; and

a plurality of processor cores in communication with each other via first hierarchical communication links, the plurality of processor cores comprising a first processor core and a second processor core, wherein

the first processor core is configured to;

determine a product of a first sub-portion of data of a first matrix received from the memory and a first sub-portion of data of a second matrix received from the memory; and

communicate, to the second processor core via one of the first hierarchical communication links, the first sub-portion of data of the second matrix; and

the second processor core is configured to;

receive the first sub-portion of data of the second matrix communicated by the first processor without accessing the memory; and

determine a product of the first sub-portion of data of the second matrix received from the first processor and a second sub-portion of data of the first matrix received from the memory.

9. The processing device according to claim 8, wherein

the first processor core is configured to determine the product and communicate the first sub-portion of data of the second matrix at a first stage;

and

the second processor core is configured to receive the first sub-portion of data of the second matrix and determine the product at a second stage after the first stage.

10. The processing device according to claim 8, wherein the first processor core is further configured to:

communicate the first sub-portion of data of the second matrix to the second processor core while the sub-portion of data of the first matrix remains in the memory; and

the second processor core is further configured to communicate, to a third processor core via one of the first hierarchical communication links, the first sub-portion of data of the second matrix while the second sub-portion of data of the first matrix remains in the memory.

11. The processing device according to claim 8, wherein each of the plurality of processor cores is a compute unit comprising one or more single instruction, multiple data (SIMD) units.

12. The processing device according to claim 8, wherein

the plurality of processor cores is one of a plurality of first hierarchical groups of processor cores, each processor core of a first hierarchical group being in communication with each other via a plurality of the first communication links;

and

the processing device further comprises a second hierarchical group of processor cores comprising the first hierarchical groups of processor cores, the first hierarchical groups being in communication with each other via a plurality of second communication links,

wherein each one of the first communication links extends a first distance, each one of the second communication links extends a second distance and the first distance is less than the second distance, and

the first sub-portion of data of the second matrix is provided to the second processor core via one of the first communication links.

13. The processing device according to claim 8, wherein

the first processor core is configured to receive remaining sub-portions of data of the second matrix from other processor cores of the plurality of processor cores and determine dot products for the remaining sub-portions of data of the second matrix and the first sub-portion of data of the first matrix while the first sub-portion of data of the first matrix remains in the memory, and

the second processor core is configured to receive the remaining sub-portions of data of the second matrix from the other processor cores of the plurality of processor cores and determine dot products for the remaining sub-portions of data of the second matrix and the second sub-portion of data of the first matrix while the second sub-portion of data of the first matrix remains in the memory.

14. The processing device according to claim 8, wherein

the first sub-portion of data of the first matrix is a sub-portion of a row of the first matrix;

and

the first sub-portion of data of the second matrix is a sub-portion of a column of the second matrix.

15. A method for use in a processing device having a plurality of processor cores for performing matrix multiplication, the method comprising:

receiving, from memory by a first processor core, a first sub-portion of data of a first matrix;

receiving, from the memory by the first processor core, a first sub-portion of data of a second matrix;

determining, by the first processor core, a product of the first sub-portion of data of the first matrix and the first sub-portion of data of the second matrix;

communicating, by the first processor core to a second processor core, the first sub-portion of data of the second matrix via one of a plurality of first hierarchical communication links;

receiving, from the memory by the second processor core, a second sub-portion of data of the first matrix; and

determining, by the second processor core, a product of the second sub-portion of data of the first matrix and the first sub-portion of data of the second matrix.

16. The method according to claim 15, further comprising communicating, by the first processor core, the first sub-portion of data of the second matrix to the second processor core without accessing the memory.

17. The method according to claim 15, further comprising communicating, by the second processor core, the first sub-portion of data of the second matrix to a third processor core.

18. The method according to claim 15, further comprising:

determining, by the first processor core, dot products for the first sub-portion of data of the first matrix and remaining sub-portions of data of the second matrix by receiving each remaining sub-portion of data of the second matrix from other processor cores via the plurality of first hierarchical communication links while the first sub-portion of data of the first matrix remains in the memory, and

determining, by the second processor core, dot products for the second sub-portion of data of the first matrix and the remaining sub-portions of data of the second matrix by receiving each remaining sub-portion of data of the second matrix from other processor cores via the plurality of first hierarchical communication links while the second sub-portion of data of the first matrix remains in the memory.

19. The method according to claim 15, wherein

the first sub-portion of data of the first matrix is a sub-portion of a row of the first matrix, and

the first sub-portion of data of the second matrix is a sub-portion of a column of the second matrix.

20. The method according to claim 15, wherein

the first processor core and the second processor core are cores of a first group of processor cores, the processor cores of the first group being in communication with each other via the plurality of first hierarchical communication links;

and

the first group of processor cores is in communication with a second group of processor cores via one of a plurality of second hierarchical communication links,

wherein each one of the first hierarchical communication links extends a first distance, each one of the second communication hierarchical links extends a second distance and the first distance is less than the second distance.