Apparatus and methods for matrix multiplication

• US 10,592,241 B2
• Filed: 10/25/2018
• Issued: 03/17/2020
• Est. Priority Date: 04/26/2016
• Status: Active Grant

• Current Assignee: Cambricon Technologies Corporation Limited
• Original Assignee: Cambricon Technologies Corporation Limited
• Inventors: Zhang, Xiao, Liu, Shaoli, Chen, Tianshi, Chen, Yunji
• Primary Examiner(s): Pan, Daniel H
• Application Number:
  US16/171,291
• Publication Number:
  US 20190065190A1
• Time in Patent Office: 509 Days
• Field of Search: 712 2- 9, 712 19, 712 22, 712 31, 712221, 712222, 712223, 712225, 706 27, 706 31, 706 25, 706 26, 706 28
• CPC Class Codes: G06F 15/8053 Vector processors
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 7/78 for changing the order of data flow, e.g. matrix transposition or LIFO buffers; Overflow or underflow handling therefor
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06F 9/30032 Movement instructions, e.g. MOVE, SHIFT, ROTATE, SHUFFLE
• CPC Class Codes: G06F 9/30036 Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
• CPC Class Codes: G06F 9/355 Indexed addressing
• CPC Class Codes: G06F 9/3557 using program counter as base address
• CPC Class Codes: G06F 9/3836 Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution
• CPC Class Codes: G06F 9/3877 using a slave processor, e.g. coprocessor peripheral processor G06F13/12; vector processor G06F15/8053
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Aspects for matrix multiplication in neural network are described herein. The aspects may include a master computation module configured to receive a first matrix and transmit a row vector of the first matrix. In addition, the aspects may include one or more slave computation modules respectively configured to store a column vector of a second matrix, receive the row vector of the first matrix, and multiply the row vector of the first matrix with the stored column vector of the second matrix to generate a result element. Further, the aspects may include an interconnection unit configured to combine the one or more result elements generated respectively by the one or more slave computation modules to generate a row vector of a result matrix and transmit the row vector of the result matrix to the master computation module.

Claims

1. An apparatus for matrix multiplication in a neural network, comprising:

a master computation module configured to

receive, in response to an instruction, a first matrix, and

transmit a row vector of the first matrix;

one or more slave computation modules respectively configured to

store a column vector of a second matrix,

receive the row vector of the first matrix, and

multiply, in response to the instruction, the row vector of the first matrix with the stored column vector of the second matrix to generate a result element; and

an interconnection unit configured to

combine the one or more result elements generated respectively by the one or more slave computation modules to generate a row vector of a result matrix, and

transmit the row vector of the result matrix to the master computation module,

wherein each of the one or more slave computation modules includes;

a slave neuron caching unit configured to store the column vector of the second matrix,

one or more multipliers configured to respectively multiply one or more first elements in the row vector of the first matrix with one or more second elements in the stored column vector of the second matrix to generate one or more multiplication results;

an adder configured to add the one or more multiplication results to generate an intermediate value of the row vector of the result matrix; and

an accumulator configured to accumulate the one or more intermediate values to generate the result element.

2. The apparatus of claim 1, wherein the instruction is a matrix-multiply-vector (MMV) instruction that includes a first starting address of the first matrix, a first length of the first matrix, a first starting address of the column vector of the second matrix, and a length of the column vector.

3. The apparatus of claim 1, wherein the instruction is a vector-multiply-matrix (VMM) instruction that includes a first starting address of the row vector, a first length of the row vector, a second starting address of the second matrix, and a length of the second matrix.

4. The apparatus of claim 1, wherein the instruction is a matrix-multiply-matrix (MM) instruction that includes a first starting address of the first matrix, a first length of the first matrix, a second starting address of the second matrix, and a second length of the second matrix.

5. The apparatus of claim 1, wherein the instruction is a matrix-multiply-scalar (MMS) instruction that includes a first starting address of the first matrix, a first length of the first matrix, and a scalar value.

6. The apparatus of claim 1, wherein the master computation module is further configured to combine the row vectors of the result matrix into the result matrix.

7. The apparatus of claim 1, wherein the interconnection unit is further configured to

receive the result elements generated by the one or more slave computation modules;

and

combine the result elements to generate the row vector of the result matrix.

8. The apparatus of claim 1, wherein each of the one or more multipliers is configured to perform a multiplication between one of the first elements and one of the second elements.

9. The apparatus of claim 1, wherein the first matrix includes one or more elements that are equal to a scalar value received in the instruction.

10. A method for matrix multiplication in a neural network, comprising:

receiving, in response to an instruction, by a master computation module, a first matrix;

transmitting, by the master computation module, a row vector of the first matrix to one or more slave computation modules;

storing, by the one or more slave computation modules, a column vector of a second matrix;

multiplying, in response to the instruction, by the one or more slave computation modules, the row vector of the first matrix with the stored column vector of the second matrix to generate a result element;

combining, by an interconnection unit, the one or more result elements generated respectively by the one or more slave computation modules to generate a row vector of a result matrix;

transmitting, by the interconnection unit, the row vector of the result matrix to the master computation module;

storing, by a slave neuron caching unit of each of the one or more slave computation modules, the column vector of the second matrix;

multiplying, by one or more multipliers of each of the one or more slave computation modules, one or more first elements in the row vector of the first matrix with one or more second elements in the stored column vector of the second matrix to generate one or more multiplication results;

adding, by an adder of each of the one or more slave computation modules, the one or more multiplication results to generate an intermediate value of the row vector of the result matrix; and

accumulating, by an accumulator of each of the one or more slave computation modules, the one or more intermediate values to generate the result element.

11. The method of claim 10, wherein the instruction is a matrix-multiply-vector (MMV) instruction that includes a first starting address of the first matrix, a first length of the first matrix, a first starting address of the column vector of the second matrix, and a length of the column vector.

12. The method of claim 10, wherein the instruction is a vector-multiply-matrix (VMM) instruction that includes a first starting address of the row vector, a first length of the row vector, a second starting address of the second matrix, and a length of the second matrix.

13. The method of claim 10, wherein the instruction is a matrix-multiply-matrix (MM) instruction that includes a first starting address of the first matrix, a first length of the first matrix, a second starting address of the second matrix, and a second length of the second matrix.

14. The method of claim 10, wherein the instruction is a matrix-multiply-scalar (MMS) instruction that includes a first starting address of the first matrix, a first length of the first matrix, and a scalar value.

15. The method of claim 10, further comprising combining, by the master computation module, the row vectors of the result matrix into the result matrix.

16. The method of claim 10, further comprising:

receiving, by the interconnection unit, the result elements generated by the one or more slave computation modules; and

combining, by the interconnection unit, the result elements to generate the row vector of the result matrix.

17. The method of claim 10, wherein the multiplying further comprises calculating, by the one or more multipliers, a multiplication between one of the first elements and one of the second elements.

18. The method of claim 10, wherein the first matrix includes one or more elements that are equal to a scalar value received in the instruction.