Matrix computation engine

• US 10,346,163 B2
• Filed: 11/01/2017
• Issued: 07/09/2019
• Est. Priority Date: 11/01/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bainville, Eric, Uliel, Tal, Norden, Erik, Gonion, Jeffry E., Sazegari, Ali
• Primary Examiner(s): Sandifer, Matthew D
• Application Number:
  US15/800,342
• Publication Number:
  US 20190129719A1
• Time in Patent Office: 615 Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00
• CPC Class Codes: G06F 9/30014 with variable precision
• CPC Class Codes: G06F 9/30036 Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
• CPC Class Codes: G06F 9/30043 LOAD or STORE instructions; Clear instruction
• CPC Class Codes: G06F 9/30109 having multiple operands in a single register
• CPC Class Codes: G06F 9/3887 controlled by a single instruction for multiple data lanes [SIMD]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

In an embodiment, a matrix computation engine is configured to perform matrix computations (e.g. matrix multiplications). The matrix computation engine may perform numerous matrix computations in parallel, in an embodiment. More particularly, the matrix computation engine may be configured to perform numerous multiplication operations in parallel on input matrix elements, generating resulting matrix elements. In an embodiment, the matrix computation engine may be configured to accumulate results in a result memory, performing multiply-accumulate operations for each matrix element of each matrix.

Claims

1. An apparatus comprising:

a processor configured to fetch a matrix computation instruction; and

a matrix computation engine coupled to the processor, wherein;

the matrix computation engine is configured to perform a matrix multiplication operation in response to the matrix computation instruction;

the matrix computation engine comprises at least two input memories configured to store input vectors of matrices for the matrix multiplication operation and an output memory configured to accumulate an output vector of matrices; and

the matrix computation engine comprises a multiply accumulation (MAC) circuit configured to perform the matrix multiplication operation on the input vectors of matrices, generating the output vector of matrices for the output memory.

2. The apparatus as recited in claim 1 wherein the MAC circuit comprises a plurality of MACs, each MAC configured to generate one matrix element of one matrix in the output vector.

3. The apparatus as recited in claim 2 wherein a number of the plurality of MACs is equal to a number of matrix elements in the output vector.

4. The apparatus as recited in claim 2 wherein a number of the plurality of MACs is less than a number of matrix elements in the output vector, and wherein the matrix computation instruction is completed by iterating the plurality of MACs with different portions of the input vectors and the output vector.

5. The apparatus as recited in claim 1 wherein the MAC circuit is configured to perform matrix multiplications on matrix elements having a first size, and wherein a first input vector stored in one of the at least two vector memories includes matrix elements having a second size less than a first size, wherein the matrix computation engine is configured to map matrix elements from the first input vector to second matrix elements having the first size using a lookup table, wherein the second matrix elements are provided to the MAC circuit for the matrix multiplication operations.

6. The apparatus as recited in claim 5 wherein the lookup table is programmable from the processor.

7. The apparatus as recited in claim 6 the lookup table is stored in one of the at least two input memories.

8. A matrix computation engine comprising:

a circuit configured to perform a matrix multiplication operation on a first vector operand and a second vector operand, producing a resulting output vector;

a first operand memory coupled to the circuit, wherein the first operand memory is configured to store the first vector operand, wherein the first vector operand includes a first plurality of matrices having matrix elements of a first size;

a second operand memory coupled to the circuit, wherein the second operand memory is configured to store a second vector operand, wherein the second vector operand includes a second plurality of matrices;

wherein the circuit is configured to perform matrix multiplication operations on matrix elements having a second size greater than the first size; and

the matrix computation engine includes a lookup table to map the matrix elements of the first plurality of matrices to the second size.

9. The matrix computation engine as recited in claim 8 wherein the second plurality of matrix elements has matrix elements of the first size, and wherein the lookup table maps the matrix elements of the second plurality of matrices to the second size.

10. The matrix computation engine as recited in claim 8 wherein the circuit comprises a plurality of multiply-accumulate circuits, wherein the plurality of multiply-accumulate circuits are couple to receive input matrix elements of the second size corresponding to matrix elements from the first operand memory and the second operand memory and configured to generate result matrix elements of a third size greater than the second size.

11. The matrix computation engine as recited in claim 10 further comprising an output memory configured to store matrix multiplication results, wherein the plurality of multiply-accumulate circuits are coupled to receive matrix elements from the output memory to accumulate with a multiplication result of the input matrix elements, wherein the matrix elements from the output memory are of the third size.

12. The matrix computation engine as recited in claim 10 where a number of the plurality of multiply-accumulate circuits is equal to a number of matrix elements in the output memory.

13. The matrix computation engine as recited in claim 10, a number of the plurality of multiply-accumulate circuits is less than a number of matrix elements in the output vector, and wherein the matrix computation instruction is completed by iterating the plurality of multiply-accumulate circuits with different portions of the input vectors and the output memory.

14. The matrix computation engine as recited in claim 8 wherein the first plurality of matrices are stored in column major order.

15. The matrix computation engine as recited in claim 8 wherein the first plurality of matrices are stored in row major order.

16. The matrix computation engine as recited in claim 8 wherein the lookup table is programmable.

17. The matrix computation engine as recited in claim 16 wherein the lookup table is stored in one of the first operand memory and the second operand memory.

18. A matrix computation engine comprising:

a first operand memory configured to store a first plurality of matrices;

a second operand memory configured to store a second plurality of matrices; and

a plurality of multiply-accumulate circuits coupled to the first operand memory and the second operand memory, wherein each multiply-accumulate circuit of the plurality of multiply accumulate circuits is configured to multiply a respective matrix element of the first plurality of matrices and a corresponding matrix element of the second plurality of matrices.

19. The matrix computation engine as recited in claim 18 further comprising an output memory configured to store a third plurality of matrices, wherein each multiply-accumulate circuit of the plurality of multiply-accumulate circuits is configured to add a corresponding matrix element in the third plurality of matrices to a result of the multiplication and write a second result of the addition to the output memory.

20. The matrix computation engine as recited in claim 18 wherein the plurality of multiply-accumulate circuits are configured to multiple matrix elements of a first size, and wherein at least the first plurality of matrices have matrix elements of a second size that is smaller than the first size, and wherein the matrix computation engine comprises a lookup table mapping matrix elements of the second size to matrix elements of the first size.