Computation engine with strided dot product

• US 10,642,620 B2
• Filed: 04/05/2018
• Issued: 05/05/2020
• Est. Priority Date: 04/05/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Uliel, Tal, Bainville, Eric, Gonion, Jeffry E., Sazegari, Ali
• Primary Examiner(s): Pan, Daniel H
• Application Number:
  US15/946,724
• Publication Number:
  US 20190310855A1
• Time in Patent Office: 761 Days
• Field of Search: 712 2- 9, 712214, 712215, 712221, 712223, 712225, 712227, 708501, 708603, 708607
• CPC Class Codes: G06F 15/8053 Vector processors
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 2212/454 Vector or matrix data
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06F 9/30014 with variable precision
• CPC Class Codes: G06F 9/30029 Logical and Boolean instructions, e.g. XOR, NOT
• CPC Class Codes: G06F 9/30036 Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
• CPC Class Codes: G06F 9/30043 LOAD or STORE instructions; Clear instruction
• CPC Class Codes: G06F 9/3802 Instruction prefetching
• CPC Class Codes: G06F 9/3836 Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution

Abstract

In an embodiment, a computation engine may perform dot product computations on input vectors. The dot product operation may have a first operand and a second operand, and the dot product may be performed on a subset of the vector elements in the first operand and each of the vector elements in the second operand. The subset of vector elements may be separated in the first operand by a stride that skips one or more elements between each element to which the dot product operation is applied. More particularly, in an embodiment, the input operands of the dot product operation may be a first vector having second vectors as elements, and the stride may select a specified element of each second vector.

Claims

1. A system comprising:

a processor configured to issue a first instruction to a computation engine;

the computation engine coupled to the processor, wherein;

the computation engine comprises;

a first memory storing, during use, a first plurality of input vectors that include first vector elements, and

a second memory storing, during use, a second plurality of input vectors that include second vector elements; and

the computation engine is configured, in response to the first instruction, to compute a dot product of a subset of the first vector elements and each of the second vector elements, wherein respective elements of the subset of the first vector elements are separated in the first plurality of input vectors by other elements not in the subset, wherein a number of the other elements is specified by a stride corresponding to the first instruction, and wherein the computation engine is further configured, in response to the first instruction, not to apply the dot product to the first vector elements that are not in the subset.

2. The system as recited in claim 1 wherein the computation engine comprises a third memory storing, during use, a plurality of results corresponding to respective dot products computed responsive to the first instruction, and wherein the computation engine is configured to sum a current value of a respective one of the plurality of results with the respective dot product and write the sum to the third memory.

3. The system as recited in claim 2 wherein the stride is based on a first precision of the plurality of results and a second precision of the first vector elements.

4. The system as recited in claim 3 wherein the stride is explicitly specified in the first instruction.

5. The system as recited in claim 3 wherein the stride is determined from the first precision and the second precision, and wherein the first precision and the second precision are explicitly specified in the first instruction.

6. The system as recited in claim 3 wherein the stride is based on a ratio of a maximum value of the first precision and a minimum value of the second precision.

7. The system as recited in claim 1 wherein the stride causes a selection of an initial element of each of the second plurality of input vectors.

8. A circuit comprising:

a first input memory storing a first plurality of input vectors, during use;

a second input memory storing a second plurality of input vectors, during use; and

a compute circuit coupled to the first input memory and the second input memory, wherein the compute circuit is configured, responsive to a first instruction, to multiply selected vector elements of the first plurality of input vectors by the second plurality of input vectors, wherein the selected vector elements are separated in the first plurality of input vectors by unselected vector elements of the first plurality of input vectors, wherein a number of the unselected vector elements is specified by a stride associated with the first instruction, and wherein the compute circuit is configured, responsive to the first instruction, not to multiply the unselected vector elements of the first plurality of input vectors by the second plurality of input vectors.

9. The circuit as recited in claim 8 wherein the compute circuit comprises a plurality of multiply accumulate (MAC) circuits, each of the plurality of MAC circuits configured to perform one of the multiplications of one of the selected vector elements and a respective vector element of one of the second plurality of input vectors.

10. The circuit as recited in claim 9 further comprising an output memory storing, during use, a plurality of results corresponding to the first instruction, and wherein the plurality of MAC circuits are configured to sum a current value of a respective one of the plurality of results with a respective multiplication result and write the sum to the output memory.

11. The circuit as recited in claim 10 wherein the stride is based on a first precision of the plurality of results and a second precision of vector elements of the first plurality of input vectors.

12. The circuit as recited in claim 11 wherein the stride is explicitly specified in the first instruction.

13. The circuit as recited in claim 11 wherein the stride is determined from the first precision and the second precision, and wherein the first precision and the second precision are explicitly specified in the first instruction.

14. The circuit as recited in claim 11 wherein the stride is based on a ratio of a maximum value of the first precision and a minimum value of the second precision.

15. The circuit as recited in claim 10 wherein a number of the plurality of MAC circuits is equal to a number of vector elements of the first plurality of input vectors multiplied by a number of vector elements of the second plurality of input vectors and divided by the stride.

16. The circuit as recited in claim 10 wherein a number of the plurality of MAC circuits is less than a number of vector elements of the first plurality of input vectors multiplied by a number of vector elements of the second plurality of input vectors and divided by the stride, and wherein the first instruction is completed using multiple iterations in the compute circuit.

17. The circuit as recited in claim 8 wherein the stride causes a selection of an initial element of each of the second plurality of input vectors.

18. A system comprising:

a processor configured to issue a first instruction to a computation engine;

the computation engine coupled to the processor, wherein;

the computation engine comprises;

a first memory storing, during use, a first plurality of input vectors that include first vector elements,

a second memory storing, during use, a second plurality of input vectors that include second vector elements, and

a third memory storing, during use, a plurality of results; and

the computation engine further comprises a plurality of multiply accumulate (MAC) circuits, wherein the plurality of MAC circuits are configured to multiply selected first vector elements by second vector elements to generate multiplication results and to add the multiplication results to the plurality of results, and the computation engine performs the multiplications and additions in response to the first instruction, and wherein the selected first vector elements are identified using a stride corresponding to the first instruction and the selected first vector elements are separated in the first plurality of input vectors by non-selected vector elements, wherein a number of the non-selected vector elements is specified by the stride, and wherein the computation engine is configured, in response to the first instruction, not to operate upon non-selected first vector elements of the first plurality of input vectors.

19. The system as recited in claim 18 wherein the stride is based on a first precision of the plurality of results and a second precision of the first vector elements.

20. The system as recited in claim 18 wherein the stride is explicitly specified in the first instruction.