Outer Product Engine

• US 20180074824A1
• Filed: 09/13/2016
• Published: 03/15/2018
• Est. Priority Date: 09/13/2016
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Sazegari, Ali, Bainville, Eric, Gonion, Jeffry E., Williams, III, Gerard R., Beaumont-Smith, Andrew J.
• Application Number:
  US15/264,002
• Publication Number:
  US 20180074824A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06F 9/30036 Instructions to perform operations on packed data, e.g. vector, tile or matrix operations
• CPC Class Codes: G06F 9/30043 LOAD or STORE instructions; Clear instruction
• CPC Class Codes: G06F 9/30101 Special purpose registers
• CPC Class Codes: G06F 9/3802 Instruction prefetching
• CPC Class Codes: G06F 9/3867 using instruction pipelines
• CPC Class Codes: G06F 9/3877 using a slave processor, e.g. coprocessor peripheral processor G06F13/12; vector processor G06F15/8053
• CPC Class Codes: G06F 9/3893 controlled in tandem, e.g. multiplier-accumulator

Abstract

In an embodiment, an outer product engine is configured to perform outer product operations. The outer product engine may perform numerous multiplication operations in parallel on input vectors, in an embodiment, generating a resulting outer product matrix. In an embodiment, the outer product engine may be configured to accumulate results in a result matrix, performing fused multiply add (FMA) operations to produce the outer product elements (multiply) and to accumulate the outer product elements with previous elements from the result matrix memory (add). A processor may fetch outer product instructions, and may transmit the instructions to the outer product engine when the instructions become non-speculative in an embodiment. The processor may be configured to retire the outer product instructions responsive to transmitting them to the outer product engine.

Claims

1. An apparatus comprising:

a processor configured to fetch an outer product instruction; and

an outer product engine coupled to the processor, wherein;

the outer product engine is configured to perform an outer product operation specified for the outer product instruction;

the outer product engine comprises at least two input memories configured to store input vectors for the outer product operation and an output memory configured to accumulate outer product results;

the processor is configured to retire the outer product instruction in response to transmitting the outer product operation to the outer product engine and prior to the outer product operation being completed by the outer product engine; and

a size of each input memory exceeds a size of vector registers in the processor.

2. The apparatus as recited in claim 1 wherein the processor is configured to transmit the outer product instruction to the outer product engine responsive to the outer product instruction becoming non-speculative in the processor.

3. The apparatus as recited in claim 2 wherein the outer product instruction comprises a load/store operation, and wherein the processor is configured to translate a virtual address of the load/store operation to a physical address prior to transmitting the outer product instruction to the outer product engine.

4. The apparatus as recited in claim 3 wherein, if the load/store operation misses in one or more caches to which the outer product engine has access, the latency of the cache miss is experienced in the outer product engine after the processor has retired the outer product instruction.

5. The apparatus as recited in claim 1 wherein the outer product instruction specifies a fused multiply-add operation, wherein the multiply portion of the fused multiply-add operation produces the outer product of the input vectors.

6. The apparatus as recited in claim 5 wherein the add portion of the fused multiply-add operation adds each element of the outer product to a corresponding element read from the output memory.

7. The apparatus as recited in claim 6 wherein outer product engine is configured to write the sum of each element and the corresponding element into the output memory.

8. The apparatus as recited in claim 1 wherein:

the outer product engine is configured to perform the outer product operation on a plurality of element sizes of the input vectors;

a first input memory of the input memories is sized to store a first number of elements of a minimum element size of the plurality of element sizes for a first input vector;

a second input memory of the input memories is sized to store a second number of elements of the minimum element size for a second input vector;

the output memory is sized to store a third number of elements that are results of the outer product operation of the first number of elements and the second number of elements, and the third number is the first number multiplied by the second number.

9. The apparatus as recited in claim 8 wherein the first number and the second number are equal.

10. The apparatus as recited in claim 8 wherein:

the first input memory is sized to store a fourth number of elements of a second element size of the plurality of element sizes and the fourth number is determined from the first number multiplied by a ratio of the minimum element size to the second element size;

the second input memory is sized to store a fifth number of elements of the second element size and the fifth number is determined from the second number multiplied by the ratio of the minimum element size to the second element size; and

the third memory stores a sixth number of elements that are results of the outer product operation on the fourth number of elements and the fifth number of elements during use, and a portion of the third memory is unused at the second element size.

11. An outer product engine comprising:

a circuit configured to perform an outer product operation on a first vector operand and a second vector operand, producing a resulting outer product matrix;

a first operand memory coupled to the circuit, wherein the first operand memory is sized to store a first number of elements of the first vector operand at a first element size and a second number of elements of the first vector operand at a second element size, wherein the second element size is larger than the first element size;

a second operand memory coupled to the circuit, wherein the second operand memory is sized to store a third number of elements of the second vector operand at the first element size and a fourth number of elements of the second vector operand at the second element size;

a third memory coupled to the circuit, wherein the third memory is sized to store the resulting outer product matrix for the outer product operation performed on the first element size, and wherein a portion of the third memory is unused for the outer product operation performed at the second element size.

12. The outer product engine is recited in claim 11 wherein the circuit is a fused multiply-add array, wherein a multiply portion of the fused multiply-add array is configured to perform a plurality of multiply operations on respective elements of the first operand and the second operand.

13. The outer product engine as recited in claim 12 wherein an add portion of the fused multiply-add array is further configured to add products of the plurality of multiply operations to respective data read from the third memory, and to write results of the addition to the third memory.

14. The outer product engine as recited in claim 12 wherein the multiply-add array is further configured to subtract products of plurality of multiply operations from respective data read from the third memory, and to write results of the subtraction to the third memory.

15. The outer product engine as recited in claim 11 further comprising an instruction buffer coupled the circuit and configured to store one or more outer product instructions received from a processor.

16. The outer product engine as recited in claim 15 wherein the instruction buffer is further configured to store load/store operations to read data to and write data from the first vector memory, the second vector memory, and the third memory.

17. An apparatus comprising:

a processor configured to fetch an outer product instruction; and

an outer product engine coupled to the processor, wherein;

the outer product engine is configured to perform an outer product operation specified for the outer product instruction;

the outer product engine comprises at least two input memories configured to store input vectors for the outer product operation and an output memory configured to accumulate outer product results; and

the outer product engine is configured to read the elements of the output memory and accumulate corresponding elements of the outer product operation with existing data in the output memory in response to the outer product instruction.

18. The apparatus as recited in claim 17 wherein the accumulation is addition.

19. The apparatus as recited in claim 17 wherein the accumulation is subtraction.

20. The apparatus as recited in claim 17 wherein the processor is configured transmit the outer product instruction to the outer product engine responsive to the outer product instruction becoming non-speculative in the processor.