Dynamic precision management for integer deep learning primitives

• US 10,643,297 B2
• Filed: 01/29/2018
• Issued: 05/05/2020
• Est. Priority Date: 05/05/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Mellempudi, Naveen, Mudigere, Dheevatsa, Das, Dipankar, Sridharan, Srinivas
• Primary Examiner(s): Crawford, Jacinta M
• Application Number:
  US15/881,991
• Publication Number:
  US 20180322607A1
• Time in Patent Office: 827 Days
• Field of Search: 345501
• CPC Class Codes: G06F 17/153 Multidimensional correlation or convolution
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 2207/382 Reconfigurable for different fixed word lengths multigauge devices G06F2207/3828
• CPC Class Codes: G06F 2207/4824 Neural networks
• CPC Class Codes: G06F 5/01 for shifting, e.g. justifying, scaling, normalising digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00
• CPC Class Codes: G06F 7/501 Half or full adders, i.e. basic adder cells for one denomination
• CPC Class Codes: G06F 7/523 Multiplying only
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining

Abstract

One embodiment provides for a graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising compute unit including a hardware logic unit having dynamic precision fixed-point logic; a decode unit to decode an instruction for execution by the compute unit, the instruction to cause the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation, the dynamic precision manager to adjust the precision of the compute operation to prevent an arithmetic overflow.

Claims

1. A graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising:

compute unit including a hardware logic unit having dynamic precision fixed-point logic;

a decode unit to decode an instruction for execution by the compute unit, the instruction to cause the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and

a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation, the dynamic precision manager to adjust the precision of the compute operation to prevent an arithmetic overflow.

2. The graphics processing unit as in claim 1, the dynamic precision fixed-point logic of the compute unit including an integer compute unit.

3. The graphics processing unit as in claim 2, wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.

4. The graphics processing unit as in claim 3, wherein the register is to store a dynamic fixed-point scale factor.

5. The graphics processing unit as in claim 4, the instruction to cause the compute unit to perform a matrix arithmetic operation for a convolution operation on input data to the neural network.

6. The graphics processing unit as in claim 5, wherein the matrix arithmetic operation includes an addition or multiplication operation.

7. The graphics processing unit as in claim 6, wherein the matrix arithmetic operation includes a multiply and accumulate operation.

8. The graphics processing unit as in claim 7, the dynamic precision manager to dynamically adjust the precision of the compute operation to prevent an arithmetic overflow at the accumulator.

9. A data processing system comprising:

one or more processors including at least one graphics processor, the at least one graphics processor including;

a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and

a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation on the set of dynamic fixed-point tensors, the dynamic precision manager to prevent an arithmetic overflow during the compute operation.

10. The data processing system as in claim 9, the dynamic precision fixed-point logic of the compute unit including an integer compute unit.

11. The data processing system as in claim 10, wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.

12. The data processing system as in claim 11, wherein the register is to store a dynamic fixed-point scale factor.

13. The data processing system as in claim 12, wherein the compute unit is to perform a matrix arithmetic operation associated with a convolution operation on input data to a neural network.

14. The data processing system as in claim 9, wherein to perform a matrix arithmetic operation on the set of dynamic fixed-point tensors includes to:

receive an input tensor associated with the matrix arithmetic operation;

divide the input tensor into multiple blocks, the multiple blocks having different fixed-point precisions;

determine a shared exponent for each of the multiple blocks;

convert each of the multiple blocks into a dynamic fixed-point format using the shared exponent for each block;

store metadata for the multiple blocks to indicate a data format and shared exponent for the multiple blocks; and

perform the matrix arithmetic operation on the divided dynamic fixed-point input tensors.

15. An electronic device comprising:

one or more processors including at least one graphics processor, the at least one graphics processor including;

a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and

a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation on the set of dynamic fixed-point tensors, the dynamic precision manager to prevent an arithmetic overflow during the compute operation, wherein to perform a matrix arithmetic operation on the set of dynamic fixed-point tensors includes to;

receive an input tensor associated with the matrix arithmetic operation;

divide the input tensor into multiple blocks, the multiple blocks having different fixed-point precisions;

determine a shared exponent for each of the multiple blocks;

convert each of the multiple blocks into a dynamic fixed-point format using the shared exponent for each block;

store metadata for the multiple blocks to indicate a data format and shared exponent for the multiple blocks; and

perform the matrix arithmetic operation on the divided dynamic fixed-point input tensors.

16. The electronic device as in claim 15, wherein the dynamic precision fixed-point logic of the compute unit including an integer compute unit.

17. The electronic device as in claim 16, wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.

18. The electronic device as in claim 17, wherein the register is to store a dynamic fixed-point scale factor.

19. The electronic device as in claim 18, wherein the compute unit is to perform a matrix arithmetic operation associated with a convolution operation on input data to a neural network.

20. The electronic device as in claim 19, wherein the matrix arithmetic operation includes a multiply and accumulate operation.