UPDATING AN ARTIFICIAL NEURAL NETWORK USING FLEXIBLE FIXED POINT REPRESENTATION

• US 20170061279A1
• Filed: 01/14/2015
• Published: 03/02/2017
• Est. Priority Date: 01/14/2015
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kloss, Carey, Arora, Prashant, Park, Alex S., Rao, Naveen G., Khosrowshahi, Amir, Yang, Andrew
• Application Number:
  US14/597,091
• Publication Number:
  US 20170061279A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

Updating an artificial neural network is disclosed. A node characteristic is represented using a fixed point node characteristic parameter. A network characteristic is represented using a fixed point network characteristic parameter. The fixed point node characteristic parameter and the fixed point network characteristic parameter are processed to determine a fixed point intermediate parameter having a larger size than either the fixed point node characteristic parameter or the fixed point network characteristic parameter. A value associated with the fixed point intermediate parameter is truncated according to a system truncation schema. The artificial neural network is updated according to the truncated value.

Claims

1. A processor for updating an artificial neural network, comprising:

a storage configured to represent a node characteristic using a fixed point node characteristic parameter and represent a network characteristic using a fixed point network characteristic parameter; and

a logic unit configured to;

operate on the fixed point node characteristic parameter and the fixed point network characteristic parameter to determine a fixed point intermediate parameter having a larger size than either the fixed point node characteristic parameter or the fixed point network characteristic parameter;

truncate a value associated with the fixed point intermediate parameter according to a system truncation schema; and

update the artificial neural network according to the truncated value.

2. The processor of claim 1, wherein the value is a result of a calculation operation performed based on one or more intermediate results of operating on the fixed point node characteristic parameter and the fixed point network characteristic parameter.

3. The processor of claim 2, wherein the result of the calculation operation and the intermediate results are fixed point numbers that are larger in size than either the fixed point node characteristic parameter or the fixed point network characteristic parameter.

4. The processor of claim 1, wherein the value is a result of performing a grouping of sub calculation operations of an instruction to operate on the fixed point node characteristic parameter and the fixed point network characteristic parameter.

5. The processor of claim 1, wherein the value is a result of performing a calculation operation using the fixed point intermediate parameter.

6. The processor of claim 1, wherein the value is the fixed point intermediate parameter.

7. The processor of claim 1, wherein the fixed point node characteristic parameter and the fixed point network characteristic parameter are represented using different fixed point representation formats.

8. The processor of claim 1, wherein the system truncation schema specifies a desired fixed point representation format of a result of the truncated value.

9. The processor of claim 1, wherein the operation on the fixed point node characteristic parameter and the fixed point network characteristic parameter to determine the fixed point intermediate parameter was initiated in response to a received processor instruction.

10. The processor of claim 9, wherein the processor instruction specifies the system truncation schema.

11. The processor of claim 9, wherein the processor instruction specifies a first fixed point representation format of the fixed point node characteristic parameter and a second fixed point representation format of the fixed point network characteristic parameter.

12. The processor of claim 9, wherein the system truncation schema was empirically determined at least in part based on a previous execution iteration of the received processor instruction.

13. The processor of claim 1, wherein the system truncation schema was determined at least in part based on a result of a previous update of the artificial neural network performed using floating point representations of the node characteristic and the network characteristic.

14. The processor of claim 1, wherein the system truncation schema was determined at least in part based on a result that a previous update of the artificial neural network resulted in overflow of a previously desired fixed point representation.

15. The processor of claim 1, wherein the system truncation schema was determined at least in part based on a result that a previous update of the artificial neural network resulted in an underflow of a previously desired fixed point representation.

16. The processor of claim 1, wherein the system truncation schema was determined at least in part based on a performance metric of the artificial neural network.

17. The processor of claim 16, wherein the performance metric includes a result of a performance test performed on the artificial neural network.

18. The processor of claim 1, wherein the system truncation schema is dynamically modified to specify a fixed number of bits that are to represent digits of the value associated with the fixed point intermediate parameter after a radix point.

19. The processor of claim 1, wherein operating on the fixed point node characteristic parameter and the fixed point network characteristic parameter to determine the fixed point intermediate parameter includes processing a processor instruction to perform matrix multiplication.

20. The processor of claim 19, wherein the value is an accumulation value of results of sub multiplication operations of the processor instruction to perform matrix multiplication.

21. The processor of claim 1, wherein the node characteristic includes an activation node of the artificial neural network.

22. The processor of claim 1, wherein the network characteristic includes a node connection weight of the artificial neural network.

23. The processor of claim 1, wherein updating the artificial neural network includes updating the node characteristic or the network characteristic using the truncated value.

24. The processor of claim 1, wherein a total number of bits of the truncated value is not equal to a total number of bits of the fixed point node characteristic parameter or the fixed point network characteristic parameter.

25. A method of updating an artificial neural network, comprising:

representing a node characteristic using a fixed point node characteristic parameter;

representing a network characteristic using a fixed point network characteristic parameter;

using a processor to operate on the fixed point node characteristic parameter and the fixed point network characteristic parameter to determine a fixed point intermediate parameter having a larger size than either the fixed point node characteristic parameter or the fixed point network characteristic parameter;

truncating a value associated with the fixed point intermediate parameter according to a system truncation schema; and

updating the artificial neural network according to the truncated value.

26. A computer program product for updating an artificial neural network, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

representing a node characteristic using a fixed point node characteristic parameter;

representing a network characteristic using a fixed point network characteristic parameter;

operating on the fixed point node characteristic parameter and the fixed point network characteristic parameter to determine a fixed point intermediate parameter having a larger size than either the fixed point node characteristic parameter or the fixed point network characteristic parameter;

truncating a value associated with the fixed point intermediate parameter according to a system truncation schema; and

updating the artificial neural network according to the truncated value.