COMPUTER ARCHITECTURE FOR MULTIPLIER-LESS MACHINE LEARNING

• US 20200134429A1
• Filed: 09/26/2019
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Raytheon Company (Rtx Corporation)
• Inventors: Parker, Michael A.
• Granted Patent: US 11,593,619 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/0442 characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
• CPC Class Codes: G06N 3/0464 Convolutional networks [CNN, ConvNet]
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A computer architecture for multiplier-less machine learning is disclosed. According to some aspects, a neural network apparatus include processing circuitry and memory. The processing circuitry accesses a plurality of weights for a neural network layer and an input vector for the neural network layer, the input vector comprising a plurality of data values. The processing circuitry provides the plurality of weights and the input vector to an addition layer. The addition layer generates data value-weight pairs and, for each data value-weight pair, creates an input block comprising a sum of the data value and the weight. The processing circuitry sorts the input blocks generated by the addition layer. The processing circuitry cancels any opposite signed input blocks from the sorted input blocks to generate a set of blocks. The processing circuitry outputs a K^th largest value from the set of blocks. K is a positive integer.

Claims

1. A neural network apparatus, the apparatus comprising:

processing circuitry and memory;

the processing circuitry to;

access a plurality of weights for a neural network layer, each weight being associated with a weight sign;

access an input vector for the neural network layer, the input vector comprising a plurality of data values, each data value being associated with a data value sign;

provide the plurality of weights and the input vector to an addition layer, the addition layer generating data value-weight pairs and, for each data value-weight pair, creating an input block comprising a sum of the data value and the weight, and a xor (exclusive or) of the data value sign and the weight sign;

sort the input blocks generated by the addition layer;

cancel any opposite signed input blocks having a same magnitude from the sorted input blocks to generate a set of blocks; and

output a K^th largest value from the set of blocks, wherein K is a positive integer.

2. The apparatus of claim 1, wherein the K^th largest value is a part of an input to an additional addition layer.

3. The apparatus of claim 1, wherein the processing circuitry is to access the input vector by:

accessing a numeric input to a neural network; and

computing the input vector based on a logarithm of the numeric input.

4. The apparatus of claim 1, wherein K is determined based on input statistics of the data.

5. The apparatus of claim 4, wherein the input statistics include a distribution of the data.

6. The apparatus of claim 1, wherein the K^th largest value corresponds to an activation energy of the neural network layer.

7. The apparatus of claim 1, wherein the addition layer is implemented using an adder circuit in the processing circuitry.

8. The apparatus of claim 1, wherein the addition layer is implemented in software stored in the memory.

9. A non-transitory machine-readable medium for executing a neural network, the machine-readable medium storing instructions which, when executed by processing circuitry of one or more machines, cause the processing circuitry to:

access a plurality of weights for a neural network layer, each weight being associated with a weight sign;

access an input vector for the neural network layer, the input vector comprising a plurality of data values, each data value being associated with a data value sign;

provide the plurality of weights and the input vector to an addition layer, the addition layer generating data value-weight pairs and, for each data value-weight pair, creating an input block comprising a sum of the data value and the weight, and a xor (exclusive or) of the data value sign and the weight sign;

sort the input blocks generated by the addition layer;

cancel any opposite signed input blocks having a same magnitude from the sorted input blocks to generate a set of blocks; and

output a K^th largest value from the set of blocks, wherein K is a positive integer.

10. The machine-readable medium of claim 9, wherein the K^th largest value is a part of an input to an additional addition layer.

11. The machine-readable medium of claim 9, wherein the processing circuitry is to access the input vector by:

accessing a numeric input to a neural network; and

computing the input vector based on a logarithm of the numeric input.

12. The machine-readable medium of claim 9, wherein K is determined based on input statistics of the data.

13. The machine-readable medium of claim 12, wherein the input statistics include a distribution of the data.

14. The machine-readable medium of claim 9, wherein the K^th largest value corresponds to an activation energy of the neural network layer.

15. A neural network method implemented at processing circuitry of one or more machines, the method comprising:

access a plurality of weights for a neural network layer, each weight being associated with a weight sign;

access an input vector for the neural network layer, the input vector comprising a plurality of data values, each data value being associated with a data value sign;

provide the plurality of weights and the input vector to an addition layer, the addition layer generating data value-weight pairs and, for each data value-weight pair, creating an input block comprising a sum of the data value and the weight, and a xor (exclusive or) of the data value sign and the weight sign;

sort the input blocks generated by the addition layer;

cancel any opposite signed input blocks having a same magnitude from the sorted input blocks to generate a set of blocks; and

output a K^th largest value from the set of blocks, wherein K is a positive integer.

16. The method of claim 15, wherein the K^th largest value is a part of an input to an additional addition layer.

17. The method of claim 15, wherein the processing circuitry is to access the input vector by:

accessing a numeric input to a neural network; and

computing the input vector based on a logarithm of the numeric input.

18. The method of claim 15, wherein K is determined based on input statistics of the data.

19. The method of claim 18, wherein the input statistics include a distribution of the data.

20. The method of claim 15, wherein the K^th largest value corresponds to an activation energy of the neural network layer.