High efficiency learning network
First Claim
1. A computational network that produces a plurality of network outputs in response to a plurality of network inputs, comprising:
- a plurality of first layer computational nodes forming a first layer of a computational network, each of said first layer computational nodes receiving at least one input resulting from at least one of a plurality of computational network inputs to produce a first layer output, said first layer output being discretized to a nonzero integer power of 2; and
;
a plurality of second layer computational nodes forming a second layer of the computational network, each of said plurality of second layer computational nodes receiving a plurality of inputs to produce a second layer output, each of said plurality of inputs being a product of a weight value and said first layer output produced by one of said plurality of first layer computational nodes, said second layer output being used to produce at least one of a plurality of network outputs.
3 Assignments
0 Petitions
Accused Products
Abstract
Nodal outputs are discretized to values of S2n where n is an integer and S is equal to +1 or -1. During forward propagation, this offers the advantage of forming a product of a nodal output and a weight using a simple shift operation rather than a multiply operation. Replacing multiply operations with shift operations through out a neural network improves response times and permits building larger networks that have broader applicability. Training is also improved by increasing the efficiency of backward propagation. The multiplications involved in backward propagation are reduced To shift operations by discretizing the errors associated with each node so that they are represented as 2n where n is an integer and S is equal to +1 or -1.
-
Citations
31 Claims
-
1. A computational network that produces a plurality of network outputs in response to a plurality of network inputs, comprising:
-
a plurality of first layer computational nodes forming a first layer of a computational network, each of said first layer computational nodes receiving at least one input resulting from at least one of a plurality of computational network inputs to produce a first layer output, said first layer output being discretized to a nonzero integer power of 2; and
;a plurality of second layer computational nodes forming a second layer of the computational network, each of said plurality of second layer computational nodes receiving a plurality of inputs to produce a second layer output, each of said plurality of inputs being a product of a weight value and said first layer output produced by one of said plurality of first layer computational nodes, said second layer output being used to produce at least one of a plurality of network outputs. - View Dependent Claims (2, 3, 4, 5)
-
-
6. A computational network that produces a plurality of network outputs in response to a plurality of network inputs, comprising:
-
a plurality of first layer computational nodes forming a first layer of a computational network, each of said first layer computational nodes receiving at least one first layer input resulting from at least one of a plurality of computational network inputs to produce a first layer output, said first layer output being discretized to a nonzero integer power of 2; and
;a plurality of second layer computational nodes forming a second layer of the computational network, each of said plurality of second layer computational nodes receiving a plurality of second layer inputs to produce a second layer output, each of said plurality of second layer inputs being a product of a weight value and said first layer output produced by one of said plurality of first layer computational nodes, said second layer output being produced by using a sum of said plurality of inputs as an operand of a first activation function, said second layer output being used to produce at least one of a plurality of network outputs. - View Dependent Claims (7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. A method of recognizing a user'"'"'s input, comprising the steps of:
-
forming a first plurality of values representative of the user'"'"'s input, said first plurality of values being discretized to a nonzero integer power of 2; producing a first plurality of products using said first plurality of values and a first plurality of weights; producing a second plurality of products using said first plurality of values and a second plurality of weights; using a first computational node belonging to a plurality of first layer computational nodes to produce a first output by forming a sum of said first plurality of products and by using said sum of said first plurality of products as an operand of a first activation function; using a second computational node belonging to said plurality of first layer computational nodes to produce a second output by forming a sum of said second plurality of products and by using said sum of said second plurality of products as an operand of a second activation function; forming a first product using said first output and a first weight value; forming a second product using said second output and a second weight value; using a second layer computational node belonging to a plurality of second layer computational nodes to produce a second layer output by forming a sum of said first and second products and by using said sum of said first and second products as an operand of a third activation function; and using said second layer output to produce at least one of a plurality of network outputs that identify the user'"'"'s input. - View Dependent Claims (20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31)
-
Specification