SCALABLE NEURAL NETWORK SYSTEM
First Claim
Patent Images
1. A neural network system, including:
- a root processor;
one or more synchronizing sub-systems (SSSs), bidirectionally coupled to the root processor; and
a plurality of neural network processors (NNPs), wherein a respective one of the plurality of NNPs is bidirectionally coupled to one of the one or more SSSs.
1 Assignment
0 Petitions
Accused Products
Abstract
A scalable neural network system may include a root processor and a plurality of neural network processors with a tree of synchronizing sub-systems connecting them together. Each synchronization sub-system may connect one parent to a plurality of children. Furthermore, each of the synchronizing sub-systems may simultaneously distribute weight updates from the root processor to the plurality of neural network processors, while statistically combining corresponding weight gradients from its children into single statistical weight gradients. A generalized network of sensor-controllers may have a similar structure.
-
Citations
40 Claims
-
1. A neural network system, including:
-
a root processor; one or more synchronizing sub-systems (SSSs), bidirectionally coupled to the root processor; and a plurality of neural network processors (NNPs), wherein a respective one of the plurality of NNPs is bidirectionally coupled to one of the one or more SSSs. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A synchronizing sub-system (SSS) of a neural network system, the SSS configured to be coupled between a root processor and a plurality of neural network processors (NNPs), the SSS including:
-
a distributer configured to distribute information to one or more NNPs coupled to the at least one SSS; and a combiner configured to receive and combine information from the one or more NNPs coupled to the at least one SSS. - View Dependent Claims (13, 14, 15, 16, 17)
-
-
18. A method of operating a neural network, the method including:
-
coupling a root processor with a plurality of neural network processors (NNPs) through at least one intermediate processing sub-system; passing information bi-directionally between the root processor and the at least one intermediate processing sub-system; and passing information bi-directionally between the at least one intermediate processing sub-system and the plurality of NNPs. - View Dependent Claims (19, 20, 21, 22, 23, 24, 25, 26, 27, 37, 38)
-
-
28. A method of operating a synchronizing sub-system (SSS) of a neural network system, the SSS configured to be coupled between a root processor and a plurality of neural network processors (NNPs), the method including:
-
communicating information bi-directionally with the root processor; and communicating information bi-directionally with the plurality of NNPs. - View Dependent Claims (29, 30, 31, 32, 33, 34, 35, 36, 39, 40)
-
Specification