DYNAMICALLY CONFIGURABLE, MULTI-PORTED CO-PROCESSOR FOR CONVOLUTIONAL NEURAL NETWORKS

• US 20110029471A1
• Filed: 02/01/2010
• Published: 02/03/2011
• Est. Priority Date: 07/30/2009
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Cadambi, Srihari, CHAKRADHAR, SRIMAT, Jakkula, Venkata S., Sankaradas, Murugan
• Granted Patent: US 8,442,927 B2
• Time in Patent Office: Days
• US Class Current: 706/25
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A coprocessor and method for processing convolutional neural networks includes a configurable input switch coupled to an input. A plurality of convolver elements are enabled in accordance with the input switch. An output switch is configured to receive outputs from the set of convolver elements to provide data to output branches. A controller is configured to provide control signals to the input switch and the output switch such that the set of convolver elements are rendered active and a number of output branches are selected for a given cycle in accordance with the control signals.

Claims

1. A coprocessor for processing convolutional neural networks, comprising

a configurable input switch coupled to an input;

a plurality of convolver elements enabled in accordance with the input switch;

an output switch configured to receive outputs from the set of convolver elements to provide data to output branches; and

a controller configured to provide control signals to the input switch and the output switch such that the set of convolver elements are rendered active and a number of output branches are selected for a given cycle in accordance with the control signals.

2. The coprocessor as recited in claim 1, wherein the convolver elements are bundled into computational elements each including Y convolvers.

3. The coprocessor as recited in claim 2, wherein Y is determined by a number of inputs from memory external to the coprocessor.

4. The coprocessor as recited in claim 2, wherein the number of output branches is X and X and Y are determined based upon a type of parallelism needed for a given neural network layer.

5. The coprocessor as recited in claim 4, wherein the type of parallelism includes one of inter-output and intra-output.

6. The coprocessor as recited in claim 1, further comprising a memory subsystem having at least two banks, wherein a first bank provides input storage and a second bank provides output storage to enable a stateless, streaming coprocessor architecture.

7. The coprocessor as recited in claim 6, further comprising a third memory bank for storing intermediate results.

8. A system for processing convolutional neural networks, comprising:

a host system;

a coprocessor coupled to the host system, the coprocessor including;

a configurable input switch coupled to an input;

a plurality of convolver elements enabled in accordance with the input switch;

an output switch configured to receive outputs from the set of convolver elements to provide data to output branches; and

a controller configured to provide control signals to the input switch and the output switch such that the set of convolver elements are rendered active and a number of output branches are selected for a given cycle in accordance with the control signals; and

a memory subsystem configured to stored inputs and outputs of the coprocessor

9. The system as recited in claim 8, wherein the convolver elements are bundled into computational elements each including Y convolvers.

10. The system as recited in claim 9, wherein Y is determined by a number of inputs from memory external to the coprocessor.

11. The system as recited in claim 9, wherein the number of output branches is X, and X and Y are determined based upon a type of parallelism needed for a given neural network layer.

12. The system as recited in claim 11, wherein the type of parallelism includes one of inter-output and intra-output.

13. The system as recited in claim 8, wherein the memory subsystem includes at least three banks, wherein a first bank provides input storage;

a second bank provides output storage; and

a third bank provides storage for intermediate results.

14. A method for processing convolutional neural networks (CNN), comprising:

determining a workload for a convolutional neural networks;

generating a control signal based upon the workload to permit a selection of a type of parallelism to be employed in processing a layer of the CNN;

configuring an input switch to enable a number of convolvers which convolve an input in accordance with the control signal;

configuring an output switch to enable a number of output branches for a given cycle in accordance with the control signal; and

processing outputs from the output branches.

15. The method as recited in claim 14, further comprising selecting the number of convolvers and the number of output branches based upon a type of parallelism needed for a given neural network layer.

16. The method as recited in claim 15, wherein the type of parallelism includes one of parallelism within a convolution operation, inter-output parallelism and intra-output parallelism.

17. The method as recited in claim 14, further comprising providing a memory subsystem having at least two banks, wherein a first bank provides input storage and a second bank provides output storage to enable a stateless, streaming coprocessor architecture.

18. The method as recited in claim 17, further comprising providing a third memory bank for storing intermediate results.

19. The method as recited in claim 14, wherein processing includes reconfiguring the input switch and the output switch in accordance with a next layer of the CNN to be processed.