SCALABLE NEURAL NETWORK PROCESSING ENGINE

• US 20190340491A1
• Filed: 05/04/2018
• Published: 11/07/2019
• Est. Priority Date: 05/04/2018
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Norden, Erik K., Fishel, Liran, Park, Sung Hee, Shin, Jaewon, Mills, Christopher L., Lee, Seungjin, Mujica, Fernando A.
• Granted Patent: US 11,537,838 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.

Claims

1. A neural processor circuit, comprising:

a plurality of neural engine circuits that are selectively activated, each of the neural engine circuits configured to perform convolution operations on input data and kernel coefficients to generate output data;

a data buffer between the plurality of neural engine circuits and a memory external to the neural processor circuit, the data buffer configured to store the input data from the memory for sending to the neural engine circuits and the output data received from the neural engine circuits; and

a kernel extract circuit configured to receive kernel data from the memory external to the neural processor circuit, the kernel extract circuit configured to send a corresponding kernel coefficient extracted from the kernel data to neural engine circuits selected for activation.

2. The neural processor circuit of claim 1, further comprising a neural task manager circuit configured to provide configuration data to the neural engine circuits to activate or deactivate one or more of the neural engine circuits for a task.

3. The neural processor circuit of claim 2, wherein the data buffer is further configured to allocate memory space for storing the input data and the output data responsive to receiving the configuration data from the neural task manager circuit.

4. The neural processor circuit of claim 3, wherein the neural processor circuit includes a buffer direct memory access (DMA) coupled to the memory and the data buffer, the buffer DMA configured to retrieve the input data from the memory and provide the input data to the memory space of the data buffer according to the configuration data.

5. The neural processor circuit of claim 2, wherein the kernel extract circuit is further configured to allocate memory space for storing the kernel data responsive to receiving the configuration data from the neural task manager.

6. The neural processor circuit of claim 2, further comprising a processor coupled to the system memory, the processor configured to generate the configuration data defining tasks that instantiate one or more neural networks.

7. The neural processor circuit of claim 2, wherein each of the plurality of neural engines circuits operates in a power saving mode when deactivated.

8. The neural processor circuit of claim 1, further comprising a neural task manager circuit configured to provide configuration data to the neural processor circuit to activate or deactivate the neural processor circuit for a task

9. The neural processor circuit of claim 8, wherein the neural processor circuit operates in a power saving mode when deactivated.

10. A method for processing input data, comprising:

selectively activating a plurality of neural engine circuits of a neural processor circuit, each of the neural engine circuits configured to perform convolution operations on input data and kernel coefficients to generate output data;

storing the input data in a data buffer of the neural engine circuit for sending to the neural engine circuits and the output data received from the neural engine circuits; and

sending a corresponding kernel coefficient extracted from kernel data to neural engine circuits selected for activation.

11. The method of claim 10, further comprising providing configuration data to the neural engine circuits to activate or deactivate one or more of the neural engine circuits for a task.

12. The method of claim 11, further comprising allocating memory space of the data buffer for storing the input data and the output data responsive to the configuration data.

13. The method of claim 12, further comprising retrieving the input data from a memory and providing the input data to the memory space of the data buffer according to the configuration data.

14. The method of claim of claim 11, further comprising:

allocating memory space of a kernel extract circuit for storing the kernel data responsive to the configuration data from the neural task manager; and

storing the kernel data in the kernel extract circuit.

15. The method of claim 11, further comprising generating the configuration data defining tasks that instantiate one or more neural networks.

16. The method of claim 11, wherein each of the plurality of neural engines circuits operates in a power saving mode when deactivated.

17. The method of claim 10, further comprising providing configuration data to the neural processor circuit activate or deactivate the neural processor circuit for a task.

18. The method of claim 17, wherein the neural processor circuit operates in a power saving mode when deactivated.

19. An integrated circuit (IC) system comprising a neural processor circuit, the neural processor circuit comprising:

a plurality of neural engine circuits that are selectively activated, each of the neural engine circuits configured to perform convolution operations on input data and kernel coefficients to generate output data;

a data buffer between the plurality of neural engine circuits and a memory external to the neural processor circuit, the data buffer configured to store the input data from the memory for sending to the neural engine circuits and the output data received from the neural engine circuits; and

a kernel extract circuit configured to receive kernel data from the memory external to the neural processor circuit, the kernel extract circuit configured to send a corresponding kernel coefficient extracted from the kernel data to neural engine circuits selected for activation.

20. The IC system of claim 19, wherein the neural processor circuit further includes a neural task manager circuit configured to provide configuration data to the neural engine circuits to activate or deactivate one or more of the neural engine circuits for a task.