NEURAL NETWORK PROCESSOR WITH A WINDOW EXPANDER CIRCUIT

• US 20190325297A1
• Filed: 04/21/2018
• Published: 10/24/2019
• Est. Priority Date: 04/21/2018
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Fowers, Jeremy, Zhang, Dan, Ghandi, Mohammadmahdi
• Granted Patent: US 11,151,445 B2
• Time in Patent Office: Days
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Neural network processors including a window expander circuit and related methods are provided. The window expander circuit may include a first logic circuit configured to store a set of data elements, corresponding to at least a subset of the input data, into a Q number of logical memories, where each of a P number of data elements of the set of the data elements is stored in each of the Q number of logical memories. The window expander circuit may further include a second logic circuit configured to receive the first set of data elements and additional data elements corresponding to the at least the subset of the input data from the Q number of logical memories and expand the at least the subset of the input data until the at least the subset of the input data is expanded based on a predetermined factor.

Claims

1. A neural network processor configured to perform convolution operations on input data and N by N matrices, wherein N is a positive integer greater than one, the neural network processor comprising:

a plurality of multiplier circuits;

a window expander circuit comprising;

a first logic circuit configured to store a set of data elements, corresponding to at least a subset of the input data, into a Q number of logical memories, wherein each of a P number of data elements of the set of the data elements is stored in each of the Q number of logical memories, wherein P is an integer equal to or greater than one and Q is an integer equal to or greater than N, and

a second logic circuit configured to receive the first set of data elements and additional data elements corresponding to the subset of the input data from the Q number of logical memories and expand the at least the subset of the input data until the at least the subset of the input data is expanded based on a predetermined factor selected at least to increase utilization of the plurality of the multiplier circuits.

2. The neural network processor of claim 1, wherein the first logic circuit comprises a finite state machine configured to store the data elements corresponding to the at least the subset of the input data into the each of the Q logical memories.

3. The neural network processor of claim 2, wherein the each of the Q logical memories comprises a random-access memory.

4. The neural network processor of claim 1, wherein the second logic circuit comprises a rotate circuit and an array structure.

5. The neural network processor of claim 1 further comprising a vector register file configured to store expanded data.

6. The neural network processor of claim 1 further configured to receive the input data via a PCI express bus.

7. The neural network processor of claim 1 further configured to receive the input data from a vector data memory, wherein the vector data memory is configured to receive the input data via a PCI express bus.

8. A method in a neural network processor configured to perform convolution operations on input data and N by N matrices, wherein N is a positive integer greater than one, wherein the neural network comprises a plurality of multiply circuits, the method comprising:

automatically determining whether the input data received by the neural network processor requires expansion; and

when the input data requires the expansion;

(1) storing a first set of data elements, corresponding to at least a subset of the input data, into a Q number of logical memories, wherein each of a P number of data elements of the first set of the data elements is stored in each of the Q number of logical memories, wherein P is an integer equal to or greater than one and Q is an integer equal to or greater than N, (2) shifting the first set of data elements from the Q number of logical memories into a first column of an array structure and storing a second set of data elements, corresponding to the subset of the input data, in the Q number of logical memories, (3) shifting the first set of the data elements from the first column of the array structure into a second column of the array structure and shifting the second set of data elements from the Q number of logical memories into the first column of the array structure, and (4) repeating storing and shifting steps using additional data elements corresponding to the subset of the input data until the subset of the input data is expanded based on a predetermined factor selected at least to increase utilization of the plurality of the multiplier circuits.

9. The method of claim 8, wherein the storing and the shifting steps are performed using a window expander circuit comprising a first logic circuit, wherein the first logic circuit comprises a finite state machine configured to store the data elements corresponding to the at least the subset of the input data into the each of the Q logical memories.

10. The method of claim 9, wherein the each of the Q logical memories comprises a random-access memory.

11. The method of claim 9, wherein the window expander circuit comprises a rotate circuit coupled between the Q logical memories and the array structure.

12. The method of claim 8 further comprising storing expanded data into a vector register file corresponding to the neural network processor.

13. The method of claim 8 further comprising receiving the input data via a PCI express bus.

14. The method of claim 8 further comprising receiving the input data from a vector data memory, wherein the vector data memory is configured to receive the input data via a PCI express bus.

15. A neural network processor configured to perform convolution operations on input data and N by N matrices, wherein N is a positive integer greater than one, the neural network processor comprising:

a plurality of multiplier circuits;

a window expander circuit comprising;

a first logic circuit configured to store a set of data elements, corresponding to at least a subset of the input data, into a Q number of logical memories, wherein each of a P number of data elements of the set of the data elements is stored in each of the Q number of logical memories, wherein P is an integer equal to or greater than one and Q is an integer equal to or greater than N, and

a second logic circuit configured to receive the first set of data elements from the Q number of logical memories and expand the at least the subset of the input data until the at least the subset of the input data is expanded based on a predetermined factor, wherein the second logic circuit comprises a rotate circuit and an array structure.

16. The neural network processor of claim 15, wherein the first logic circuit comprises a finite state machine configured to store the data elements corresponding to the subset of the input data into the each of the Q logical memories.

17. The neural network processor of claim 16, wherein the each of the Q logical memories comprises a random-access memory.

18. The neural network processor of claim 15, wherein the rotate circuit is configured to selectively rotate the at least the subset of the input data before providing the at least the subset of the input data to the array structure.

19. The neural network processor of claim 18, wherein an extent of a rotation of the at least the subset of the input data is determined based on a stride associated with the convolution operations.

20. The neural network processor of claim 15, wherein the window expander circuit is configured to receive the input data via a PCI express bus.