NEURAL NETWORK CONVOLUTION COMPUTATION METHOD AND DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM

• US 20190311242A1
• Filed: 06/13/2019
• Published: 10/10/2019
• Est. Priority Date: 12/14/2016
• Status: Active Grant

• Current Assignee: Shanghai Cambricon Information Technology Co., Ltd.
• Original Assignee: Shanghai Cambricon Information Technology Co., Ltd.
• Inventors: CHEN, Tianshi, ZHUANG, Yimin, GUO, Qi, LIU, Shaoli, CHEN, Yunji
• Granted Patent: US 10,635,965 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

Aspects of a neural network convolution device are described herein. The aspects may include a matrix transformer and a matrix multiplication module. The matrix transformer may be configured to receive an input data matrix and a weight matrix, transform the input data matrix into a transformed input data matrix based on a first transformation matrix, and transform the weight matrix into a transformed weight matrix based on a second transformation matrix. The matrix multiplication module may be configured to multiply one or more input data elements in the transformed input data matrix with one or more weight elements in the transformed weight matrix to generate an intermediate output matrix. The matrix transformer may be further configured to transform the intermediate output matrix into an output matrix based on an inverse transformation matrix.

Claims

1. A neural network convolution device, comprising:

a matrix transformer configured to

receive an input data matrix and a weight matrix,

transform the input data matrix into a transformed input data matrix based on a first transformation matrix, and

transform the weight matrix into a transformed weight matrix based on a second transformation matrix; and

a matrix multiplication module configured to multiply one or more input data elements in the transformed input data matrix with one or more weight elements in the transformed weight matrix to generate an intermediate output matrix, wherein the matrix transformer is further configured to transform the intermediate output matrix into an output matrix based on an inverse transformation matrix.

2. The neural network convolution device of claim 1, wherein the first transformation matrix, the second transformation matrix, and the inverse transformation matrix are predetermined based on a first size of the input data matrix, a second size of the weight matrix, and a stride for moving a convolution kernel.

3. The neural network convolution device of claim 1, further comprising a sparse processing unit configured to generate a sparse sequence based on the weight elements in the transformed weight matrix.

4. The neural network convolution device of claim 3, further comprising a mapping unit configured to generate a correspondence table that maps the sparse sequence to the transformed input data matrix.

5. The neural network convolution device of claim 4, wherein the matrix multiplication module is further configured to select the one or more input data elements based on the correspondence table prior to the multiplication.

6. The neural network convolution device of claim 3, wherein the sparse processing unit is further configured to set elements in the sparse sequence that correspond to zero values in the transformed weight matrix to zero.

7. The neural network convolution device of claim 1, wherein the matrix transformer is further configured to sequentially multiply a transpose matrix of the first transformation matrix, the input data matrix, and the first transformation matrix to generate the transformed input data matrix.

8. The neural network convolution device of claim 1, wherein the matrix transformer is further configured to sequentially multiply the second transformation matrix, the weight matrix, and a transpose matrix of the second transformation matrix to generate the transformed weight matrix.

9. The neural network convolution device of claim 1, wherein the matrix transformer includes a shift module and an adder configured to collaboratively transform the input data matrix and the weight matrix.

10. The neural network convolution device of claim 1, further comprising a controller configured to:

receive an instruction to convolute the input data matrix with the weight matrix, and

decode the instruction into one or more micro-instructions that include addresses for retrieving the input data matrix and the weight matrix.

17. The method of claim 1, further comprising sequentially multiplying, by the matrix transformer, a transpose matrix of the first transformation matrix, the input data matrix, and the first transformation matrix to generate the transformed input data matrix.

18. The method of claim 1, further comprising sequentially multiplying, by the matrix transformer, the second transformation matrix, the weight matrix, and a transpose matrix of the second transformation matrix to generate the transformed weight matrix.

19. The method of claim 1, further comprising collaboratively transforming, by a shift module and an adder in the matrix transformer, the input data matrix and the weight matrix.

20. The method of claim 1, further comprising:

receiving, by a controller, an instruction to convolute the input data matrix with the weight matrix; and

decoding, by the controller, the instruction into one or more micro-instructions that include addresses for retrieving the input data matrix and the weight matrix.

11. A method for convolution in a neural network, comprising:

receiving, by a matrix transformer, an input data matrix and a weight matrix;

transforming, by the matrix transformer, the input data matrix into a transformed input data matrix based on a first transformation matrix;

transforming, by the matrix transformer, the weight matrix into a transformed weight matrix based on a second transformation matrix;

multiplying, by a matrix multiplication module, one or more input data elements in the transformed input data matrix with one or more weight elements in the transformed weight matrix to generate an intermediate output matrix; and

transforming, by the matrix transformer, the intermediate output matrix into an output matrix based on an inverse transformation matrix.

12. The method of claim 11, wherein the first transformation matrix, the second transformation matrix, and the inverse transformation matrix are predetermined based on a first size of the input data matrix, a second size of the weight matrix, and a stride for moving a convolution kernel.

13. The method of claim 11, further comprising generating, by a sparse processing unit, a sparse sequence based on the weight elements in the transformed weight matrix.

14. The method of claim 13, further comprising generating, by a mapping unit, a correspondence table that maps the sparse sequence to the transformed input data matrix.

15. The method of claim 14, further comprising selecting, by the matrix multiplication module, the one or more input data elements based on the correspondence table prior to the multiplication.

16. The method of claim 13, further comprising setting, by the sparse processing unit, elements in the sparse sequence that correspond to zero values in the transformed weight matrix to zero.