Approximating fully-connected layers with multiple arrays of 3x3 convolutional filter kernels in a CNN based integrated circuit

• US 10,366,328 B2
• Filed: 03/14/2018
• Issued: 07/30/2019
• Est. Priority Date: 09/19/2017
• Status: Active Grant

• Current Assignee: Gyrfalcon Technology Inc.
• Original Assignee: Gyrfalcon Technology Inc.
• Inventors: Yang, Lin, Dong, Patrick Z., Dong, Jason Z., Sun, Baohua
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/920,842
• Publication Number:
  US 20190087725A1
• Time in Patent Office: 503 Days
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Multiple 3×3 convolutional filter kernels are used for approximating operations of fully-connected (FC) layers. Image classification task is entirely performed within a CNN based integrated circuit. Output at the end of ordered convolutional layers contains P feature maps with F×F pixels of data per feature map. 3×3 filter kernels comprises L layers with each organized in an array of R×Q of 3×3 filter kernels, Q and R are respective numbers of input and output feature maps of a particular layer of the L layers. Each input feature map of the particular layer comprises F×F pixels of data with one-pixel padding added around its perimeter. Each output feature map of the particular layer comprises (F−2)×(F−2) pixels of useful data. Output of the last layer of the L layers contains Z classes. L equals to (F−1)/2 if F is an odd number. P, F, Q, R and Z are positive integers.

Claims

1. A digital integrated circuit comprising:

a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising;

a CNN processing block configured for simultaneously performing convolutional operations using input data and pre-trained filter coefficients of a plurality of ordered convolutional layers, and further configured for classifying the input data using a plurality of 3×

3 filter kernels to approximate operations of fully-connected (FC) layers, wherein output of the plurality of ordered convolutional layers has P feature maps with F×

F pixels of data per feature map and the plurality of 3×

3 filter kernels comprises L layers with each of the L layers organized in an array of R×

Q of 3×

3 filter kernels, wherein Q and R are respective numbers of input and output feature maps of a particular layer of the L layers, wherein L is equal to (F−

1)/2 when F is an odd number, and wherein P, F, Q and R are positive integers;

a first set of memory buffers operatively coupling to the CNN processing block for storing the input data; and

a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients.

2. The digital integrated circuit of claim 1, wherein the array of R×

Q of 3×

3 filter kernels obtains results of convolutional operations of the particular layer.

3. The digital integrated circuit of claim 2, when each input feature map of the particular layer comprises F×

F pixels of data with one-pixel padding added around perimeter, each output feature map of the particular layer comprises (F−

2)×

(F−

2) pixels of useful data located in center portion of said each output feature map.

4. The digital integrated circuit of claim 3, wherein output of the last layer of the L layers contains Z classes for image classification, each of the Z classes contains 1×

1 pixel of useful data stored in center pixel of the F×

F pixels, where Z is a positive integer.

5. The digital integrated circuit of claim 3, wherein output of the last layer of the L layers contains S nodal feature values, each of the S nodal feature values contains 1×

1 pixel of useful data stored in center pixel of the F×

F pixels, where S is a positive integer.

6. The digital integrated circuit of claim 5, wherein the plurality of 3×

3 filter kernels further includes at least one inner-product replacement array of U×

T of 3×

3 filter kernels used for replacing inner-product operations in one of the FC layers that contains T number of input nodal feature values and U number of output nodal feature values, where T and U are positive integers.

7. The digital integrated circuit of claim 6, wherein FC layer weights of the innerproduct operations are respectively stored in center pixel of each 3×

3 filter kernel in the at least one inner-product replacement array of U×

T of 3×

3 filter kernels.

8. The digital integrated circuit of claim 7, wherein numerical zeros are stored in eight perimeter pixels of said each 3×

3 filter kernel in the at least one inner-product replacement array of U×

T of 3×

3 filter kernels.

9. The digital integrated circuit of claim 2, when each input feature map of the particular layer comprises F×

F pixels of data without padding, each output feature map of the particular layer comprises (F−

2)×

(F−

2) pixels.

10. The digital integrated circuit of claim 1, wherein the CNN processing block is further configured for performing operations of activation and pooling.