Implementation of ResNet in a CNN based digital integrated circuit

• US 10,339,445 B2
• Filed: 02/14/2018
• Issued: 07/02/2019
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: Gyrfalcon Technology Inc.
• Original Assignee: Gyrfalcon Technology Inc.
• Inventors: Yang, Lin, Dong, Patrick Z., Young, Charles Jin, Sun, Baohua
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/897,143
• Publication Number:
  US 20180174031A1
• Time in Patent Office: 503 Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

Operations of a combination of first and second original convolutional layers followed by a short path are replaced by operations of a set of three particular convolutional layers. The first contains 2N×N filter kernels formed by placing said N×N filter kernels of the first original convolutional layer in left side and N×N filter kernels of an identity-value convolutional layer in right side. The second contains 2N×2N filter kernels formed by placing the N×N filter kernels of the second original convolutional layer in upper left corner, N×N filter kernels of an identity-value convolutional layer in lower right corner, and N×N filter kernels of two zero-value convolutional layers in either off-diagonal corner. The third contains N×2N of kernels formed by placing N×N filter kernels of a first identity-value convolutional layer and N×N filter kernels of a second identity-value convolutional layer in a vertical stack. Each filter kernel contains 3×3 filter coefficients.

Claims

1. A digital integrated circuit for feature extraction comprising:

a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising;

a CNN processing block configured for simultaneously obtaining convolution operations results using input data and pre-trained filter coefficients of a plurality of convolutional layers including at least one set of three particular convolutional layers for performing equivalent operations of a combination of first and second original convolutional layers followed by a short path, the equivalent operations containing convolutional operations of the first and the second original convolutional layers followed by element-wise add operations with an input that contains N feature maps and an output also contains N feature maps, each of the first and the second original convolutional layers contains N×

N of 3×

3 filter kernels, where N is a positive integer;

a first set of memory buffers operatively coupling to the CNN processing block for storing the input data; and

a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients;

wherein first of the three particular convolutional layers contains 2N×

N of 3×

3 filter kernels formed by placing said N×

N of 3×

3 filter kernels of the first original convolutional layer in left side and N×

N of 3×

3 filter kernels of an identity-value convolutional layer in right side.

2. The digital integrated circuit of claim 1, wherein each of the 3×

3 filter kernels of the identity-value convolutional layer contains numerical value zero in each of nine positions of each off-diagonal kernel, and contains numerical value zero in each of eight perimeter positions and numerical value one in center position of each diagonal kernel.

11. The digital integrated circuit of claim 1, wherein the CNN processing block is further configured for performing activation and pooling operations.

12. The digital integrated circuit of claim 1, wherein the short path is used in a residual network (ResNet).

13. The digital integrated circuit of claim 1, wherein the short path is used in SelectiveNet.

3. A digital integrated circuit for feature extraction comprising:

a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising;

a CNN processing block configured for simultaneously obtaining convolution operations results using input data and pre-trained filter coefficients of a plurality of convolutional layers including at least one set of three particular convolutional layers for performing equivalent operations of a combination of first and second original convolutional layers followed by a short path, the equivalent operations containing convolutional operations of the first and the second original convolutional layers followed by element-wise add operations with an input that contains N feature maps and an output also contains N feature maps, each of the first and the second original convolutional layers contains N×

N of 3×

3 filter kernels, where N is a positive integer;

a first set of memory buffers operatively coupling to the CNN processing block for storing the input data; and

a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients;

wherein second of the three particular convolutional layers contains 2N×

2N of 3×

3 filter kernels formed by placing said N×

N of 3×

3 filter kernels of the second original convolutional layer in upper left corner, N×

N of 3×

3 filter kernels of an identity-value convolutional layer in lower right corner, and N×

N of 3×

3 filter kernels of two zero-value convolutional layers in either off diagonal corner.

4. The digital integrated circuit of claim 3, wherein each of the 3×

3 filter kernels of the identity-value convolutional layer contains numerical value zero in each of nine positions of each off-diagonal kernel, and contains numerical value zero in each of eight perimeter positions and numerical value one in center position of each diagonal kernel.

5. The digital integrated circuit of claim 3, wherein each of the 3×

3 filter kernels of the zero-value convolutional layer contains numerical value zero in all nine positions of all kernels.

8. The digital integrated circuit of claim 3, wherein the short path is used in SelectiveNet.

9. The digital integrated circuit of claim 8, wherein the short path is used in a residual network (ResNet).

10. The digital integrated circuit of claim 8, wherein the short path is used in SelectiveNet.

14. The digital integrated circuit of claim 3, wherein the CNN processing block is further configured for performing activation and pooling operations.

15. The digital integrated circuit of claim 3, wherein the short path is used in a residual network (ResNet).

16. The digital integrated circuit of claim 8, wherein the CNN processing block is further configured for performing activation and pooling operations.

6. A digital integrated circuit for feature extraction comprising:

a plurality of cellular neural networks (CNN) processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising;

a CNN processing block configured for simultaneously obtaining convolution operations results using input data and pre-trained filter coefficients of a plurality of convolutional layers including at least one set of three particular convolutional layers for performing equivalent operations of a combination of first and second original convolutional layers followed by a short path, the equivalent operations containing convolutional operations of the first and the second original convolutional layers followed by element-wise add operations with an input that contains N feature maps and an output also contains N feature maps, each of the first and the second original convolutional layers contains N×

N of 3×

3 filter kernels, where N is a positive integer;

a first set of memory buffers operatively coupling to the CNN processing block for storing the input data; and

a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients;

wherein third of the three particular convolutional layers contains N×

2N of 3×

3 filter kernels formed by placing N×

N of 3×

3 filter kernels of a first identity value convolutional layer and N×

N of 3×

3 filter kernels of a second identity value convolutional layer in a vertical stack.

7. The digital integrated circuit of claim 6, wherein each of 3×

3 filter kernels of the first and the second identity-value convolutional layers contains numerical value zero in each of nine positions of each off-diagonal kernel, and contains numerical value zero in each of eight perimeter positions and numerical value one in center position of each diagonal kernel.