OPERATION APPARATUS AND METHOD FOR CONVOLUTIONAL NEURAL NETWORK

• US 20180089562A1
• Filed: 06/09/2017
• Published: 03/29/2018
• Est. Priority Date: 09/28/2016
• Status: Active Grant

• Current Assignee: SK Hynix Inc.
• Original Assignee: SK Hynix Inc.
• Inventors: JIN, Young-Jae, MOON, Young-Suk, KIM, Hong-Sik
• Granted Patent: US 11,449,745 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 17/15 Correlation function computation including computation of convolution operations arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 2207/4824 Neural networks
• CPC Class Codes: G06F 7/501 Half or full adders, i.e. basic adder cells for one denomination
• CPC Class Codes: G06F 7/523 Multiplying only
• CPC Class Codes: G06F 7/5443 Sum of products for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

Disclosed herein is a convolutional neural network (CNN) operation apparatus, including at least one channel hardware set suitable for performing a feature extraction layer operation and a classification layer operation based on input data and weight data, and a controller coupled to the channel hardware set. The controller may control the channel hardware set to perform the feature extraction layer operation and perform a classification layer operation when the feature extraction layer operation is completed.

Claims

1. A convolutional neural network (CNN) operation apparatus, comprising:

at least one channel hardware set suitable for performing a feature extraction layer operation and a classification layer operation based on input data and weight data; and

a controller coupled to the channel hardware set, and suitable for controlling the channel hardware set to perform the feature extraction layer operation and perform a classification layer operation when the feature extraction layer operation is completed.

2. The operation apparatus of claim 1, further comprising an information processing unit including a weight register for storing the weight data, and

wherein the channel hardware set comprises:

an input unit suitable for storing the input data;

a convolution block suitable for performing convolution operation on the input data and the weight data;

a pooling block suitable for performing pooling operation on the data on which the convolution operation has been performed; and

an output unit suitable for outputting the data on which the convolution operation has been performed or the data on which the pooling operation has been performed.

3. The operation apparatus of claim 2, wherein the controller controls the convolution block and pooling block to perform a convolution layer operation and a pooling layer operation in the feature extraction layer operation.

4. The operation apparatus of claim 3, wherein the controller performs the classification layer operation when the feature extraction layer operation is terminated and controls the convolution block to perform a fully connected layer operation in the classification layer operation.

5. The operation apparatus of claim 4, wherein the convolution block comprises:

a plurality of process elements suitable for receiving the input data and the weight data and performing multiplication operation on the received input data and weight data;

a plurality of adders suitable for adding result values of an identical identification (ID) calculated by the PEs together;

an accumulation (ACC) block suitable for accumulating the added result value having the same ID output by the adders and outputting the accumulated value to the pooling block when the accumulated value reaches a maximum value; and

a process element controller suitable for inputting the input data of the input unit and the weight data of the weight register to a ready process element among the plurality of process elements.

6. The operation apparatus of claim 5, wherein the process element controller checks values of start and end layer counts, prefetches corresponding weight data from the weight register based on the values of start and end layer counts, and inputs the input data and the prefetched weight data to the ready process element.

7. The operation apparatus of claim 3, wherein the pooling block performs the pooling operation on convolution output values if two or more convolution output values are present while convolution operation is performed by the convolution block.

8. The operation apparatus of claim 4, wherein:

the information processing unit further comprises an address map register suitable for storing an address map provided for each layer, and

the controller checks a current layer configuration of data stored in an output register, accesses data of the address map register, and outputs the accessed data to the input unit.

9. The operation apparatus of claim 8, wherein the address map is received from an external system in an initialization state.

10. The operation apparatus of claim 4, wherein:

the controller sets a zero flag of the input data when the input data is 0 and stores the zero-padded input data in the input unit, and

the convolution block omits convolution multiplication operation for input data whose zero flag has been set.

11. The operation apparatus of claim 10, wherein each of the plurality of the process elements comprises:

a multiplier suitable for multiplying the input data and the weight data;

a logical gate suitable for performing logical operation on the input data and the weight data; and

a selector suitable for selecting an output data of the multiplier or zero data based on an output of the logical gate and outputting the selected data,

wherein the selector is suitable for selecting the zero data if the input data or the weight data is a zero flag.

12. The operation apparatus of claim 11, wherein the controller sets the zero flag when the weight data is 0 and inputs the input data or the weight data to a corresponding process element.

13. The operation apparatus of claim 4, wherein the controller checks a value of a channel count after an extraction of features of channel data is completed, assigns input data of remaining channels to the channel hardware set again if the value of the channel count is not a maximum value, and performs the feature extraction layer operation.

14. The operation apparatus of claim 4, wherein the controller inputs result data of all of channels computed in the feature extraction layer operation to a single channel hardware set when a fully connected layer operation of a classification layer is performed.

15. The operation apparatus of claim 14, wherein:

the information processing unit further comprises a drop-out signal generator suitable for generating a drop-out signal, and

the convolution block processes the input data as zero data when the drop-out signal is generated.

16. The operation apparatus of claim 15, wherein:

the controller sets a zero flag of the input data when the input data is 0,

the convolution block comprises a plurality of process elements,

each suitable for performing convolution multiplication operation on the input data and the weight data, and omitting convolution multiplication operation when input data whose zero flag has been set, weight data whose zero flag has been set or the drop-out signal is received.

17. An operating method of a convolutional neural network (CNN), the operating method comprising:

performing a feature extraction layer operation by activating a convolution block and a pooling block of at least one channel hardware set; and

performing a classification layer operation by activating the convolution block of the channel hardware set when the feature extraction layer operation is completed.

18. The operating method of claim 17, wherein the performing of the feature extraction layer operation comprises:

storing input data;

performing convolution operation on the input data and weight data;

performing pooling operation on the data on which the convolution operation has been performed; and

outputting the data on which the convolution operation has been performed or the data on which the pooling operation has been performed.

19. The operating method of claim 18, wherein the performing of the convolution operation comprises:

multiplying the input data and the weight data;

adding result values which belong to the multiplied result values and which have an identical identification (ID); and

accumulating the added result value of the same ID.

20. The operating method of claim 19, wherein the performing of the convolution operation further comprises:

checking values of start and end layer counts; and

prefetching corresponding weight data from the weight register based on the values of start and end layer counts.

21. The operating method of claim 18, wherein the performing of the pooling operation comprises performing the pooling operation on convolution output values if two or more convolution operation output values are present in the convolution operation.

22. The operating method of claim 21, wherein the performing of the pooling operation comprises selecting a maximum value of two convolution operation output values.

23. The operating method of claim 21, wherein the performing of the pooling operation comprises calculating an average value of two convolution operation output values.

24. The operating method of claim 18, wherein the storing of the input data comprises:

checking a layer configuration of output data of the pooling operation;

mapping the output data to an address of an address map register based on the checked layer configuration; and

storing the output data of the pooling operation as the input data.

25. The operating method of claim 24, further comprising receiving an address map for storing the address map register from an external system in an initialization state.

26. The operating method of claim 18, wherein:

the storing of the input data comprises setting a zero flag of the input data when the input data is 0, and

the performing of the convolution operation comprises omitting multiplication operation for input data whose zero flag has been set.

27. The operating method of claim 26, wherein the performing of the convolution operation comprises:

multiplying the input data and the weight data;

analyzing zero flags of the input data and the weight data;

outputting zero data when the zero flag of the input data or the weight data is a set state; and

outputting the multiplying result when the zero flags of the input data and the weight data are a reset state.

28. The operating method of claim 22, wherein the performing of the feature extraction layer operation comprises:

checking a value of a channel count when an extraction of features of channel data is completed; and

assigning input data of remaining channels to the channel hardware set again if the value of the channel count is not a maximum value and performing the feature extraction layer operation again.

29. The operating method of claim 18, wherein the performing of the classification layer operation comprises:

inputting result data of all of channels extracted from the feature extraction layer into a single channel hardware set;

performing the convolution operation on the input data and the weight data; and

outputting the data on which the convolution operation has been performed.

30. The operating method of claim 29, wherein the performing of the convolution operation comprises:

performing, by each of a plurality of process elements, convolution multiplication operation on the input data and the weight data, and

omitting convolution multiplication operation when input data whose zero flag has been set, weight data whose zero flag has been set or the drop-out signal is received.