Hierarchical category classification scheme using multiple sets of fully-connected networks with a CNN based integrated circuit as feature extractor

• US 10,366,302 B2
• Filed: 11/21/2017
• Issued: 07/30/2019
• Est. Priority Date: 10/10/2016
• Status: Active Grant

• Current Assignee: Gyrfalcon Technology Inc.
• Original Assignee: Gyrfalcon Technology Inc.
• Inventors: Yang, Lin, Dong, Patrick Z., Sun, Baohua
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/820,253
• Publication Number:
  US 20180101748A1
• Time in Patent Office: 616 Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G10L 17/18 Artificial neural networks; Connectionist approaches
• CPC Class Codes: G10L 25/18 the extracted parameters being spectral information of each sub-band

Abstract

CNN based integrated circuit is configured with a set of pre-trained filter coefficients or weights as a feature extractor of an input data. Multiple fully-connected networks (FCNs) are trained for use in a hierarchical category classification scheme. Each FCN is capable of classifying the input data via the extracted features in a specific level of the hierarchical category classification scheme. First, a root level FCN is used for classifying the input data among a set of top level categories. Then, a relevant next level FCN is used in conjunction with the same extracted features for further classifying the input data among a set of subcategories to the most probable category identified using the previous level FCN. Hierarchical category classification scheme continues for further detailed subcategories if desired.

Claims

1. A system for classifying input data using a hierarchical category classification scheme comprising:

a cellular neural networks (CNN) based integrated circuit being loaded with pre-trained filter coefficients of convolutional layers for extracting features out of an input data that belong to a particular domain; and

a multi-processor computing unit configured for using multiple hierarchically-ordered groups of pre-trained fully-connected networks (FCNs) in a hierarchical category classification scheme that contains a set of top level categories and each of the top level categories contains at least one set of subcategories, the extracted features being repeatedly processed through corresponding ones of the multiple hierarchically-ordered groups to identify the input data as a most probable category, the multiple hierarchically-ordered groups of FCNs containing a root level group configured for the set of top level categories and at least one next level group configured for the at least one set of subcategories.

2. The system of claim 1, wherein the CNN based integrated circuit comprises a plurality of cellular neural networks (CNN) processing engines operatively coupled to the at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine comprising:

a CNN processing block configured for simultaneously obtaining convolution operations results using the input data and the pre-trained filter coefficients;

a first set of memory buffers operatively coupling to the CNN processing block for storing the input data to be fed into the CNN block; and

a second set of memory buffers operative coupling to the CNN processing block for storing the pre-trained filter coefficients to be fed into the CNN processing block.

3. The system of claim 1, wherein the particular domain comprises converted voice spectrum data.

4. The system of claim 1, wherein the particular domain comprises imagery data for face recognition.

5. The system of claim 1, wherein the particular domain comprises imagery data for optical character recognition.

6. The system of claim 1, wherein the particular domain comprises converted voice spectrum data for voice recognition.

7. The system of claim 1, wherein the root level group is used for classifying the input data by identify the input data as the most probable category among the set of top level categories.

8. The system of claim 1, wherein a relevant one of the at least one next level group is used for classifying the input data among a corresponding set of the at least one set of subcategories.

9. The system of claim 1, wherein each of the multiple hierarchically-ordered groups contains at least one FCN layer with each FCN layer containing multiple neurons or cells.

10. The system of claim 9, wherein said each FCN layer is reduced in size by removing those neurons or cells have input value of zero for more than a threshold percentage of all data in a desired dataset.

11. The system of claim 10, wherein the desired dataset comprises a labeled dataset for training in the particular domain.

12. The system of claim 10, wherein the threshold percentage is determined from the desired training dataset.

13. The system of claim 10, wherein the threshold percentage is 90%.

14. The system of claim 9, wherein said each FCN layer is reduced in size by removing those connections when numerical value of corresponding weights is equal to zero.

15. The system of claim 9, wherein said each FCN layer is reduced in size by removing those connections when numerical value of corresponding weights is less than a threshold value.

16. The system of claim 15, wherein the threshold value is determined that the contribution of the connection is too small for corresponding feature represented by the neuron.

17. The system of claim 1, wherein the pre-trained filter coefficients of convolutional layers comprises bi-valued 3×

3 filter coefficients or weights.

18. The system of claim 1, wherein the hierarchical category classification scheme is derived from Visual Geometry Group'"'"'s VGG16 model with 13 convolutional layers and three FCN layers.

19. The system of claim 17, wherein the pre-trained filter coefficients of convolutional layers are used unaltered in conjunction with each group of the multiple hierarchically-ordered groups of FCNs.