Natural language processing using a CNN based integrated circuit

• US 10,083,171 B1
• Filed: 09/19/2017
• Issued: 09/25/2018
• Est. Priority Date: 08/03/2017
• Status: Active Grant

• Current Assignee: Gyrfalcon Technology Inc.
• Original Assignee: Gyrfalcon Technology Inc.
• Inventors: Yang, Lin, Dong, Patrick Z., Sun, Baohua
• Primary Examiner(s): OSINSKI, MICHAEL S
• Application Number:
  US15/709,220
• Time in Patent Office: 371 Days
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/53 Processing of non-Latin text kana-to-kanji conversion G06F40/129; vowelisation G06F40/232
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A string of natural language texts is received and formed a multi-layer 2-D symbol in a computing system. The 2-D symbol comprises a matrix of N×N pixels of K-bit data representing a “super-character”. The matrix is divided into M×M sub-matrices with each sub-matrix containing (N/M)×(N/M) pixels. K, N and M are positive integers, and N is preferably a multiple of M. Each sub-matrix represents one ideogram defined in an ideogram collection set. “Super-character” represents a meaning formed from a specific combination of a plurality of ideograms. The meaning of the “super-character” is learned by classifying the 2-D symbol via a trained convolutional neural networks model having bi-valued 3×3 filter kernels in a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit.

Claims

1. A method of natural language processing using a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit, the method comprising:

receiving a string of natural language texts in a computing system;

forming, with a two-dimensional symbol creation module installed in the computing system, a multi-layer two-dimensional (2-D) symbol from the received string of natural language texts based on a set of 2-D symbol creation rules, the 2-D symbol being a matrix of N×

N pixels of K-bit data that contains a super-character, wherein the matrix is divided into M×

M sub-matrices with each of the sub-matrices containing (N/M)×

(N/M) pixels, said each of the sub-matrices representing one ideogram defined in an ideogram collection set, and the super-character representing a meaning formed from a specific combination of a plurality of ideograms, where K, N and M are positive integers or whole numbers, and N is a multiple of M; and

learning the meaning of the super-character by classifying the 2-D symbol via a trained convolutional neural networks model having bi-valued 3×

3 filter kernels in a Cellular Neural Networks or Cellular Nonlinear Networks (CNN) based integrated circuit.

2. The method of claim 1, wherein the trained convolutional neural networks model is achieved with following operations:

(a) obtaining a convolutional neural networks model by training the convolutional neural networks model based on image classification of a labeled dataset, which contains a number of multi-layer 2-D symbols, the convolutional neural networks model including multiple ordered filter groups, each filter in the multiple ordered filter groups containing a standard 3×

3 filter kernel;

(b) modifying the convolutional neural networks model by converting the respective standard 3×

3 filter kernels to corresponding bi-valued 3×

3 filter kernels of a currently-processed filter group in the multiple ordered filter groups based on a set of kernel conversion schemes;

(c) retraining the modified convolutional neural networks model until a desired convergence criterion is met; and

(d) repeating (b)-(c) for another filter group until all of the multiple ordered filter groups are converted to the bi-valued 3×

3 filter kernels.

3. The method of claim 2, further comprises transforming said all of the multiple ordered filter groups from a floating point number format to a fixed point number format to accommodate data structure required in the CNN based integrated circuit.

4. The method of claim 2, said modifying the convolutional neural networks model further comprises optionally combining two or more consecutive filter groups for the converting operation.

5. The method of claim 2, said retraining the modified convolutional neural networks model further comprises optionally freezing the already converted filter groups.

6. The method of claim 2, wherein the set of kernel conversion schemes comprises each of the nine coefficients in the corresponding bi-valued 3×

3 filter kernel is assigned a value equal to the average of absolute coefficient values multiplied by the sign of corresponding coefficients in the standard 3×

3 filter kernel.

7. The method of claim 2, wherein the desired convergence criterion comprises completing a predefined number of said retraining operations.

8. The method of claim 2, wherein the desired convergence criterion comprises converging of accuracy loss due to filter kernel conversion.

9. The method of claim 2, wherein the set of 2-D symbol creation rules comprises:

determining a size of the received string of natural language texts;

if the size of the receiving string is greater than M×

M, modifying the received string via at least one language text reduction scheme such that the size of the string is equal to M×

M, and then converting the string to the multi-layer 2-D symbol;

if the size of the receiving string is equal to M×

M, converting the received string to the multi-layer 2-D symbol; and

if the size of the receiving string is less than M×

M and a padding operation is desired, adding at least one text to pad the string such that the size of the string is equal to M×

M in accordance with at least one language text increase scheme, and then converting the padded string to the multi-layer 2-D symbol;

otherwise, converting the received string to the multi-layer 2-D symbol, which contains at least one empty space.

10. The method of claim 9, wherein said at least one language text reduction scheme comprises deleting at least one unimportant text from the received string according to at least one relevant grammar based rule.

11. The method of claim 10, wherein the at least one relevant grammar based rule is associated with the received string of natural language texts.

12. The method of claim 10, wherein said at least one language text reduction scheme comprises a randomized text reduction scheme.

13. The method of claim 12, wherein the randomized text reduction scheme comprises truncating the string such that the size of the string is reduced to M×

M.

14. The method of claim 12, wherein the randomized text reduction scheme comprises arbitrarily selecting certain texts in the string such that the size of the string is reduced to M×

M.

15. The method of claim 9, wherein said at least one language text increase scheme comprises identifying one or more key texts from the received string and then repeatedly appending said one or more key texts to the string.

16. The method of claim 9, wherein said at least one language text increase scheme comprises repeatedly appending one or more texts from the received string to the string.

17. The method of claim 2, wherein the super-character contains a maximum of M×

M ideograms.

18. The method of claim 1, wherein K is 5, N is 224, M is 4, M×

M is 16 and N/M is 56.

19. The method of claim 1, wherein K is 5, N is 224, M is 8, M×

M is 64 and N/M is 28.