Deep learning device for local processing classical chinese poetry and verse

• US 10,417,342 B1
• Filed: 07/03/2018
• Issued: 09/17/2019
• Est. Priority Date: 07/03/2018
• Status: Active Grant

• Current Assignee: Gyrfalcon Technology Inc.
• Original Assignee: Gyrfalcon Technology Inc.
• Inventors: Yang, Lin, Dong, Patrick Z., Sun, Baohua
• Primary Examiner(s): Vo, Huyen X
• Application Number:
  US16/027,108
• Time in Patent Office: 441 Days
• Field of Search: 704 1- 10, 704230-277
• CPC Class Codes: G06F 13/20 for access to input/output bus
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/53 Processing of non-Latin text kana-to-kanji conversion G06F40/129; vowelisation G06F40/232
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/18057 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A local processing device contains a bus, an input interface and at least one cellular neural networks (CNN) based integrated circuit (IC). The input interface is receiving a 2-D symbol representing a Chinese poetry or verse. The 2-D symbol is a matrix of N×N pixels of K-bit data that contains a “super-character”. The matrix is divided into M×M sub-matrices each containing (N/M)×(N/M) pixels. Each of the sub-matrices represents an ideogram. K, N and M are positive integers, and N is a multiple of M. CNN based IC is configured for understanding semantic meaning of the Chinese poetry or verse within the “super-character” contained in the 2-D symbol. The ideogram is created by embedded fonts of all of the characters contained in a corresponding phrase of the Chinese poetry or verse or is a pictogram representing artistic conception of each sentence of the Chinese poetry or verse.

Claims

1. A device comprising:

a bus;

an input interface coupled to the bus for receiving a two-dimensional (2-D) symbol graphically representing a Chinese poetry or verse, the 2-D symbol being a matrix of N×

N pixels of K-bit data, the matrix being divided into M×

M sub-matrices each containing (N/M)×

(N/M) pixels, said each of the sub-matrices containing an ideogram created by embedding fonts of all characters contained in a corresponding phrase of the Chinese poetry or verse, where K, N and M are positive integers, and N is a multiple of M; and

at least one cellular neural networks (CNN) based integrated circuit (IC) operatively coupled to the bus and being configured for processing and understanding semantic meaning of the Chinese poetry or verse contained in the 2-D symbol using an image classification technique, each of the at least one CNN based IC comprising a plurality of CNN processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine includes;

a CNN processing block configured for simultaneously obtaining convolutional operations results using a corresponding portion of the 2-D symbol and pre-trained filter coefficients in a deep learning model;

a first set of memory buffers operatively coupled to the CNN processing block for storing the corresponding portion; and

a second set of memory buffers operative coupled to the CNN processing block for storing the pre-trained filter coefficients.

2. The device of claim 1, wherein the deep learning model contains a plurality of ordered convolutional layers.

3. The device of claim 2, wherein the deep learning model further contains layers of pooling and activation operations.

4. The device of claim 1, wherein the pre-trained filter coefficients are obtained by training the deep learning model using a training dataset containing labeled 2-D symbols that graphically represent classical Chinese poems or verses.

5. The device of claim 4, wherein each of the labeled 2-D symbols in the training dataset is assigned with a sentiment.

6. The device of claim 5, wherein the sentiment is in form of a Chinese expression reflecting the semantic meaning of said each classical Chinese poem or verse.

7. The device of claim 5, wherein the sentiment is in form of a picture reflecting the semantic meaning of said each classical Chinese poem or verse.

8. The device of claim 7, wherein each classical Chinese poem or verse comprises multiple Chinese characters in a particular format in terms of how many phrases and how many characters per phrase.

9. The device of claim 1 further comprises a display as an output interface.

10. The device of claim 9 further comprises a processing unit as a controller.

11. The device of claim 9 further comprises a memory.

12. The device of claim 1, wherein the input interface comprises a camera.

13. The device of claim 1, wherein the input interface comprises a touch screen display.

14. A device comprising:

a bus;

an input interface coupled to the bus for receiving a two-dimensional (2-D) symbol graphically representing a Chinese poetry or verse, the 2-D symbol being a matrix of N×

N pixels of K-bit data, the matrix being divided into M×

M sub-matrices each containing (N/M)×

(N/M) pixels, said each of the sub-matrices containing a pictogram representing an artistic perception of each sentence of the Chinese poetry or verse, where K, N and M are positive integers, and N is a multiple of M; and

at least one cellular neural networks (CNN) based integrated circuit (IC) operatively coupled to the bus and being configured for processing and understanding semantic meaning of the Chinese poetry or verse within the 2-D symbol using an image classification technique, each of the at least one CNN based IC comprising a plurality of CNN processing engines operatively coupled to at least one input/output data bus, the plurality of CNN processing engines being connected in a loop with a clock-skew circuit, each CNN processing engine includes;

a CNN processing block configured for simultaneously obtaining convolutional operations results using a corresponding portion of the 2-D symbol and pre-trained filter coefficients in a deep learning model;

a first set of memory buffers operatively coupled to the CNN processing block for storing the corresponding portion; and

a second set of memory buffers operative coupled to the CNN processing block for storing the pre-trained filter coefficients.

15. The device of claim 14, wherein the pictogram comprises an encoded graphical symbol.

16. The device of claim 14, wherein the pictogram comprises a graphical image of the Chinese poetry or verse.