SYSTEM AND METHOD FOR CROSS-DOMAIN TRANSFERABLE NEURAL COHERENCE MODEL

• US 20200134016A1
• Filed: 10/31/2019
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: Royal Bank of Canada (The PNC Financial Services Group, Inc.)
• Original Assignee: Royal Bank of Canada (The PNC Financial Services Group, Inc.)
• Inventors: CAO, Yanshuai, XU, Peng Z., SAGHIR, Hamidreza, KANG, Jin Sung, LONG, Leo, CHEUNG, Jackie C. K.
• Granted Patent: US 11,270,072 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2207/4824 Neural networks
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

Systems and methods of automatically generating a coherence score for text data is provided. The approach includes receiving a plurality of string tokens representing decomposed portions of the target text data object. A trained neural network is provided that has been trained against a plurality of corpuses of training text across a plurality of topics. The string tokens are arranged to extract string tokens representing adjacent sentence pairs of the target text data object. For each adjacent sentence pair, the neural network generates a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object, which are then aggregated for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object.

Claims

1. A system of automatically generating a coherence score for a target text data object, the system comprising a processor operating in conjunction with non-transitory computer memory and a data storage, the processor configured to:

receive, at a string token receiver, a plurality of string tokens representing decomposed portions of the target text data object;

maintain, on the data storage, a neural network trained against a plurality of corpuses of training text across a plurality of topics, the neural network trained using string tokens of adjacent sentence pairs of the training text as positive training examples and string tokens of non-adjacent sentence pairs of the training text as negative training examples;

arrange the string tokens to extract string tokens representing adjacent sentence pairs of the target text data object;

for each adjacent sentence pair, determine, using the neural network, a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object;

aggregate the generated local coherence scores for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object; and

store the global coherence score or the generated local coherence scores in a data storage.

2. The system of claim 1, wherein the string tokens are real-valued vectors representing a first sentence s and a second sentence t, and wherein the neural network extracts features from each adjacent sentence pairs through applying representations to the real-valued vectors s and t that include at least one of:

(1) concatenation of the two vectors (s, t), (2) element-wise difference (s−

t), or (3) element-wise product (s*t), or absolute element-wise difference |s−

t|;

wherein the concatenated feature representation of the two vectors is provided into a one-layer perceptron of the neural network to generate the local coherence score for the adjacent sentence pair.

3. The system of claim 1, wherein the processor is configured to execute a discrimination task whereby the target text data object is compared to one or more random or pseudo-random permutations of portions of the target text data object using the trained neural network, and the processor is configured to output, in a data structure, a representation of a permutation of the one or more permutations having a highest global coherence score.

4. The system of claim 1, wherein the processor is configured to execute an insertion task whereby the target text data object includes a portion that has been inserted out of order relative to an original order in the target text data object and the trained neural network is utilized to generate one or more coherence scores for one or more permutations of portions of the target text data object, and the processor is configured to output, in a data structure, a representation of a permutation of the one or more permutations having a highest global coherence score, the permutation having the highest global coherence score identified as indicative of the original order.

5. The system of claim 1, wherein the processor is configured to execute a reconstruction task whereby the target text data object includes one or more portions that have been shuffled out of order relative to an original order in the target text data object and the trained neural network is utilized to generate one or more coherence scores for one or more permutations of portions of the target text data object, and the processor is configured to output, in a data structure, a representation of a permutation of the one or more permutations having a highest global coherence score, the permutation having the highest global coherence score identified as indicative of the original order.

6. The system of claim 1, wherein the neural network is configured for parallel operation, and wherein the determination using the neural network of each local coherence score is conducted across parallel computational pathways, wherein copies of the neural network are established for parallel operation of the neural network, and wherein the parallel computational pathways include using different threads or cores of one or more processors.

7. The system of claim 1, wherein the target text data object is an automatically generated summary or an original text data object, and the global coherence score is utilized in assessing a quality metric of summarization of the original text data object.

8. The system of claim 7, wherein the processor is configured to responsive to the quality metric of summarization of the original text data object being determined to be below a pre-defined threshold, transmit control signals adapted to initiate generation of a second automatically generated summary but with at least one different parameter than the original text data object.

9. The system of claim 1, wherein the plurality of topics of the training text does not include a topic of the target text object.

10. The system of claim 1, wherein the plurality of topics of the training text includes a topic of the target text object.

11. A method of automatically generating a coherence score for a target text data object, the method comprising:

receiving a plurality of string tokens representing decomposed portions of the target text data object;

providing a neural network trained against a plurality of corpuses of training text across a plurality of topics, the neural network trained using string tokens of adjacent sentence pairs of the training text as positive examples and string tokens of non-adjacent sentence pairs of the training text as negative examples;

arranging the string tokens to extract string tokens representing adjacent sentence pairs of the target text data object;

for each adjacent sentence pair, determining, using the neural network, a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object;

aggregating the generated local coherence scores for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object; and

storing the global coherence score or the generated local coherence scores in a data storage.

12. The method of claim 11, wherein the neural network is configured for parallel operation, and wherein the determination using the neural network of each local coherence score is conducted across parallel computational pathways.

13. The method of claim 12, wherein copies of the neural network are established for parallel operation of the neural network.

14. The method of claim 12, wherein the parallel computational pathways include using different threads or cores of one or more processors.

15. The method of claim 11, wherein the string tokens are real-valued vectors representing a first sentence s and a second sentence t, and wherein the neural network extracts features from each adjacent sentence pairs through applying representations to the real-valued vectors s and t that include at least one of:

(1) concatenation of the two vectors (s, t), (2) element-wise difference (s−

t), (3) element-wise product (s*t), or absolute element-wise difference |s−

t|.

16. The method of claim 15, wherein the concatenated feature representation of the two vectors is provided into a one-layer perceptron of the neural network to generate the local coherence score for the adjacent sentence pair.

17. The method of claim 11, wherein the target text data object is an automatically generated summary or an original text data object, and the global coherence score is utilized in assessing a quality metric of summarization of the original text data object.

18. The method of claim 17, comprising:

responsive to the quality metric of summarization of the original text data object being determined to be below a pre-defined threshold, transmitting control signals adapted to initiate generation of a second automatically generated summary but with at least one different parameter than the original text data object.

19. The method of claim 11, wherein the plurality of topics of the training text does not include a topic of the target text object.

20. The method of claim 11, wherein the plurality of topics of the training text includes a topic of the target text object.

21. A non-transitory computer readable medium storing machine interpretable instructions, which when executed by a processor, cause the processor to perform a method of automatically generating a coherence score for a target text data object, the method comprising:

receiving a plurality of string tokens representing decomposed portions of the target text data object;

providing a neural network trained against a plurality of corpuses of training text across a plurality of topics, the neural network trained using string tokens of adjacent sentence pairs of the training text as positive examples and string tokens of non-adjacent sentence pairs of the training text as negative examples;

arranging the string tokens to extract string tokens representing adjacent sentence pairs of the target text data object;

for each adjacent sentence pair, determining, using the neural network, a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object;

aggregating the generated local coherence scores for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object; and

storing the global coherence score or the generated local coherence scores in a data storage.

22. A non-transitory computer readable medium storing a trained neural network as machine interpretable instructions, the trained neural network which when executed by a processor, causes the processor to perform a method of automatically generating a coherence score for a target text data object, the method comprising:

receiving a plurality of string tokens representing decomposed portions of the target text data object;

providing the trained neural network that was trained against a plurality of corpuses of training text across a plurality of topics, the trained neural network trained using string tokens of adjacent sentence pairs of the training text as positive examples and string tokens of non-adjacent sentence pairs of the training text as negative examples;

arranging the string tokens to extract string tokens representing adjacent sentence pairs of the target text data object;

for each adjacent sentence pair, determining, using the trained neural network, a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object;

aggregating the generated local coherence scores for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object; and

storing the global coherence score or the generated local coherence scores in a data storage.