TRAINING OF MACHINE READING AND COMPREHENSION SYSTEMS

• US 20200134449A1
• Filed: 07/16/2019
• Published: 04/30/2020
• Est. Priority Date: 10/26/2018
• Status: Active Application

• Current Assignee: Naver Corporation
• Original Assignee: Naver Corporation
• Inventors: PEREZ, Julien, GRAIL, Quentin
• Application Number:
  US16/513,208
• Publication Number:
  US 20200134449A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A method of using a first neural network includes: by the first neural network, receiving a text; by the first neural network, receiving a question concerning the text; and by the first neural network, determining an answer to the question using the text, where the first neural network is trained to answer the question about the text adversarially by a second neural network that is trained to maximize a likelihood of failure of the first neural network to correctly answer questions.

Claims

1. A computer-executable method for training a machine reading network, comprising:

by a first neural network, analyzing a first text corpus extracted from a training dataset comprising a plurality of text corpuses;

by a second neural network, obfuscating words of a passage of the first text corpus in order to train the first neural network, the obfuscation producing a first corrupted text corpus;

by the second neural network, generating a first question about content of the first corrupted text corpus;

by the first neural network, analyzing the first corrupted text corpus and determining a first answer to the first question based on the first corrupted text corpus;

by the second neural network, analyzing the first answer of the first neural network and generating a first performance index for the first neural network based on the first answer, the first performance index representing a first answering performance of the first neural network given the obfuscation of the first corrupted text corpus;

by the second neural network storing the first performance index in memory;

by the second neural network, generating a second corrupted text corpus by obfuscating words of a second text corpus extracted from the training dataset, the obfuscation being based on the first performance index of the first neural network;

by the second neural network, generating a second question about content of the second corrupted text corpus;

by the first neural network, analyzing the second corrupted text corpus and determining a second answer to the second question based on the second corrupted text corpus;

by the second neural network, analyzing the second answer of the first neural network and generating a second performance index for the first neural network based on the second answer, the second performance index representing a second answering performance of the first neural network given the obfuscation of the second corrupted text corpus; and

by the second neural network storing the second performance index in memory.

2. The method of claim 1, wherein the obfuscation includes marking the words as unknown in the first and second corrupted text corpuses.

3. The method of claim 1 further comprising:

by the second neural network, generating a third corrupted text corpus by obfuscating words of a third text corpus extracted from the training dataset, the obfuscation being based on the first performance index of the first neural network and the second performance index of the first neural network;

by the second neural network, generating a third question about content of the third corrupted text corpus;

by the first neural network, analyzing the third corrupted text corpus and determining a third answer to the third question based on the third corrupted text corpus;

by the second neural network, analyzing the third answer of the first neural network and generating a third performance index for the first neural network based on the third answer, the third performance index representing a third answering performance of the first neural network given the obfuscation of the third corrupted text corpus; and

by the second neural network storing the third performance index in memory.

4. A system comprising memory storing instructions for performing the method of claim 1 and a processor, in communication with the memory, which executes the instructions.

5. The method of claim 1 wherein the first neural network is a gated end-to-end memory network (GMemN2N), and

wherein the second neural network is a GMemN2N.

6. The method of claim 1 further comprising:

by the first neural network, receiving a text corpus and a question regarding the text corpus from a user device;

by the first neural network, determining an answer to the question based on the text corpus; and

by the first neural network, transmitting the answer to the user device.

7. A network for training a machine reading network, comprising a neural network configured to train a machine reading network by implementing the method of claim 1.

8. A computer-readable medium, on which is stored a computer program product comprising code instructions for executing the method of claim 1 for training a machine reading network.

9. A computer-executable method for training a machine reading network, comprising:

training a first neural network to analyze text corpuses and to answer questions regarding passages of the text corpuses; and

training a second neural network to generate a first corrupted text corpus by obfuscating words of a first passage of a first text corpus in order to minimize a probability of the first neural network correctly answering a first question regarding the first corrupted text corpus,

wherein the training the second neural network includes training the second neural network to;

analyze a first answer of the first neural network to the first question;

assess a bias in the first answer caused by the obfuscation of the words of the passage of the first text corpus; and

based on the bias in the first answer of the first neural network, generate a second corrupted text corpus by obfuscating words of a second passage of a second text corpus.

10. The method of claim 9 further comprising:

by the second neural network, rating performances of the first neural network in answering questions and storing data indicative of the performances of the first neural network,

wherein generating the second corrupted text corpus includes obfuscating words of the second passage of the second text corpus based on the data indicative of the performances in order to maximize the bias of the first neural network.

11. The method of claim 9, wherein the first neural network is a gated end-to-end memory network (GMemN2N), and

wherein the second neural network is a GMemN2N.

12. The method of claim 9 wherein the generating the second text corpus includes, based on the bias in the first answer of the first neural network, generating the second corrupted text corpus by obfuscating words of the second passage of the second text corpus to minimize a second probability of the first neural network correctly answering a second question regarding the second corrupted text corpus.

13. A method comprising:

the method of claim 9;

after the training of the first neural network;

receiving, by the first neural network from a user device, a text corpus and a question about the text corpus;

determining, by the first neural network, an answer to the question using the text corpus; and

transmitting, by the first neural network to the user device, the answer to the question.

14. A non-transitory computer readable medium comprising code including instructions to execute the method of claim 9 for training a machine reading network.

15. A method of using a first neural network, comprising;

by the first neural network, receiving a text;

by the first neural network, receiving a question concerning the text; and

by the first neural network, determining an answer to the question using the text,

wherein the first neural network is trained to answer the question about the text adversarially by a second neural network that is trained to maximize a likelihood of failure of the first neural network to correctly answer questions.

16. The method of claim 15 further comprising, by the second neural network, adversarially training the first neural network by selectively obfuscating words in stories of a training dataset.

17. The method of claim 16 further comprising, by the second neural network, presenting the first neural network with progressively harder questions regarding the stories.

18. The method of claim 15 further comprising, by the second neural network, altering the text by selectively obfuscating a word in each text for which the first neural network provides an answer.

19. The method of claim 18 further comprising, by the second neural network, choosing the word to obfuscate to maximize a probability of failure by the first neural network.

20. A method comprising:

the method of claim 15;

after the training of the first neural network;

receiving, by the first neural network from a user device, a text corpus and a question about the text corpus;

determining, by the first neural network, an answer to the question using the text corpus; and

transmitting, by the first neural network to the user device, the answer to the question.