SYSTEM AND METHODS FOR GENERATING TREEBANKS FOR NATURAL LANGUAGE PROCESSING BY MODIFYING PARSER OPERATION THROUGH INTRODUCTION OF CONSTRAINTS ON PARSE TREE STRUCTURE

• US 20160259851A1
• Filed: 03/03/2016
• Published: 09/08/2016
• Est. Priority Date: 03/04/2015
• Status: Abandoned Application

• Current Assignee: Allen Institute For Artificial Intelligence
• Original Assignee: Allen Institute For Artificial Intelligence
• Inventors: Hopkins, Mark Andrew, Schaake, Mark Edwin, Skjonsberg, Samuel Stuart
• Application Number:
  US15/059,722
• Publication Number:
  US 20160259851A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/345 Summarisation for human users
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/226 Validation

Abstract

Systems, apparatuses, and methods for generating a parser training set and ultimately a correct treebank for a corpus of text, based on using an existing parser that was trained on a different corpus. Also disclosed are systems, apparatuses, and methods for improving the operation of a parser in the case of using a less familiar set of training data than is typically used to train conventional parsers. This can be used to generate a more effective and accurate parser for a new corpus (and hence more accurate parse trees) with significantly less effort than would be required if it was necessary to generate a standard size training set.

Claims

1. A method for modifying the operation of a parser, comprising:

receiving data representing an input sentence;

generating a display of a structure representing the input sentence based on a specific parsing process;

receiving one or more inputs representing changes to the displayed structure;

generating a corrected structure representing the input sentence based on the specific parsing process as modified by the received inputs; and

training a parser to reliably learn a parsing process based on the specific parsing process as modified by the one or more received inputs.

2. The method of claim 1, wherein the display is a tree structure.

3. The method of claim 2, further comprising using the trained parser to generate a treebank.

4. The method of claim 1, wherein the one or more inputs are provided by an annotator and include or represent a rule, a constraint, a condition or an exclusion.

5. The method of claim 1, wherein the one or more inputs are provided by an automated decision process and include or represent a rule, a constraint, a condition or an exclusion.

6. The method of claim 1, wherein the parser is trained by using a classifier that operates to learn a sequence of parsing operations.

7. The method of claim 1, wherein the specific parsing process was developed based on a first corpus of sentences, and the input sentence is from a second corpus of sentences.

8. The method of claim 2, wherein the one or more inputs include or represent a rule, a constraint, a condition or an exclusion, and result in a requirement for a connection between two nodes of the structure or the prevention of a connection between two nodes of the structure.

9. An apparatus, comprising:

an electronic data processing element;

a set of instructions stored on a non-transient medium and executable by the electronic data processing element, which when executed cause the apparatus to

receive data representing an input sentence;

generate a display of a structure representing the input sentence based on a specific parsing process;

receive one or more inputs representing changes to the displayed structure;

generate a corrected structure representing the input sentence based on the specific parsing process as modified by the received inputs; and

train a parser to reliably learn a parsing process based on the specific parsing process as modified by the one or more received inputs.

10. The apparatus of claim 9, wherein the display is a tree structure.

11. The apparatus of claim 10, further comprising instructions causing the apparatus to use the trained parser to generate a treebank.

12. The apparatus of claim 9, wherein the one or more inputs are provided by an annotator and include or represent a rule, a constraint, a condition or an exclusion.

13. The apparatus of claim 9, wherein the one or more inputs are provided by an automated decision process and include or represent a rule, a constraint, a condition or an exclusion.

14. The method of claim 9, wherein the parser is trained by using a classifier that operates to learn a sequence of parsing operations.

15. The method of claim 9, wherein the specific parsing process was developed based on a first corpus of sentences, and the input sentence is from a second corpus of sentences.

16. The method of claim 11, wherein the one or more inputs include or represent a rule, a constraint, a condition or an exclusion, and result in a requirement for a connection between two nodes of the structure or the prevention of a connection between two nodes of the structure.

17. A system comprising:

a data storage element containing data representing one or more sentences or strings of characters;

an electronic data processing element;

a set of instructions stored on a non-transient medium and executable by the electronic data processing element, which when executed cause the system to

generate a visual display of a structure representing the result of parsing one of the sentences or strings of characters using a first parsing process;

receive one or more inputs representing changes to the displayed structure;

generate a visual display of a corrected structure, the corrected structure representing the result of parsing the sentence using the first parsing process as modified by the received inputs; and

train a parser execute a second parsing process, the second parsing process being based on the first parsing process as modified by the one or more received inputs.

18. The system of claim 17, wherein

the visual display is a tree structure;

the first parsing process was developed based on inputs from a first domain;

the one or more sentences or strings of characters are members of a second domain that would not be reliably parsed using the first parsing process; and

the one or more received inputs include or represent a rule, a constraint, a condition or an exclusion.

19. The system of claim 17, further comprising instructions causing the system to use the trained parser to generate a treebank using the data representing one or more sentences or strings of characters.