Anaphora resolution for medical text with machine learning and relevance feedback

• US 10,366,161 B2
• Filed: 08/02/2017
• Issued: 07/30/2019
• Est. Priority Date: 08/02/2017
• Status: Active Grant

• Current Assignee: Merative US LP
• Original Assignee: International Business Machines Corporation
• Inventors: Allen, Corville O., DeLima, Roberto, Ezen Can, Aysu, Sizemore, Robert C.
• Primary Examiner(s): Patel, Shreyans A
• Application Number:
  US15/666,694
• Publication Number:
  US 20190042559A1
• Time in Patent Office: 727 Days
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

The program directs a computer processor to resolve an anaphor in electronic natural language text. The program detects a plurality of entities and an anaphor in a span of parsed natural language text comprising one or more sentences, and extracts pairs of related entities based on domain knowledge. The program constructs a set of tuples, wherein each tuple is a data type comprising an anaphor, an antecedent entity (AE) appearing before the anaphor in the span of parsed natural language text, and an entity (E) appearing after the anaphor in the span of parsed natural language text, wherein the anaphor refers to the AE and relates the AE to the E. The program resolves the anaphor by determining which entity in the plurality of entities the anaphor references, using the constructed set of tuples, and selecting an AE among one or more candidate AEs.

Claims

1. A method for directing a computer processor to resolve an anaphor in electronic natural language text, comprising:

detecting a plurality of entities and an anaphor in a span of parsed natural language text comprising one or more sentences;

extracting pairs of related entities among the plurality of entities, based on domain knowledge;

constructing a set of tuples, wherein each tuple is a data type comprising an anaphor, an antecedent entity (AE) appearing before the anaphor in the span of parsed natural language text, and an entity (E) appearing after the anaphor in the span of parsed natural language text, wherein the anaphor refers to the AE and relates the AE to the E, and wherein the constructing is based on initial training data and the extracted pairs of related entities; and

resolving the anaphor by determining which entity in the plurality of entities the anaphor references, using the constructed set of tuples, and selecting an AE among one or more candidate AEs.

2. The method of claim 1, wherein the domain knowledge comprises a collection of known pairs of related entities, or a logical knowledge graph comprising known pairs of related entities, or a set of natural language statements about the domain, or any combination thereof.

3. The method of claim 1, wherein the initial training data comprises a set of training tuples constructed using a set of training sentences, each training tuple comprising:

an anaphor in the set of training sentences, an antecedent entity (AE) in the set of training sentences, and an entity (E) in the set of training sentences, wherein the anaphor is known to reference the AE, and the anaphor is known to relate the AE to the E.

4. The method of claim 1, wherein for a given anaphor to be resolved, resolving the anaphor comprises:

determining, using the constructed set of tuples, that the anaphor potentially refers to only one candidate AE; and

determining a first instance of the only one candidate AE, appearing prior to the anaphor, as the entity in the plurality of entities to which the anaphor refers.

5. The method of claim 1, wherein for a given anaphor to be resolved, resolving the anaphor comprises:

determining, using the constructed set of tuples, that there are multiple candidate AEs which the anaphor may reference, in relation to an entity under consideration; and

performing disambiguation logic to determine which entity, among the multiple candidate AEs, the anaphor references in relation to the entity under consideration.

6. The method of claim 5, wherein performing disambiguation logic comprises:

identifying a set of features in a set of training data comprising training tuples corresponding to a training set of natural language text, each training tuple comprising;

an unresolved anaphor, a set of candidate antecedent entities (AEs), and an entity (E), wherein at least one candidate AE is known to resolve the unresolved anaphor; and

generating a statistical model of the training tuples wherein the statistical model is based on features of the training tuples that most accurately predict the at least one candidate AE known to resolve the unresolved anaphor.

7. The method of claim 6, wherein the statistical model is further based on features of the training tuples that most accurately predict at least one candidate AE known not to resolve the unresolved anaphor.

8. The method of claim 6, wherein performing disambiguation logic further comprises applying the statistical model to the constructed set of tuples to select an AE among the one or more candidate AEs.

9. The method of claim 8, wherein applying the statistical model to the constructed set of tuples comprises generating scores for the constructed set of tuples and selecting a candidate AE among the one or more candidate AEs based on the selected candidate AE having a score within a threshold of a desired score.

10. The method of claim 1, further comprising:

obtaining relevance feedback, from a user, to evaluate the AE selected among the one or more candidate AEs; and

updating the statistical model based on the relevance feedback.

11. The method of claim 1, further comprising:

expanding the initial training data based on the information obtained either from performing disambiguation logic or utilizing relevance feedback, or both.

12. A computer program product for directing a computer processor to resolve an anaphor in electronic natural language text, comprising a non-transitory tangible storage device having program code embodied therewith, the program code executable by a processor of a computer to perform a method, the method comprising:

detecting, by the processor, a plurality of entities and an anaphor in a span of parsed natural language text comprising one or more sentences;

extracting, by the processor, pairs of related entities among the plurality of entities, based on domain knowledge;

constructing, by the processor, a set of tuples, wherein each tuple is a data type comprising an anaphor, an antecedent entity (AE) appearing before the anaphor in the span of parsed natural language text, and an entity (E) appearing after the anaphor in the span of parsed natural language text, wherein the anaphor refers to the AE and relates the AE to the E, and wherein the constructing is based on initial training data and the extracted pairs of related entities; and

resolving, by the processor, the anaphor by determining which entity in the plurality of entities the anaphor references, using the constructed set of tuples, and selecting an AE among one or more candidate AEs.

13. The computer program product of claim 12, wherein for a given anaphor to be resolved, resolving the anaphor comprises:

determining, by the processor, using the constructed set of tuples, that there are multiple candidate AEs which the anaphor may reference, in relation to an entity under consideration; and

performing, by the processor, disambiguation logic to determine which entity, among the multiple candidate AEs, the anaphor references in relation to the entity under consideration.

14. The computer program product of claim 13, wherein performing disambiguation logic comprises:

identifying, by the processor, a set of features in a set of training data comprising training tuples corresponding to a training set of natural language text, each training tuple comprising;

an unresolved anaphor, a set of candidate antecedent entities (AEs), and an entity (E), wherein at least one candidate AE is known to resolve the unresolved anaphor; and

generating, by the processor, a statistical model of the training tuples wherein the statistical model is based on features of the training tuples that most accurately predict the at least one candidate AE known to resolve the unresolved anaphor.

15. The computer program product of claim 14, wherein performing disambiguation logic further comprises applying the statistical model to the constructed set of tuples to select an AE among the one or more candidate AEs.

16. The computer program product of claim 12, further comprising:

obtaining, by the processor, relevance feedback, from a user, to evaluate the AE selected among the one or more candidate AEs; and

updating, by the processor, the statistical model based on the relevance feedback.

17. A computer system for performing electronic natural language processing on unstructured data, comprising:

one or more computer devices each having one or more processors and one or more tangible storage devices; and

a program embodied on at least one of the one or more storage devices, the program having a plurality of program instructions for execution by the one or more processors, the program instructions comprising instructions for;

detecting, by the computer, a plurality of entities and an anaphor in a span of parsed natural language text comprising one or more sentences;

extracting, by the computer, pairs of related entities among the plurality of entities, based on domain knowledge;

constructing, by the computer, a set of tuples, wherein each tuple is a data type comprising an anaphor, an antecedent entity (AE) appearing before the anaphor in the span of parsed natural language text, and an entity (E) appearing after the anaphor in the span of parsed natural language text, wherein the anaphor refers to the AE and relates the AE to the E, and wherein the constructing is based on initial training data and the extracted pairs of related entities; and

resolving, by the computer, the anaphor by determining which entity in the plurality of entities the anaphor references, using the constructed set of tuples, and selecting an AE among one or more candidate AEs.

18. The computer system of claim 17, wherein for a given anaphor to be resolved, resolving the anaphor comprises:

determining, by the computer, using the constructed set of tuples, that there are multiple candidate AEs which the anaphor may reference, in relation to an entity under consideration; and

performing, by the computer, disambiguation logic to determine which entity, among the multiple candidate AEs, the anaphor references in relation to the entity under consideration.

19. The computer system of claim 18, wherein performing disambiguation logic comprises:

identifying, by the computer, a set of features in a set of training data comprising training tuples corresponding to a training set of natural language text, each training tuple comprising;

an unresolved anaphor, a set of candidate antecedent entities (AEs), and an entity (E), wherein at least one candidate AE is known to resolve the unresolved anaphor; and

generating, by the computer, a statistical model of the training tuples wherein the statistical model is based on features of the training tuples that most accurately predict the at least one candidate AE known to resolve the unresolved anaphor.

20. The computer system of claim 17, further comprising:

obtaining, by the computer, relevance feedback, from a user, to evaluate the AE selected among the one or more candidate AEs; and

updating, by the computer, the statistical model based on the relevance feedback.