Pattern identification and correction of document misinterpretations in a natural language processing system

• US 9,678,947 B2
• Filed: 11/21/2014
• Issued: 06/13/2017
• Est. Priority Date: 11/21/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Allen, Corville O., Dettman, Elizabeth T., Payne, Michael T., Schroeder, Michael W., Freed, Andrew R.
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US14/550,218
• Publication Number:
  US 20160147734A1
• Time in Patent Office: 935 Days
• Field of Search: 704 9, 704 10, 704235, 704257, 715256, 715257, 715260
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

An approach is provided in which a knowledge manager analyzes multiple document phrases using a natural language processing model and generates multiple interpretations based upon the analysis. The knowledge manager identifies misinterpretation patterns by comparing the multiple interpretations with multiple corrections that include corrections to the multiple interpretations. In turn, the knowledge manager generates interpretation rules based upon the identified patterns and applies the interpretation rules to the natural language processing model.

Claims

1. An information handling system comprising:

one or more processors;

a memory coupled to at least one of the processors; and

a set of computer program instructions stored in the memory and executed by at least one of the processors in order to perform actions of;

generating a plurality of interpretations of a plurality of document phrases using a natural language processing model, wherein the plurality of interpretations are different than the plurality of document phrases;

receiving a plurality of corrections corresponding to a set of words included in the plurality of interpretations;

determining one or more misinterpretation patterns in response to evaluating the plurality of corrections against the plurality of interpretations and the plurality of document phrases;

generating one or more interpretation rules based upon the one or more misinterpretation patterns;

generating a synonym interpretation rule of a selected one of the one or more interpretation rules, wherein the synonym interpretation rule includes a synonym word of a selected one of the set of words included in the selected interpretation rule;

applying the one or more interpretation rules and the synonym interpretation rule to the natural language processing model; and

evaluating a new document phrase by the natural language processing model, wherein the evaluating further comprises;

generating a probabilistic score of the new document phrase based upon a sentence similarity between the new document phrase and the one or more interpretation rules or the synonym interpretation rule; and

ranking, based on the probabilistic score, a new interpretation of the new document phrase.

2. The information handling system of claim 1 wherein the one or more processors perform additional actions comprising:

determining a correct interpretation of the set of words in response to detecting that each of the plurality of interpretations are incorrect; and

including the set of words and the correct interpretation in a first one of the one or more interpretation rules.

3. The information handling system of claim 2 wherein the one or more processors perform additional actions comprising:

identifying one or more complex sentence patterns corresponding to the first interpretation rule based upon one or more complex sentence structures, wherein each of the one or more complex sentence patterns include one or more additional entities selected from the group consisting of an adjective and a preposition; and

generating one or more complex interpretation rules based upon the one or more complex sentence patterns, wherein each of the one or more complex interpretation rules includes the set of words, the correct interpretation, and one or more of the additional entities.

4. The information handling system of claim 1 wherein the natural language processing model performs syntactic analysis and semantic analysis on each of the plurality of document phrases.

5. A computer program product stored in a computer readable storage medium, comprising computer program code that, when executed by an information handling system, causes the information handling system to perform actions comprising:

generating a plurality of interpretations of a plurality of document phrases using a natural language processing model, wherein the plurality of interpretations are different than the plurality of document phrases;

receiving a plurality of corrections corresponding to a set of words included in the plurality of interpretations;

determining one or more misinterpretation patterns in response to evaluating the plurality of corrections against the plurality of interpretations and the plurality of document phrases;

generating one or more interpretation rules based upon the one or more misinterpretation patterns;

generating a synonym interpretation rule of a selected one of the one or more interpretation rules, wherein the synonym interpretation rule includes a synonym word of a selected one of the set of words included in the selected interpretation rule;

applying the one or more interpretation rules and the synonym interpretation rule to the natural language processing model; and

evaluating a new document phrase by the natural language processing model, wherein the evaluating further comprises;

generating a probabilistic score of the new document phrase based upon a sentence similarity between the new document phrase and the one or more interpretation rules or the synonym interpretation rule; and

ranking, based on the probabilistic score, a new interpretation of the new document phrase.

6. The computer program product of claim 5 wherein the information handling system performs additional actions comprising:

determining a correct interpretation of the set of words in response to detecting that each of the plurality of interpretations are incorrect; and

including the set of words and the correct interpretation in a first one of the one or more interpretation rules.

7. The computer program product of claim 6 wherein the information handling system performs additional actions comprising:

identifying one or more complex sentence patterns corresponding to the first interpretation rule based upon one or more complex sentence structures, wherein each of the one or more complex sentence patterns include one or more additional entities selected from the group consisting of an adjective and a preposition; and

generating one or more complex interpretation rules based upon the one or more complex sentence patterns, wherein each of the one or more complex interpretation rules includes the set of words, the correct interpretation, and one or more of the additional entities.

8. The computer program product of claim 5 wherein the natural language processing model performs syntactic analysis and semantic analysis on each of the plurality of document phrases.