Disambiguating concepts in natural language

• US 10,565,314 B2
• Filed: 04/26/2019
• Issued: 02/18/2020
• Est. Priority Date: 05/15/2017
• Status: Active Grant

• Current Assignee: Merative US LP
• Original Assignee: International Business Machines Corporation
• Inventors: Bull, Brendan, Felt, Paul Lewis
• Primary Examiner(s): Shah, Bharatkumar S
• Application Number:
  US16/396,501
• Publication Number:
  US 20190251173A1
• Time in Patent Office: 298 Days
• Field of Search: 704 9
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3347 using vector based model
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Abstract

A computer receives a plurality of text and determines a concept is present in the plurality of text. The computer determines a set of hypotheses for the determined concept, wherein the set of hypotheses is a plurality of natural language representations of the determined concept. The computer substitutes the determined concept in the plurality of text with a hypothesis from the determined set of hypotheses. The computer determines the hypothesis is valid based on analyzing the plurality of text with a neural network, wherein the neural network is trained for hypothesis validation. Based on determining that the hypothesis is valid, the computer storing the plurality of text with the determined hypothesis in place of the substituted concept and displays the stored plurality of text.

Claims

1. A processor-implemented method for concept disambiguation in a natural language input, the method comprising:

receiving a plurality of text;

determining a concept in the plurality of text, wherein the concept is determined using a word embedding, wherein the word embedding converts the concept to a vector;

determining a set of hypotheses for the determined concept, wherein the set of hypotheses is a plurality of natural language representations of the determined concept, wherein the plurality of natural language representations are determined based on searching an embedded concept ontology data with the vector;

analyzing a plurality of concept ontology data for the determined set of hypotheses corresponding to the determined concept;

associating the determined set of hypotheses with the determined concept based on analyzing the plurality of concept ontology data;

substituting the determined concept in the plurality of text with a hypothesis from the determined set of hypotheses, wherein the hypothesis is selected from the determined set of hypotheses based on prioritizing the hypothesis with a longer surface forms;

determining the hypothesis is valid based on analyzing the plurality of text with a neural network, wherein the neural network is trained for hypothesis validation using positive training examples having an example concept and negative training examples, wherein the negative training examples are based on substituting randomly the example concept of the positive training example with a syntactically invalid concept;

storing the plurality of text with the determined hypothesis in place of the substituted concept based on determining that the hypothesis is valid;

displaying the stored plurality of text;

determining a next hypothesis from the determined set of hypotheses for the determined concept;

based on determining the next hypothesis, determining that the next hypothesis is valid based on analyzing the plurality of text with the neural network;

storing the plurality of text with the determined next hypothesis based on determining that the next hypothesis is valid; and

displaying the plurality of text with the next hypothesis.