DISAMBIGUATION IN CONCEPT IDENTIFICATION

• US 20160371587A1
• Filed: 11/30/2015
• Published: 12/22/2016
• Est. Priority Date: 06/16/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: XU, QIONG KAI, CAO, FENG, NI, YUAN, ZHU, HUI JIA
• Granted Patent: US 9,740,683 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A method, a disambiguation decoder and a system is provided for disambiguation in concept identification. A set of candidate concepts of a surface form in a sequence of surface forms in a plaintext is obtained. First probabilities for the candidate concepts are then determined, wherein a first probability indicates likelihood that the surface form represents the respective candidate concept. One of the candidate concepts is selected for the surface form based on the first probabilities and adjacency of the surface forms in the sequence.

Claims

1. A computer-implemented method, comprising:

obtaining, with a processing device, a set of candidate concepts of a surface form in a sequence of surface forms in a plaintext;

determining first probabilities for the candidate concepts, a first probability indicating likelihood that the surface form represents the respective candidate concept; and

selecting one of the candidate concepts for the surface form based on the first probabilities and adjacency of the surface forms in the sequence.

2. The method of claim 1, wherein selecting one of the candidate concepts for the surface form based on the first probabilities and adjacency of the surface forms in the sequence comprises:

determining occurrence probabilities for the candidate concepts based on the first probabilities and the adjacency of the surface forms in the sequence, an occurrence probability indicating likelihood that the respective candidate concept is mentioned in the plaintext; and

selecting one of the candidate concepts for the surface form based on the occurrence probabilities.

3. The method of claim 2, wherein determining occurrence probabilities for the candidate concepts comprises responsive to a determination that the surface form is located at a start of the sequence, determining the first probabilities for the candidate concepts as the occurrence probabilities.

4. The method of claim 2, wherein determining occurrence probabilities for the candidate concepts comprises:

responsive to a determination that the surface form has a preceding surface form in the sequence, for each of the candidate concepts;

determining second probabilities with respect to preceding candidate concepts of the preceding surface form, a second probability indicating likelihood that the candidate concept is mentioned after the respective preceding candidate concept;

determining candidate occurrence probabilities based on the occurrence probabilities for the preceding candidate concepts, the first probability and the second probabilities; and

selecting a maximal candidate occurrence probability from among the candidate occurrence probabilities as the occurrence probability.

5. The method of claim 2, wherein selecting one of the candidate concepts for the surface form based on the occurrence probabilities comprises responsive to a determination that the surface form is located at an end of the sequence, selecting a candidate concept with the maximal occurrence probability from the candidate concepts.

6. The method of claim 2, wherein selecting one of the candidate concepts for the surface form based on the occurrence probabilities comprises responsive to a determination that the surface form has a subsequent surface form in the sequence, selecting one of the candidate concepts according to the occurrence probability for the selected candidate concept of the subsequent surface form.

7. The method of claim 1, further comprising spotting words in the plaintext to generate the sequence of surface forms.