System, method and program for discriminating named entity

• US 20020031269A1
• Filed: 09/07/2001
• Published: 03/14/2002
• Est. Priority Date: 09/08/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Fukushima, Toshikazu
• Granted Patent: US 6,975,766 B2
• Time in Patent Office: Days
• US Class Current: 382/228
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/94 Hypermedia Hyperlinking G06F40/134

Abstract

A named entity discriminating system capable of discriminating names entities such as location names, personal names, and organization names in text with a high degree of accuracy is provided. A reading means reads text from a hypertext database. A single text analyzing means analyzes each text read by the reading means and detects candidates for the named entity in the text. A complex text analyzing means estimates the likelihood of the candidate named entity detected by the single text analyzing means by an analysis with reference to referring link text or linked text of the text in which the candidate named entity appears.

Claims

1. A named entity discriminating system for detecting named entities composed of location names, personal names and organization names in text, comprising:

a reading means for reading text from a hypertext database;

a single text analyzing means for analyzing the text read by the reading means to detect candidates for the named entity in the text; and

a complex text analyzing means for estimating the likelihood of the candidate named entity detected by the single text analyzing means by an analysis with reference to referring link text and/or linked text of the text in which the candidate named entity appears.

2. A named entity discriminating system as claimed in claim 1, wherein the complex text analyzing means estimates the likelihood of the candidate named entity detected by the single text analyzing means based on:

appearance frequency information of words co-occurring with the candidate named entity in referring link text and/or linked text of the text in which the candidate named entity appears;

co-occurrence relations between the candidate named entity and a key character string in referring link text of the text in which the candidate named entity appears;

or

appearance frequency information of words co-occurring with the candidate named entity in a specified range before and after a key character string in referring link text of the text in which the candidate named entity appears.

3. A named entity discriminating system as claimed in claim 2, wherein when no co-occurring word is detected in the referring link text and/or linked text of the text in which the candidate named entity appears, the complex text analyzing means detects the co-occurring words referring to referring link text or linked text of the text in which no co-occurring word is detected.

4. A named entity discriminating system as claimed in claim 2, wherein the complex text analyzing means follows a link from the text in which the candidate named entity appears to referring link text or linked text, and searches prescribed link layers from the text specified as a reference range for the co-occurring word.

5. A named entity discriminating system as claimed in claim 2, wherein the complex text analyzing means estimates the likelihood of the candidate named entity based on the weight which is assigned to each co-occurring word according to the position thereof.

6. A named entity discriminating method, comprising steps of:

reading text from a hypertext database;

analyzing the text to detect candidates for the named entity; and

estimating the likelihood of the candidate named entity by an analysis with reference to referring link text and/or linked text of the text in which the candidate named entity appears.

7. A named entity discriminating method as claimed in claim 6, wherein the likelihood of the candidate named entity is estimated based on:

appearance frequency information of words co-occurring with the candidate named entity in referring link text and/or linked text of the text in which the candidate named entity appears;

co-occurrence relations between the candidate named entity and a key character string in referring link text of the text in which the candidate appears;

or

appearance frequency information of words co-occurring with the candidate named entity in a specified range before and after a key character string in referring link text of the text in which the candidate named entity appears.

8. A named entity discriminating method as claimed in claim 7, wherein when no co-occurring word is detected in the referring link text and/or linked text of the text in which the candidate appears, the co-occurring words are detected referring to referring link text or linked text of the text in which no co-occurring word is detected.

9. A named entity discriminating method as claimed in claim 7, wherein the likelihood of the candidate named entity is estimated by following a link from the text in which the candidate named entity appears to referring link text or linked text, and searching prescribed link layers from the text specified as a reference range for the co-occurring word.

10. A named entity discriminating method as claimed in claim 7, wherein the likelihood of the candidate named entity is estimated based on the weight which is assigned to each co-occurring word according to the position thereof.

11. A named entity discriminating program to execute processes including:

a reading process for reading text from a hypertext database;

a single text analyzing process for analyzing the text read in the reading process to detect candidates for the named entity; and

a complex text analyzing means for estimating the likelihood of the candidate named entity detected in the single text analyzing process by an analysis referring to referring link text and/or linked text of the text in which the candidate named entity appears.

12. A named entity discriminating program as claimed in claim 11, wherein the complex text analyzing process estimates the likelihood of the candidate named entity detected by the single text analyzing process based on:

appearance frequency information of words co-occurring with the candidate named entity in referring link text and/or linked text of the text in which the candidate named entity appears;

co-occurrence relation between the candidate named entity and a key character string in referring link text of the text in which the candidate named entity appears;

or

appearance frequency information of words co-occurring with the candidate named entity in a specified range before and after a key character string in referring link text of the text in which the candidate named entity appears.

13. A named entity discriminating program as claimed in claim 12, wherein when no co-occurring word is detected in the referring link text and/or linked text of the text in which the candidate named entity appears, the complex text analyzing process detects the co-occurring words referring to referring link text or linked text of the text in which no co-occurring word is detected.

14. A named entity discriminating program as claimed in claim 12, wherein the complex text analyzing process follows a link from the text in which the candidate named entity appears to referring link text or linked text, and searches prescribed link layers from the text specified as a reference range for the co-occurring word.

15. A named entity discriminating program as claimed in claim 12, wherein the complex text analyzing process estimates the likelihood of the candidate named entity based on the weight which is assigned to each co-occurring word according to the position thereof.