System, method and program for discriminating named entity

• US 6,975,766 B2
• Filed: 09/07/2001
• Issued: 12/13/2005
• Est. Priority Date: 09/08/2000
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Fukushima, Toshikazu
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US09/947,696
• Publication Number:
  US 20020031269A1
• Time in Patent Office: 1,558 Days
• Field of Search: 382/181, 382/209, 382/224, 382/228, 382/229, 382/305, 382/231, 707 1- 10, 707/102, 715/501.1, 715/513, 715515-516, 715/532
• US Class Current: 382/229
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/94 Hypermedia Hyperlinking G06F40/134

Abstract

A named entity discriminating system capable of discriminating names entities such as location names, personal names, and organization names in text with a high degree of accuracy is provided. A reading means reads text from a hypertext database. A single text analyzing means analyzes each text read by the reading means and detects candidates for the named entity in the text. A complex text analyzing means estimates the likelihood of the candidate named entity detected by the single text analyzing means by an analysis with reference to referring link text or linked text of the text in which the candidate named entity appears.

Claims

1. A named entity discriminating system for detecting named entities as belonging to one of location names, personal names and organization names in text, comprising:

a reading means for reading text from a hypertext database;

a single text analyzing means for analyzing at least a first text from a referring link page and separately analyzing a second text from a linked page, said linked page being linked to said referring link page by a hyperlink in said hypertext database, said analyzing means providing named entity candidates for at least one named entity appearing in at least one of said first text and second text, said named entity candidates being one of location names, personal names and organization names; and

a complex text analyzing means for estimating the likelihood of the named entity candidates detected by the single text analyzing means by an analysis with reference to the first text in the referring link page and the second text in the linked page.

2. A named entity discriminating system as claimed in claim 1, wherein the complex text analyzing means estimates the likelihood of the candidate named entity detected by the single text analyzing means based on:

appearance frequency information of words co-occurring with the named entity candidates in at least one of the referring link page and the linked page;

co-occurrence relations between the named entity candidates and a key character string in the referring link page of the text in which the named entity candidates appears;

or

appearance frequency information of words co-occurring with the named entity candidates in a specified range before and after a key character string in the referring link page of the text in which the named entity candidates appears.

3. A named entity discriminating system as claimed in claim 2, wherein when no co-occurring word is detected in the referring link page and/or linked page of the text in which the named entity candidates appears, the complex text analyzing means detects the co-occurring words referring to the referring link page or the linked page of the text in which no co-occurring word is detected.

4. A named entity discriminating system as claimed in claim 2, wherein the complex text analyzing means follows a link from the text in which the named entity candidates appears to referring link page or linked page, and searches prescribed link layers from the text specified as a reference range for the co-occurring word.

5. A named entity discriminating system as claimed in claim 2, wherein the complex text analyzing means estimates the likelihood of the named entity candidates based on the weight which is assigned to each co-occurring word according to the position thereof.

6. A named entity discriminating method, comprising steps of:

reading text from a hypertext database;

analyzing at least a first text from a referring link page and separately analyzing a second text from a linked page, said linked page being linked to said referring link page by a hyperlink in said hypertext database, said analyzing means providing named entity candidates for at least one named entity appearing in at least one of said first text and second text, said named entity candidates being one of location names, personal names and organization name; and

estimating the likelihood of the named entity candidates by an analysis with reference to the first text in the referring link page and the second text in the linked page.

7. A named entity discriminating method as claimed in claim 6, wherein the likelihood of the named entity candidates is estimated based on:

appearance frequency information of words co-occurring with the named entity candidates in at least one of the referring link page and the linked page;

co-occurrence relations between the named entity candidates and a key character string in the referring link page of the text in which the named entity candidates appears;

or

appearance frequency information of words co-occurring with the named entity candidates in a specified range before and after a key character string in the referring link page of the text in which the named entity candidates appears.

8. A named entity discriminating method as claimed in claim 7, wherein when no co-occurring word is detected in the referring link page and/or linked page of the text in which the named entity candidates appears, the co-occurring words are detected referring to the referring link page or the linked page of the text in which no co-occurring word is detected.

9. A named entity discriminating method as claimed in claim 7, wherein the likelihood of the candidate named entity is estimated by following a link from the text in which the named entity candidates appears to the referring link page or the linked page, and searching prescribed link layers from the text specified as a reference range for the co-occurring word.

10. A named entity discriminating method as claimed in claim 7, wherein the likelihood of the named entity candidates is estimated based on the weight which is assigned to each co-occurring word according to the position thereof.

11. A storage medium for storing a computer program which, when executed on a computer, causes the computer to execute processes including:

a reading process for reading text from a hypertext database;

a single text analyzing process for analyzing at least a first text from a referring link page and separately analyzing a second text from a linked page, said linked page being linked to said referring link page by a hyperlink in said hypertext database, said analyzing process providing named entity candidates for at least one named entity appearing in at least one of said first text and second text, said named entity candidates being one of location names, personal names and organization names; and

a complex text analyzing process for estimating the likelihood of the named entity candidates detected in the single text analyzing process by an analysis referring to the referring link page and the linked page of the text in which the named entity candidates appears.

12. The storage medium for storing a computer program as claimed in claim 11, wherein the complex text analyzing process estimates the likelihood of the named entity candidates detected by the single text analyzing process based on:

appearance frequency information of words co-occurring with the named entity candidates in the referring link page and the linked page;

co-occurrence relation between the named entity candidates and a key character string in the referring link page of the text in which the named entity candidates appears;

or

appearance frequency information of words co-occurring with the named entity candidates in a specified range before and after a key character string in the referring link page of the text in which the named entity candidates appears.

13. The storage medium for storing a computer program as claimed in claim 12, wherein when no co-occurring word is detected in the referring link page and/or the linked page of the text in which the named entity candidates appears, the complex text analyzing process detects the co-occurring words referring to the referring link page or the linked text of the text in which no co-occurring word is detected.

14. The storage medium for storing a computer program as claimed in claim 12, wherein the complex text analyzing process follows a link from the text in which the named entity candidates appears to the referring link page or the linked page, and searches prescribed link layers from the text specified as a reference range for the co-occurring word.

15. The storage medium for storing a computer program as claimed in claim 12, wherein the complex text analyzing process estimates the likelihood of the named entity candidates based on the weight which is assigned to each co-occurring word according to the position thereof.