Data processing device, data processing method, and data processing program

• US 8,775,158 B2
• Filed: 08/02/2006
• Issued: 07/08/2014
• Est. Priority Date: 08/04/2005
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Sakao, Yousuke, Ikeda, Takahiro, Satoh, Kenji
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US11/996,484
• Publication Number:
  US 20100063795A1
• Time in Patent Office: 2,897 Days
• Field of Search: 704/9, 704/8, 704/7, 704/4, 704/275, 704/272, 704/270.1, 704/2, 704/10, 704/1, 717/146, 715/234, 709/219, 706/50
• US Class Current: 704/9
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

[PROBLEMS] To provide a data processing device such as a text mining device capable of extracting characteristic structures properly even in case a plurality of words indicating identical contents or a plurality of words semantically associated are contained in input data. [MEANS FOR SOLVING PROBLEMS] Association node extraction unit (22) of a text mining device (10) extracts association nodes containing semantically associated words from a graph obtained as a result of syntax analysis. Association node joint unit (23) transforms the graph by joint of a part of or a whole of the association nodes. Characteristic structure extraction unit (24) extracts a characteristic structure from the graph transformed by the association node joint unit.

Claims

1. A data processing device generating a graph which expresses an input data structure by a plurality of nodes having a single word as content thereof and by a dependency branch connecting two nodes in a dependent relationship within the plurality of nodes, and extracting a characteristic structure characterizing the input data from the graph, the device comprising:

an association node extraction unit for extracting nodes semantically associated with each other, which are nodes corresponding to words representing same or similar content, from each of sentence structures in a given sentence structure collection, and outputting information on the sentence collection and association nodes in each of the sentence structures;

an association node joint unit for joining the nodes semantically associated with each other in each of the sentence structures by a semantic association branch based on the information on the sentence structure collection to newly generate a structure that expresses a concept which is not present, but implied, in each original sentence structure, and the association nodes in each of the sentence structures received from the association node extraction unit so as to transform each of the sentence structures in the sentence structure collection, and outputting a structure collection obtained by the transformation; and

a characteristic structure extraction unit for extracting a characteristic partial structure based on the sentence structure collection transformed by joining the semantic association branch received from the association node joint unit,

wherein the characteristic structure extraction unit performs characteristic structure extraction processing by distinguishing a branch indicating a dependent relationship in the graph structure from the semantic association branch.

2. The data processing device, as claimed in claim 1, wherein

the association node joint unit has a function of categorizing the association nodes into strong association nodes which are in a strong semantic association and weak association nodes which are in a weak semantic association, and a function of joining the strong association nodes into one node.

3. The data processing device, as claimed in claim 2, wherein

the association node joint unit has a function of connecting the weak association nodes by a semantic association branch, and

the characteristic structure extraction unit has a function in which a partial structure of the graph is not extracted as the characteristic structure when the partial structure includes nodes connected by the semantic association branch, in addition, at least one of those nodes is not connected with another node by the dependency branch.

4. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has an anaphoric nodes extracting function of extracting anaphoric nodes including a pronoun or a zero pronoun, and an antecedent in an anaphoric relationship, as the association node, and

the association node joint unit has a function of node categorization for categorizing the anaphoric nodes as the strong association node.

5. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has a function of same surface node extraction for extracting same surface nodes including words in the same surface as the association nodes, and

the association node joint unit has a function of node categorization for categorizing the same surface nodes as the weak association nodes.

6. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has a function of synonymous node extraction for extracting synonymous nodes including words regarded as synonyms in a thesaurus as the association nodes, and

the association node joint unit has a function of node categorization for categorizing the synonymous nodes into the weak association nodes.

7. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has a function of designated synonymous node extraction for extracting designated synonymous nodes including synonyms designated by a user as the association nodes, and

the association node joint unit has a function of node categorization for categorizing the designated synonymous nodes as the weak association nodes.

8. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has a function of related word node extraction for extracting related word nodes including words regarded as related words in a thesaurus as the association nodes,

the association node joint unit has a function of node categorization for categorizing the related word nodes into the weak association nodes.

9. The data processing device, as claimed in claim 3, wherein

the association node extraction unit has a function of designated related word node extraction for extracting designated related word nodes including related words designated by a user as the association nodes,

the association node joint unit has a function of node categorization for categorizing the designated related word nodes as the weak association nodes.

10. The data processing device, as claimed in claim 2, comprising:

a semantic association level calculation unit for calculating a semantic association level which indicates strength of semantic association between words included in the association nodes;

wherein

the association node joint unit categorizes the association nodes into the strong association nodes and the weak association nodes based on the semantic association level.

11. The data processing device, as claimed in claim 10, wherein

the association node joint unit categorizes the association nodes, of which the semantic association level is less than a first threshold, as the weak association nodes, and categorizes the association nodes, of which the semantic association level is equal to the first threshold or more, into the strong association node.

12. The data processing device, as claimed in claim 11, wherein

the association node joint unit does not join the association nodes of which the semantic association level is less than a second threshold.

13. A data processing means for generating a graph which expresses an input data structure by a plurality of nodes having a single word as content thereof and by a dependency branch connecting two nodes in a dependent relationship within the plurality of nodes, and extracting a characteristic structure characterizing the input data from the graph, the means comprising:

an association node extraction means for extracting nodes semantically associated with each other, which are nodes corresponding to words representing same or similar content, from each of sentence structures in a given sentence structure collection, and outputting information on the sentence structure collection and association nodes in each of the sentence structures;

an association node joint means for joining the nodes semantically associated with each other in each of the sentence structures by a semantic association branch based on the information on the sentence structure collection and the association nodes in each of the sentence structures received from the association node extraction means so as to transform each of the sentence structures in the sentence structure collection to newly generate a structure that expresses a concept which is not present, but implied, in each original sentence structure, and outputting a structure collection obtained by the transformation; and

a characteristic structure extraction means for extracting a characteristic partial structure based on the sentence structure collection transformed by joining the semantic association branch received from the association node joint means,

wherein the characteristic structure extraction means performs characteristic structure extraction processing by distinguishing a branch indicating a dependent relationship in the graph structure from the semantic association branch.

14. A data processing method generating a graph which expresses an input data structure by a plurality of nodes having a single word as content thereof and by a dependency branch which connects two nodes in a dependent relationship within the plurality of nodes, and extracting a characteristic structure characterizing the input data from the graph, the method comprising:

extracting nodes semantically associated with each other, which are nodes corresponding to words representing same or similar content, from each of sentence structures in a given sentence structure collection, and acquiring information on the sentence structure collection and association nodes in each of the sentence structures;

joining the nodes associated with each other in each of the sentence structures by a semantic association branch based on the acquired information on the sentence structure collection and the association nodes in each of the sentence structures so as to transform each of the sentence structures in the sentence structure collection to newly generate a structure that expresses a concept which is not present, but implied, in each original sentence structure, and acquiring the structure collection obtained by the transformation; and

extracting a characteristic partial structure based on the acquired sentence structure collection transformed by joining the semantic association branch; and

performing, characteristic structure extraction processing by distinguishing a branch indicating a dependent relationship in the graph structure from the semantic association branch during the characteristic structure extraction processing.

15. A non-transitory computer readable recording medium storing a data processing program which controls processing for generating a graph which expresses an input data structure by a plurality of nodes having a single word as content thereof and by a dependency branch connecting two modes in a dependent relationship within the plurality of nodes and extracting a characteristic structure characterizing the input data from the graph, the program making a computer execute the functions of:

extracting nodes semantically associated with each other, which are nodes corresponding to words representing same or similar content, from each of sentence structures in a given sentence structure collection, and acquiring information on the sentence structure collection and association nodes in each of the sentence structures;

joining the nodes semantically associated with each other in each of the sentence structures by a semantic association branch based on the acquired information on the sentence structure collection and the association nodes in each of the sentence structure so as to transform each of the sentence structures in the sentence structure collection to newly generate a structure that expresses a concept which is not present, but implied, in each original sentence structure, and acquiring the structure collection obtained by the transformation; and

extracting a characteristic partial structure based on the acquired sentence structure collection transformed by joining the semantic association branch; and

performing characteristic structure extraction processing by distinguishing a branch indicating a dependent relationship in the graph structure from the semantic association branch during the characteristic structure extraction processing.