LEARNING DEVICE AND LEARNING METHOD

• US 20180285347A1
• Filed: 03/13/2018
• Published: 10/04/2018
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Mizobuchi, Yuji
• Granted Patent: US 10,747,955 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 40/242 Dictionaries
• CPC Class Codes: G06F 40/247 Thesauruses; Synonyms
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/289 Phrasal analysis, e.g. finite state techniques or chunking
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

A learning device includes a memory and a processor configured to execute a process including acquiring a plurality of documents, clustering the plurality of documents with respect to each of a first plurality of words, the first plurality of words being included in the plurality of documents, assigning a common label to a first word and a second word among the first plurality of words in a case where a cluster relating to the first word and a cluster relating to the second word resemble each other, and re-clustering, on the basis of the common label, the plurality of documents including the first word and the second word after the assigning the common label.

Claims

1. A learning device comprising:

a memory; and

a processor coupled to the memory and the processor configured to execute a process including,

acquiring a plurality of documents,

clustering the plurality of documents with respect to each of a first plurality of words, the first plurality of words being included in the plurality of documents,

assigning a common label to a first word and a second word among the first plurality of words in a case where a cluster relating to the first word and a cluster relating to the second word resemble each other, and

re-clustering, on the basis of the common label, the plurality of documents including the first word and the second word after the assigning the common label.

2. The learning device according to claim 1, wherein it is determined that the cluster relating to the first word and the cluster relating to the second word resemble each other when a distance between a center of gravity of the cluster relating to the first word and a center of gravity of the cluster relating to the second word is less than a first threshold value, or when a difference between distribution of the cluster relating to the first word and distribution of the cluster relating to the second word is less than a second threshold value.

3. The learning device according to claim 2, wherein the first threshold value is determined based on a distance between centers of gravity of respective clusters relating to a second plurality of words, the second plurality of words having meanings that resemble each other, among the first plurality of words, or the second threshold value is determined based on a difference in distribution of respective clusters relating to the second plurality of words.

4. The learning device according to claim 1, the process further including, suppressing, even in a case where a cluster relating to the third word and a cluster relating to the fourth word resemble each other, assignment of a common label to a third word and a fourth word among the first plurality of words when it is determined that a number of samples of the cluster relating to the third word or the cluster relating to the fourth word among the first plurality of words, or a combination thereof is no less than a third threshold value, or when it is determined that a difference between density of samples of the cluster relating to the third word and density of samples of the cluster relating to the fourth word is no less than a fourth threshold value.

5. The learning device according to claim 1, the process further including,

referring to a word meaning dictionary that stores a meaning of a word, and

assigning a common label to a third word and a fourth word among the first plurality of words when it is detected that the third word and the fourth word are synonyms.

6. The learning device according to claim 1, the process further including,

extracting the first plurality of words from the plurality of documents so as to generate a word dictionary, and

generating a context representing whether or not each of the plurality of documents includes each of the first plurality of words included in the word dictionary, and

wherein the clustering is processing of clustering the context.

7. The learning device according to claim 6, wherein the generating a context is performed on an assumption that a plurality of words assigned with a common label are identical words.

8. The learning device according to claim 1, wherein the assigning a common label includes assigning a first label to the first word included in documents constituting a first cluster and assigning a second label to the first word included in documents constituting a second cluster in a case where the cluster relating to the first word includes the first cluster and the second cluster, and assigning the first label to the second word in a case where the cluster relating to the second word resembles the first cluster.

10. The learning method according to claim 1, wherein it is determined that the cluster relating to the first word and the cluster relating to the second word resemble each other when a distance between a center of gravity of the cluster relating to the first word and a center of gravity of the cluster relating to the second word is less than a first threshold value, or when a difference between distribution of the cluster relating to the first word and distribution of the cluster relating to the second word is less than a second threshold value.

11. The learning method according to claim 10, wherein the first threshold value is determined based on a distance between centers of gravity of respective clusters relating to a second plurality of words, the second plurality of words having meanings that resemble each other, among the first plurality of words, or the second threshold value is determined based on a difference in distribution of respective clusters relating to the second plurality of words.

12. The learning method according to claim 1, further comprising:

suppressing, even in a case where a cluster relating to the third word and a cluster relating to the fourth word resemble each other, assignment of a common label to a third word and a fourth word among the first plurality of words when it is determined that a number of samples of the cluster relating to the third word or the cluster relating to the fourth word among the first plurality of words, or a combination thereof is no less than a third threshold value, or when it is determined that a difference between density of samples of the cluster relating to the third word and density of samples of the cluster relating to the fourth word is no less than a fourth threshold value.

9. A learning method executed by a computer, the learning method comprising:

acquiring a plurality of documents;

clustering the plurality of documents with respect to each of a first plurality of words, the first plurality of words being included in the plurality of documents;

assigning a common label to a first word and a second word among the first plurality of words in a case where a cluster relating to the first word and a cluster relating to the second word resemble each other; and

re-clustering, on the basis of the common label, the plurality of documents including the first word and the second word after the assigning the common label.

13. The learning method according to claim 9, further comprising:

referring to a word meaning dictionary that stores a meaning of a word; and

assigning a common label to a third word and a fourth word among the first plurality of words when it is detected that the third word and the fourth word are synonyms.

14. The learning method according to claim 9, further comprising:

extracting the first plurality of words from the plurality of documents so as to generate a word dictionary; and

generating a context representing whether or not each of the plurality of documents includes each of the first plurality of words included in the word dictionary, and

wherein the clustering is processing of clustering the context.

15. The learning method according to claim 14, wherein the generating a context is performed on an assumption that a plurality of words assigned with a common label are identical words.

16. The learning method according to claim 9, wherein the assigning a common label includes assigning a first label to the first word included in documents constituting a first cluster and assigning a second label to the first word included in documents constituting a second cluster in a case where the cluster relating to the first word includes the first cluster and the second cluster, and assigning the first label to the second word in a case where the cluster relating to the second word resembles the first cluster.

17. A non-transitory computer-readable medium storing a learning program that causes a computer to execute a process comprising:

acquiring a plurality of documents;

clustering the plurality of documents with respect to each of a first plurality of words, the first plurality of words being included in the plurality of documents;

assigning a common label to a first word and a second word among the first plurality of words in a case where a cluster relating to the first word and a cluster relating to the second word resemble each other; and

re-clustering, on the basis of the common label, the plurality of documents including the first word and the second word after the assigning the common label.