DOCUMENT SIMILARITY CALCULATION DEVICE

• US 20120330955A1
• Filed: 05/15/2012
• Published: 12/27/2012
• Est. Priority Date: 06/27/2011
• Status: Abandoned Application

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: MIURA, Mitsugu
• Application Number:
  US13/472,414
• Publication Number:
  US 20120330955A1
• Time in Patent Office: Days
• US Class Current: 707/737
• CPC Class Codes: G06F 16/3347 using vector based model

Abstract

A document similarity calculation device, configured to calculate a similarity indicating a degree of how much a plurality of documents are similar, includes: an associative word group storage portion for storing an associative word group composed of words associated with one another, a word-in-document frequency matrix generation portion for generating a matrix of word frequency in document which is a matrix each element of which is the frequency of a word present in a document with respect to each combination of the word and the document, a word-in-document frequency matrix transformation portion for transforming the generated matrix of word frequency in document based on the stored associative word group so as to reduce the number of dimensions of the matrix of word frequency in document, and a similarity calculation portion for calculating the similarity based on the transformed matrix of word frequency in document.

Claims

1. A document similarity calculation device for calculating a similarity indicating a degree of how much a plurality of documents are similar to one another, the document similarity calculation device comprising:

a unit of storing associative word group for storing an associative word group composed of words associated with one another;

a unit of generating matrix of word frequency in document for generating a matrix of word frequency in document which is a matrix each element of which is the frequency of a word present in a document with respect to each combination of the word and the document;

a unit of transforming matrix of word frequency in document for transforming the generated matrix of word frequency in document based on the stored associative word group so as to reduce the number of dimensions of the matrix of word frequency in document; and

a unit of calculating similarity for calculating the similarity based on the transformed matrix of word frequency in document.

2. The document similarity calculation device according to Claim 1, wherein the unit of transforming matrix of word frequency in document is configured to transform the matrix of word frequency in document by replacing the row composed of the elements with respect to every word included in the stored associative word group with a row each element of which is the sum of the elements with respect to every word included in the associative word group respectively.

3. The document similarity calculation device according to Claim 1 further comprising a unit of extracting associative word group for extracting an associative word group based on the transformed matrix of word frequency in document, wherein the unit of storing associative word group is configured to store the extracted associative word group.

4. The document similarity calculation device according to Claim 3, wherein the unit of extracting associative word group is configured to extract the associative word group by decomposing a singular value of the transformed matrix of word frequency in document.

5. The document similarity calculation device according to Claim 1, wherein the unit of calculating similarity is configured to calculate the similarity based on the generated matrix of word frequency in document if the number of documents as the base of generating the matrix of word frequency in document is smaller than a preset threshold value.

6. The document similarity calculation device according to Claim 1 further comprising:

a unit of accepting search word for accepting a search word inputted by a user;

a unit of extracting associative document for extracting an associative document associated with the accepted search word;

a unit of extracting similar document for extracting a similar document analogous to the extracted associative document based on the calculated similarity; and

a unit of outputting search result for outputting information for identifying the extracted associative document and the extracted similar document.

7. A document similarity calculation method for calculating a similarity indicating a degree of how much a plurality of documents are similar to one another, the document similarity calculation method comprising:

prestoring an associative word group composed of words associated with one another;

generating a matrix of word frequency in document which is a matrix each element of which is the frequency of a word present in a document with respect to each combination of the word and the document;

transforming the generated matrix of word frequency in document based on the stored associative word group so as to reduce the number of dimensions of the matrix of word frequency in document; and

calculating the similarity based on the transformed matrix of word frequency in document.

8. The document similarity calculation method according to Claim 7, the method comprising:

transforming the matrix of word frequency in document by replacing the row composed of the elements with respect to every word included in the stored associative word group with a row each element of which is the sum of the elements with respect to every word included in the associative word group respectively.

9. A medium being readable by an information processing device and storing a document similarity calculation program comprising instructions for causing the information processing device to carry out a process for calculating a similarity indicating a degree of how much a plurality of documents are similar to one another, the process comprising:

prestoring an associative word group composed of words associated with one another;

generating a matrix of word frequency in document which is a matrix each element of which is the frequency of a word present in a document with respect to each combination of the word and the document;

transforming the generated matrix of word frequency in document based on the stored associative word group so as to reduce the number of dimensions of the matrix of word frequency in document; and

calculating the similarity based on the transformed matrix of word frequency in document.

10. The medium according to Claim 9, wherein the process is configured to transform the matrix of word frequency in document by replacing the row composed of the elements with respect to every word included in the stored associative word group with a row each element of which is the sum of the elements with respect to every word included in the associative word group respectively.