Document retrieval system and document retrieval method

• US 8,046,368 B2
• Filed: 02/12/2008
• Issued: 10/25/2011
• Est. Priority Date: 04/27/2007
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Ohi, Hiroko, Niwa, Yoshiki, Obara, Kiyohiro
• Primary Examiner(s): HICKS, MICHAEL J
• Application Number:
  US12/029,694
• Publication Number:
  US 20080270386A1
• Time in Patent Office: 1,351 Days
• Field of Search: None
• US Class Current: 707/741
• CPC Class Codes: G06F 16/3347 using vector based model

Abstract

A document retrieval is performed with similarities between documents in numeric data taken into consideration. To this end, generated is a set E of intervals in which each element of a set D of numeric values representing a feature A is included in any one of the intervals. Each numeric value in each document is indexed by assigning, with 1, an interval including an element x of the set D, and with 0, an interval without the element x. Each document data including numeric values is indexed by indexing its text part with term frequencies, and by indexing its numeric-value part with the above-described numeric value indexing scheme. By use of indices thus created for each of the document data, similarities between the document data are calculated using a vector space model or a probability model, and the document data are presented in order of similarity.

Claims

1. A document retrieval system, comprising:

a document database for storing data for a plurality of documents;

an arithmetic unit that;

includes a numeric value data reading unit configured to read, from the data on the documents stored in the document database, numeric value data for which numeric value intervals are to be generated;

calculates indices used for indexing numeric values and texts in each of the documents stored in the document database, each of the indices used for indexing the text being a group of a term constituting the text and a frequency of the term in the document, each of the indices used for indexing the numeric value being a group of a label describing a feature represented by the numeric value, an interval including the numeric value, and a frequency of the numeric value in the document;

receives a designation of a document as a retrieval input; and

computes a similarity between the designated document and each of the documents stored in the document database by use of the indices; and

a numeric value distribution percentage designating unit configured to designate the numeric value intervals from distribution percentages of numeric values based on a distribution of the numeric value data, wherein

the arithmetic unit includes a numeric value range designating unit configured to generate the numeric value intervals based on distribution percentages inputted by the numeric value distribution percentage designating unit; and

the numeric value distribution percentage designating means generates numeric value intervals whose numeric value widths are so adjusted that each of the numeric value intervals includes an equal number of numeric values.

2. The document retrieval system according to claim 1, wherein the interval constitutes a set E of intervals in which each element in a set D of numeric values representing a feature A is included in at least any one of the intervals.

3. The document retrieval system according to claim 2, wherein the set E of the intervals includes a group of intervals partially overlapping one another.

4. The document retrieval system according to claim 1, further comprising:

a numeric value extracting processing unit;

a numeric conversion table for converting a numeric value into an interval, the numeric conversion table is dedicated to each feature type; and

an index creating unit,

wherein the numeric value extracting processing unit extracts a group of a feature and a numeric value from each of the documents stored in the document database, and

the index creating unit indexes the extracted group of the feature and the numeric value in accordance with the numeric conversion table.

5. The document retrieval system according to claim 4, further comprising:

a term-to-numeric-value conversion table for associating a group of a feature and a term with a group of the feature and the term'"'"'s corresponding numeric value or interval,

wherein the numeric value extracting processing unit extracts a group of a feature and a term from the document, and converts the extracted group of the feature and the term into a group of the feature and the term'"'"'s corresponding numeric value or interval in accordance with the term-to-numeric-value conversion table, and

in accordance with the numeric conversion table, the index creating unit indexes the group of the feature and the term which has been converted into the group of the feature and the numeric value or interval in accordance with the term-to-numeric-value conversion table.

6. A document retrieval method comprising:

receiving a designation of a document as a retrieval input;

reading from data on documents stored in the document database, numeric value data for which numeric value intervals are to be generated;

calculating a similarity between the document designated as the retrieval input and each of the documents stored in the document database by use of indices of the designated document and indices of each document stored in the document database, the indices used for indexing numeric values and texts in a corresponding document, each of the indices used for indexing the text being a group of a term constituting the text and a frequency of the term in the corresponding document, each of the indices used for indexing the numeric value being a group of a label describing a feature represented by the numeric value, an interval including the numeric value, and a frequency of the numeric value in the corresponding document;

presenting the documents stored in the document database in order of similarity;

deciding the numeric value intervals from distribution percentages of numeric values based on a distribution of the numeric value data; and

generating the numeric value intervals based on distribution percentages inputted by the numeric value distribution percentage designating unit;

wherein the generating the numeric value intervals generates numeric value intervals whose numeric value widths are so adjusted that each of the numeric value intervals includes an equal number of numeric values.

7. The document retrieval method according to claim 6, wherein the interval constitutes a set E of intervals in which each element in a set D of numeric values representing a feature A is included in at least any one of the intervals.

8. The document retrieval method according to claim 7, wherein the set E of the intervals includes a group of intervals partially overlapping one another.

9. The document retrieval method according to claim 6, further comprising:

extracting a group of a feature and a numeric value from each of the documents stored in the document database; and

indexing the extracted group of the feature and the numeric value in accordance with a numeric conversion table for converting a numeric value into an interval, the numeric conversion table is dedicated to each feature type.

10. The document retrieval method according to claim 9, further comprising:

extracting a group of a feature and a term from the document;

converting the extracted group of the feature and the term into a group of the feature and the term'"'"'s corresponding numeric value or interval in accordance with a term-to-numeric-value conversion table for associating a group of a feature and a term with a group of the feature and the term'"'"'s corresponding numeric value or interval; and

in accordance with the numeric conversion table, indexing the group of the feature and the term which has been converted into the group of the feature and the numeric value or interval in a preceding operation.

11. A document retrieval method comprising:

extracting a group of a feature and a numeric value from each of a plurality of document data stored in a document database, to obtain numeric value data for which numeric value intervals are to be generated;

converting the extracted numeric value into an interval in accordance with a numeric conversion table, and then indexing the extracted numeric value with a group of the feature, the interval and a frequency, the numeric conversion table being dedicated to each feature type, and used for converting an numeric value into an interval;

indexing each text in the document with a group of a term constituting the text and a frequency of the term in the document;

calculating a similarity between document data designated as a retrieval input and each of the documents stored in the document database by use of data on the document indexed as above;

presenting the document data stored in the document database in order of similarity;

deciding the numeric value intervals from distribution percentages of numeric values based on a distribution of the numeric value data;

generating the numeric value intervals based on distribution percentages inputted by the numeric value distribution percentage designating unit; and

wherein the generating the numeric value intervals generates numeric value intervals whose numeric value widths are so adjusted that each of the numeric value intervals includes an equal number of numeric values.

12. The document retrieval method according to claim 11, wherein the interval constitutes a set E of intervals in which each element in a set D of numeric values representing a feature A is included in at least any one of the intervals.

13. The document retrieval method according to claim 11, further comprising:

extracting a group of a feature and a term from a document;

converting the extracted group of the feature and the term into a group of the feature and the term'"'"'s corresponding numeric value or interval in accordance with a term-to-numeric-value conversion table for associating a group of a feature and a term with a group of the feature and the term'"'"'s corresponding numeric value or interval; and

in accordance with the numeric conversion table, indexing the group of the feature and the term which has been converted into the group of the feature and the numeric value or interval in the preceding step.