Document management method and apparatus and document search method and apparatus

• US 20060206527A1
• Filed: 03/10/2006
• Published: 09/14/2006
• Est. Priority Date: 03/11/2005
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Hattori, Masakazu
• Granted Patent: US 7,979,438 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/33 Querying

Abstract

A document management method includes shifting a character string of characters from document data and clipping it, determining that a management Gram obtained by the clipping is one of a first Gram of low frequency and a second Gram of high frequency, storing first post data in a first post region in association with a Gram value obtained by computing the character string of first Gram, the first post data having a set of a document identification (ID) indicating the document data including the first Gram and an intra-document offset indicating a character string position thereof, and storing second post data in a second post region in association with the character string of second Gram, the second post data having a set of a document identification (ID) indicating document data including the second Gram and an intra-document offset indicating a character string position thereof.

Claims

1. A document management method for managing document data stored in a document data region of a storage unit, comprising:

shifting a character string of a given number of characters from document data and clipping the character string to generate a management Gram;

determining that the management Gram is one of a first Gram of relatively low occurrence frequency less than a threshold and a second Gram of relatively high occurrence frequency not less than the threshold;

storing first post data in a first post region of a storage unit in association with a Gram value obtained by computing the character string of the first Gram, the first post data being configured with a set of a document identification (ID) indicating the document data including the character string of the first Gram and an intra-document offset indicating a position of the character string of the first Gram; and

storing second post data in a second post region of the storage unit in association with the character string of the second Gram, the second post data being configured with a set of a document identification (ID) indicating document data including the character string of the second Gram and an intra-document offset indicating a position of the character string of the second Gram.

2. The document management method according to claim 1, wherein the determining includes determining the management Gram as the first Gram when Rk(g)<

V1 is satisfied

where V1 indicates a minimum order in order of decreasing occurrence frequency of the management Gram and Rk(g) indicates an order of the management Gram in all management Grams arranged in the order of decreasing occurrence frequency.

3. The document management method according to claim 1, wherein the determining includes determining the management Gram to be the first Gram when Oc(g)<

O ave×

V2 is satisfied,

where V2 indicates a value representing a multiple of an average occurrence frequency of Gram for calculating the occurrence frequency of the first Gram in an initial state of beginning to store the document data in the document data region, Oc(g) indicates the occurrence frequency of the management Gram, and Oave=Σ

gOc (g) indicates an average occurrence frequency of the management Gram.

4. The document management method according to claim 1, further including computing a hash value of the character string of the first Gram to obtain the Gram value.

5. The document management method according to claim 1, wherein clipping the character string includes clipping two Japanese characters from document data of a Japanese document while shifting the character string in units of one character.

6. A document retrieval method for searching document data stored in a document data region according to a retrieval key word, the method comprising:

preparing a storage unit containing a document data storage area in which document data is stored, a first post region storing first post data configured with a set of a document identification (ID) indicating document data including a character string of a first Gram and an intra-document offset indicating a position of the character string, in association with a Gram value obtained by computing the character string of the first Gram, and a second post region storing second post data configured with a set of a document identification (ID) indicating document data including a character string of a second Gram and an intra-document offset indicating a position of the character string of the second Gram, in association with the character string of the second Gram;

shifting a character string of a given number of characters from a retrieval key ward and clipping the character string to generate a retrieval Gram;

reading the first post data from the first post region by scanning the first post region according to a Gram value obtained by computing a character string of the retrieval Gram;

reading the second post data from the second post region by scanning the second post region according to the character string of the retrieval Gram; and

searching the document data region for document data matching with the retrieval key word, using the first read post data and the second post-data.

7. The document retrieval method according to claim 6, wherein the first post region stores a flag indicating one of presence and absence of the management Gram corresponding to the Gram value for every Gram value, and reading the first post data includes checking the flag in the first post region at the time of scanning the first post region and skipping a region of the first post region which fails to include the management Gram.

8. The document management method for searching document data of a Japanese document according to claim 6, wherein clipping the character string includes clipping two Japanese characters from document data of a Japanese document while shifting the character string in units of one character.

9. A document management apparatus including comprising:

a storage unit having a document data region in which document data is stored;

a determination unit configured to determine that a management Gram corresponds to one of a first Gram of relatively low occurrence frequency less than a threshold and a second Gram of relatively high occurrence frequency not less than the threshold, the management Gram being generated by shifting a character string of a given number of characters from the document data of the storage unit and clipping the character string;

a first write-in unit configured to store first post data in a first post region of the storage unit in association with a Gram value obtained by computing the character string of the first Gram, the first post data being configured with a set of a document identification (ID) indicating the document data including the character string of the first Gram and an intra-document offset indicating a position of the character string of the first Gram; and

a second write-in unit configured to store second post data in a second post region of the storage unit in association with the character string of the second Gram, the second post data being configured with a set of a document identification (ID) indicating document data including the character string of the second Gram and an intra-document offset indicating a position of the character string of the second Gram;

a generation unit configured to generate a retrieval Gram by shifting a character string of a given number of characters from a retrieval key word clipping the character string, the retrieval key word being used for searching the document data stored in the document data region of the storage unit;

a first readout unit configured to read first post data from the first post region according to a Gram value obtained by computing the character string of the retrieval Gram;

a second readout unit configured to read second post data from the second post region according to the character string of the retrieval Gram; and

a search unit configured to search the document data region for document data matching with the retrieval key word using the first post data and the second post data.

10. The document management apparatus according to claim 9, wherein the determination unit comprises a determination unit configured to determine the management Gram as the first Gram when Rk(g)<

V1 is satisfied

where V1 indicates a minimum order in order of decreasing occurrence frequency of the management Gram and Rk(g) indicates an order of the management Gram in all management Grams arranged in the order of decreasing occurrence frequency.

11. The document management method according to claim 9, wherein the determination unit comprises a determination unit configured to determine the management Gram to be the first Gram when Oc(g)<

O ave×

V2 is satisfied,

where V2 indicates a value representing a multiple of an average occurrence frequency of Gram for calculating the occurrence frequency of the first Gram in an initial state of beginning to store the document data in the document data region, Oc(g) indicates the occurrence frequency of the management Gram, and Oave=Σ

gOc (g) indicates an average occurrence frequency of the management Gram.

12. The document management apparatus according to claim 9, further including a computation unit configured to compute a hash value of the character string of the first Gram to obtain the Gram value.

13. The document management apparatus according to claim 9, wherein the determination unit includes a clipping unit configured to clip two Japanese characters from document data of a Japanese document while shifting the character string in units of one character.

14. A document search apparatus for searching document data stored in a document data region according to a retrieval key word, the apparatus comprising:

a storage unit containing a document data storage area in which document data is stored, a first post region and a second post region, wherein the first post region stores first post data configured with a set of a document identification (ID) indicating document data including a character string of a first Gram and an intra-document offset indicating a position of the character string, in association with a Gram value obtained by computing the character string of the first Gram, and the second post region stores second post data configured with a set of a document identification (ID) indicating document data including a character string of a second Gram and an intra-document offset indicating a position of the character string of the second Gram, in association with the character string of the second Gram;

a clipping unit configured to shift a character string of a given number of characters from a retrieval key ward and clip the character string to generate a retrieval Gram;

a reading unit configured to read the first post data from the first post region by scanning the first post region according to a Gram value obtained by computing a character string of the retrieval Gram;

a reading unit configured to read the second post data from the second post region by scanning the second post region according to the character string of the retrieval Gram; and

a searching unit configured to search the document data region for document data matching with the retrieval key word, using the first read post data and the second post-data.

15. The document search apparatus according to claim 14, wherein the first post region stores a flag indicating one of presence and absence of the management Gram corresponding to the Gram value for every Gram value, and reading the first post data includes checking the flag in the first post region at the time of scanning the first post region and skipping a region of the first post region which fails to include the management Gram.

16. The document search apparatus for searching document data of a Japanese document according to claim 14, wherein the clipping unit is configured to clip two Japanese characters from document data of a Japanese document while shifting the character string in units of one character.