Apparatus, method, and computer program product for extracting structured document

• US 8,037,403 B2
• Filed: 01/11/2007
• Issued: 10/11/2011
• Est. Priority Date: 01/13/2006
• Status: Active Grant

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation), Toshiba Digital Solutions Corporation (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Inaba, Masumi, Kawamura, Takahiro, Nagano, Shinichi, Hasegawa, Tetsuo
• Primary Examiner(s): Hong; Stephen
• Assistant Examiner(s): NAZAR, AHAMED I
• Application Number:
  US11/622,216
• Publication Number:
  US 20070179937A1
• Time in Patent Office: 1,734 Days
• Field of Search: 715/234, 715/241, 715/254, 707/999.003, 707/999.006, 707/E17.095, 707/E17.122
• US Class Current: 715/234
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/80 of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML content-based retrieval of web data G06F16/95
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

An apparatus for retrieving a structured document including a first specifying unit that specifies a plurality of object documents from a plurality of structured documents being accessible via a network, the object document being the structured document according to retrieval condition; a first extracting unit that extracts text included in the object document; a second extracting unit that extracts metadata appended to the object document, the metadata being first data concerning the text of the object document and second data indicating a link relation between the object document and the structured documents; and a first calculating unit that calculates importance of each of the object documents, based on the text and the metadata of each of the object documents.

Claims

1. An apparatus for retrieving a structured document comprising:

a processor that is programmed to retrieve the structured document, wherein the processor causes;

a first specifying unit to specify a plurality of object documents from a plurality of structured documents being accessible via a network, the object document being the structured document according to retrieval condition,

a first extracting unit to extract a text included in the object document,

a second extracting unit to extract a metadata appended to the object document, the metadata being first data indicating the text of the object document and second data indicating a link relation between the object document and related documents, each of the related documents being the structured document associated with the object document,

a second specifying unit to specify whether a description supporting the object document is included in the text of the each related document,

an analyzing unit to analyze whether author information of a related document is included in a metadata appended to the related document based on the metadata appended to the object document, and

a first calculating unit to calculate higher importance for the object document related to the related document having the author information thereof included in the metadata compared with important of the object document related to the related document not having the author information thereof included in the metadata; and

further to calculate higher importance for the object document corresponding to the related document including the description supporting the object document compared with importance of the object document corresponding to the related document not including a description supporting the object document.

2. The apparatus according to claim 1, wherein the second data indicates a track-back.

3. The apparatus according to claim 1, wherein the metadata includes, as first data indicates content of a site or an article of the structured document.

4. The apparatus according to claim 1, wherein the processor further causes:

a third specifying unit to specify a date of creation of a related document included in a metadata appended to the related document wherein

the second specifying unit specifies a data of creation of the object document included in the metadata appended to the object document, and

the first calculating unit calculates importance of the object document based on duration between the date of creation of the object document and the date of creation of the related document.

5. The apparatus according to claim 4, wherein

the third specifying unit specifies a date of creation of each related document, and

the first calculating unit calculates higher importance for the object document for which number of the related documents created within a predetermined period from the creation date of the object document is equal to or larger than a defined number compared with importance of the object document for which the number of the related documents created within the predetermined period from the creation date of the object document lower than the defined number.

6. The apparatus according to claim 1, wherein the processor further causes a counting unit to count number of opinions on the text of the object document included in the metadata appended to the object document, the opinions being opinions of different authors, wherein

the first calculating unit calculates importance of the object data based on the number of opinions counted by the counting unit.

7. The apparatus according to claim 1, wherein the second specifying unit specifies a date of writing of an opinion on the text of the object document included in the metadata, and

the first calculating unit calculates importance of the object document based on the date of writing specified by the second specifying unit.

8. The apparatus according to claim 1, wherein the first calculating unit calculates lower importance for the object document not having information indicating a link relation with other structured document included in the metadata compared with importance of the object document having information indicating a link relation with other structured documents included in the metadata.

9. The apparatus according to claim 1, further comprising:

a storage unit that stores therein author information of a first author and a second author when, in the metadata of a first object document of the first author among the object documents, the first object document is associated with a plurality of second object documents of the second author among the object documents and, in the metadata of a second structured document, the second object document is associated with a plurality of first object documents, wherein

the second specifying unit specifies author information indicating an author of the structured document from the metadata appended to the structured document, and

the first calculating unit calculates higher importance for the object document of the author indicated in the author information stored in the storage unit compared with importance of the object document of an author other than the author indicated in the author information.

10. The apparatus according to claim 1, further comprising:

a storage unit that stores author information of an author of the structured documents when the number of the structured documents specified by the second specifying unit is equal to or larger than a defined number set in advance, wherein

the second specifying unit specifies the number of structured documents, the number of related documents of which associated in the metadata is equal to or larger than a defined value set in advance, the structured documents being written by same author, and

the first calculating unit calculates higher importance for the object document of the author indicated in the author information compared to importance of the object documents of authors other than the author indicated in the author information held by the storing unit.

11. The apparatus according to claim 1, further comprising:

a first storage unit that stores therein a plurality of contents related to each other in association with respective attributes; and

a second storage unit that stores therein, when there are structured documents with content described in the text thereof being associated with an identical attribute in the first storage unit and the number of such structured documents written by same author is equal to or larger than a defined number set in advance, author information of the author of the structured documents in association with the attribute,

wherein the processor further causes

a retrieving unit to retrieve, based on the metadata, structured documents written by an identical author out of the structured documents, wherein

the second specifying unit specifies content described in the text, and

the first calculating unit calculates higher importance for the object document of the author associated with an attribute of a retrieval object in the second storage unit compared with importance of the object documents of authors other than the author indicated in the author information.

12. The apparatus according to claim 1, further comprising:

a storage unit that stores therein, when the number of structured documents specified by the second specifying unit is equal to or larger than a defined number set in advance, author information of the structured documents, wherein

the second specifying unit specifies structured documents that do not include data indicating a link relation with other structured documents in the metadata appended to the structured document among the structured documents and are written by an identical author, and

the first calculating unit calculates lower importance for the object documents of the author indicated in the author information held by the storage unit compared with importance of the object documents of authors other than the author indicated in the author information.

13. The apparatus according to claim 1, wherein

the second specifying unit specifies a plurality of opinions on the text of the structured document from the metadata and specifies opinions with identical content and opinions with content opposite to the identical content out of specified opinions, and

the processor further causes a second calculating unit to calculate higher importance, when a predetermined percentage of a plurality of opinions on the text of the object document include certain content, for opinions opposite to the certain content importance compared with importance of the certain content.

14. The apparatus according to claim 1, further comprising:

a storage unit that stores therein, when a first object document of a first author among the object documents is associated with a plurality of second object documents of a second author among the object documents in the metadata of the first object document and the second object documents is associated with a plurality of first object documents in the metadata of the second object document, the author information of the first author and the second author, wherein

the second specifying unit specifies author information indicating an author of the structured document from the metadata, the processor further causes;

a third specifying unit to specify a plurality of opinions on the text of the object documents from the metadata and to specify authors of the respective opinions, and

a second calculating unit that calculates higher importance for opinions of the authors corresponding to the author information held by the storage unit among the opinions included in the text of the object documents compared with importance of opinions of authors other than the authors corresponding to the author information.

15. The apparatus according to claim 1, further comprising:

a storage unit that stores therein, when data indicating a link relation with the other structured documents is not included in the metadata and the number of structured documents with identical author information is equal to or larger than a defined number set in advance, the author information, wherein

the second specifying unit specifies author information indicating an author of the structured document from the metadata, and further specifies author information indicating an author who has written each of a plurality of opinions on the text from the metadata of the object document, and

the processor further causes a second calculating unit to calculate lower importance for opinions of the authors indicated in the author information stored in the storage unit among the opinions on the text compared with importance of opinions of authors other than the authors indicated in the author information.

18. The apparatus according claim 1, wherein the second specifying unit specifies, when a plurality of related documents are associated with the object document in the metadata, whether the description supporting the object document is included, on the basis of an affirmative expression included in the text of the each related document;

and

the first calculating unit calculates higher important for the object document for which the number of the related documents containing the description supporting the object document is equal to or larger than a defined number set compared with the object document for which the number of the related documents is less than the defined number set.

19. The apparatus according to claim 1, wherein the first calculating unit calculates higher importance for a second object document compared with importance of a first object document when,

in the metadata of the first object document among the object documents, a plurality of structured documents including the second object document among the object documents are associated with the first object document,

a plurality of structured documents are associated with the second object document in the metadata of the second object document, and

the number of the structure documents associated with the second object document is large compared with the number of the structured documents associated with the first object document.

16. A method of retrieving a structured document that is accessible via a network, the method comprising:

specifying a plurality of object documents from a plurality of structured documents, the object document being the structured document according to retrieval condition,

extracting a text included in the object document,

extracting a metadata appended to the object document, the metadata being first data indicating the text of the object document and second data indicating a link relation between the object document and related documents, each of the related documents being the structured document associated with the object document,

specifying whether a description supporting the object document is included in the text of the each related document,

analyzing whether author information of a related document is included in a metadata appended to the related document based on the metadata appended to the object document, and

calculating higher importance for the object document related to the related document having the author information thereof included in the metadata compared with importance of the object document related to the related document not having the author information thereof included in the metadata, and further calculating higher importance for the object document corresponding to the related document including the description supporting the object document compared with importance of the object document corresponding to the related document not including a description supporting the object document.

17. A computer program product that is executable by a computer and has a computer-readable recording medium including a plurality of commands for retrieving a structured document, wherein the commands cause the computer to execute:

specifying a plurality of object documents from a plurality of structured documents, the object document being the structured document according to retrieval condition,

extracting a text included in the object document,

extracting a metadata appended to the object document, the metadata being first data indicating the text of the object document and second data indicating a link relation between the object document and related documents, each of the related documents being the structured document associated with the object document,

specifying whether a description supporting the object documents is included in the text of the each related document,

analyzing whether author information of a related document is include in a metadata appended to the related document based on the metadata appended to the object document, and

calculating higher importance for the object document related to the related document having the author information thereof included in the metadata compared with importance of the object document related to the related document not having the author information thereof included in the metadata, and further calculating higher importance for the object document corresponding to the related document including the description supporting the object document compared with importance of the object document corresponding to the related document not including a description supporting the object document.