Document searching apparatus, method thereof, and record medium thereof

• US 7,225,181 B2
• Filed: 01/24/2001
• Issued: 05/29/2007
• Est. Priority Date: 02/04/2000
• Status: Expired due to Fees

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Tsuda, Hiroshi
• Primary Examiner(s): Corrielus; Jean M.
• Assistant Examiner(s): To; Baoquoc N.
• Application Number:
  US09/768,062
• Publication Number:
  US 20010020238A1
• Time in Patent Office: 2,316 Days
• Field of Search: 707/3, 707/4, 707/5, 707/10
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A document searching apparatus for searching a document group having a link relation for particular document is disclosed, that comprises a link importance assigning unit weighting the link relation and assigning the link importance which indicates importance of the document based on the weighted link relation to each document, and an accessing unit accessing the particular document based on the link importance. Thus, important document can be automatically searched.

Claims

1. A document searching apparatus for searching a document group having a link relation for a document, comprising:

a link importance assigning unit weighting the link relation and assigning link importance which indicates importance of the document based on the weighted link relation to the document; and

an accessing unit accessing the document based on the link importance,

wherein said link importance assigning unit includes;

a URL similarity calculating unit calculating a URL similarity that is a text similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs,

wherein said link importance assigning unit calculates the link importance based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases.

2. The document searching apparatus as set forth in claim 1, further comprising:

a keyword extracting unit extracting text parts from the documents and extracting a keyword from the document contents.

3. The document searching apparatus as set forth in claim 2,

wherein said keyword extracting unit calculates an occurrence frequency of the keyword in the document, and

wherein said keyword extracting unit further comprises:

a keyword-document correlation calculating unit calculating the correlation of the keyword and the document based on the link importance and the occurrence frequency of the keyword.

4. The document searching apparatus as set forth in claim 3, further comprising:

a monitoring unit monitoring accesses from a user and generating an access log, and

wherein said keyword-document correlation calculating unit calculates the correlation based on the keyword occurrence frequency, the link importance, and the access log.

5. The document searching apparatus as set forth in claim 3, further comprising:

a document type determining unit determining a document type of the document based on the URL similarity, the number of links from the document, and the number of links to the document,

wherein said keyword-document correlation calculating unit selects the document based on the document type and calculates the correlation for the selected document.

6. The document searching apparatus as set forth in claim 3, further comprising:

an index creating unit creating an index for accessing the document corresponding to pronunciation characters or spelling of the extracted keyword.

7. The document searching apparatus as set forth in claim 6, further comprising:

a selecting unit allowing the user to select a portion of the pronunciation characters or spelling of the keyword,

wherein said index creating unit places less than a predetermined number of documents highly correlated with the keyword in the index based on the correlation calculated by said keyword-document correlation calculating unit, and

wherein said accessing unit accesses the document based on the selected keyword.

8. The document searching apparatus as set forth in claim 1, further comprising:

a collecting unit collecting the document from a network.

9. The document searching apparatus as set forth in claim 1,

wherein said link importance assigning unit causes the weight of the link relation between the documents with a high URL similarity to be decreased.

10. The document searching apparatus as set forth in claim 1,

wherein said link importance assigning unit causes the document that is linked from an important document and whose URL similarity is low to be important.

11. The document searching apparatus as set forth in claim 1,

wherein said link importance assigning unit causes the importance of a document linked from many documents whose URL similarity are high to be decreased.

13. The document searching apparatus as set forth in claim 1,

wherein the URL similarity is determined based on characters of a URL containing a server address.

12. A document searching apparatus for searching a document group having a link relation for a document, comprising:

a link importance assigning unit weighting the link relation and assigning link importance which indicates importance of the document based on the weighted link relation to the document; and

an accessing unit accessing the document based on the link importance, and

wherein said link importance assigning unit includes;

a URL similarity calculating unit calculating a URL similarity that is a similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs,

wherein said link importance assigning unit calculates the link importance based on the URL similarity and the link relation of the document,

wherein the link importance of each document is defined as a solution of the following simultaneous linear equation (1), assuming that C_q is constant (the lower limit of the importance that depends on each page) for each p∈

DOC and that when a page p is linked to a page q, the link weight lw(p, q) is defined by the formula (2);

$\begin{array}{cc}\mathrm{Wq}=\mathrm{Cq}+\underset{p\in <br><br>\mathrm{Refed}<br><br><br><br>\left(q\right)}{\sum <br><br>}<br><br>\mathrm{Wp}*\mathrm{lw}<br><br><br><br>\left(p,q\right)& \left(1\right)\\ \mathrm{lw}<br><br>\left(p,q\right)=\mathrm{diff}<br><br>\left(p,q\right)/\underset{i\in <br><br>\mathrm{Ref}<br><br>\left(p\right)}{\sum <br><br>}<br><br>\mathrm{diff}<br><br>\left(p,i\right)=\frac{1}{\mathrm{sim}<br><br>\left(p,q\right)<br><br>\underset{i\in <br><br>\mathrm{Ref}<br><br>\left(p\right)}{\sum <br><br>}<br><br>\frac{1}{\mathrm{sim}<br><br>\left(p,i\right)}}& \left(2\right)\end{array}$

where DOC={p1, p2, . . . , pN} is a set of documents calculated for the link importance;

Wp is the link importance of the page p;

Ref(p) is a set of pages linked from the page p;

Refed(p) is a set of pages linking to the page p;

sim(p, q) is the URL similarity of the pages p and q;

diff(p, q)=1/sim(p, q) is the difference.

14. A document index creating apparatus for creating an index of a document group having a link relation, comprising:

a link importance assigning unit assigning a link importance to the document based on the link relation;

a keyword extracting unit extracting a keyword from the document;

an index creating unit creating an index for accessing the keyword based on pronunciation characters or spelling of the extracted keyword; and

an accessing unit accessing document assigned the link importance corresponding to the keyword when the pronunciation characters or spelling of the keyword are selected from the index,

wherein said link importance assigning unit includes;

a URL similarity calculating unit calculating a URL similarity that is a text similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs,

wherein said link importance assigning unit calculates the link importance based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases.

15. A document index creating apparatus for creating an index of a document group having a link relation, comprising:

a link importance assigning unit assigning a link importance to the document depending on whether or not URLs of the documents are similar;

a keyword extracting unit extracting a keyword from the document; and

an index creating unit creating an index for accessing the document corresponding to pronunciation characters or spelling of the extracted keyword based link importance,

wherein said link importance assigning unit includes;

a URL similarity calculating unit calculating a URL similarity that is a text similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs,

wherein said link importance assigning unit calculates the link importance based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases.

16. A link list creating system for creating a link list for a document group having a link relation, comprising:

a collecting unit collecting the documents from a network;

a link importance assigning unit assigning a link importance of the document as an importance calculated based on the link relation to the document;

a URL charcter string determining unit determining a URL having a particular charcterisitic of a charcter string from the documents;

an index creating unit creating a link list for listing less than a predetermined number of links to the documents based on the link importance and the particular characterisitic of the character string of the URL; and

a document type determining unit determining a document type based on a URL similarity represnting a text similarity between character strings of URLs of the documents and being an appearance of written characters of URLs, the number of links to the document, and the number of links from the documents,

wherein said index creating unit selects the document based on the document type and creates a link list of the selected document, and

wherein said link importance assigning unit calculates the link importance based on an inverse URL similarity and the link relation of the document, so that a link wieght increase as URL similarity decreases.

17. A document searching method for searching a document group having a link relation for a document, comprising:

assigning a link importance as an importance of the document calculated with weighting the link relation to the document, comprising;

calculating a URL similarity that is a text similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs;

calculating the link importance based on the URL similarity and the link relation of the document with said link importance being based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases; and

accessing the document based on the link importance.

18. The document searching method as set forth in claim 17, further comprising:

extracting a keyword from the document.

19. The document searching method as set forth in claim 17, further comprising:

calculating an occurrence frequency of the keyword in the document, and

calculating the correlation of the keyword and the document based on the link importance and the occurrence frequency of the keyword.

20. The document searching method as set forth in claim 19, further comprising:

monitoring accesses from a user and generating an access log; and

calculating the correlation based on the keyword occurrence frequency, the link importance, and the access log.

21. The document searching method as set forth in claim 19, further comprising:

determining a document type of the document based on the URL similarity, the number of links to the document, and the number of links from the document; and

selecting the document based on the document type and calculating the correlation of the selected document.

22. The document searching method as set forth in claim 18, further comprising:

creating an index for accessing the document corresponding to pronunciation characters or spelling of the extracted keyword.

23. The document searching method as set forth in claim 17, further comprising:

placing less than a predetermined number of documents which are correlated with the keyword in the index;

selecting a portion of the pronunciation characters or spelling of the keyword; and

accessing the document corresponding to the selected portion of the pronunciation characters or spelling of the selected keyword.

24. The document searching method as set forth in claim 17, further comprising:

collecting the document from a network.

25. A link list creating method for creating a link list for a document group having a link relation, comprising:

collecting the document from a network;

assigning a link importance which indicates importance of the document to the document based on the link relation;

determining a URL having a particular characteristic of a character string from the URLs of each document;

creating a link list for listing less than a predetermined number of links to the document based on the link importance and the particular characteristic of the character string of the URL

determining a document type based on a URL similarity that is a text similarity between character strings of URLs (Uniform Resource Locators) of the documents and that is an appearance of written characters of URLs, the number of links to the document, and the number of links from the document, and

selecting the document based on the document type, and

wherein the creating creates the link list for the selected document based on the link importance and the particular characteristic of the character string of the URL, and

wherein said link importance being based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases.

26. A computer readable record medium for recording a program that causes a computer to execute a process for creating a link list for a document group having a link relation, the program comprising:

collecting documents from a network;

assigning a link importance which indicates importance of the document to each document based on the link relation, including;

calculating a URL similarity that is a text similarity of character strings of URLs that represent the location of the documents and that is an appearance of written characters of URLs,

calculating the link importance based on the URL similarity and the link relation of the document with said link importance being based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases;

determining a URL having a particular characteristic of a character string from the URLs of documents; and

creating a link list for listing less than a predetermined number of links to the documents based on the link importance and the particular characteristic of the character string of the URL.

27. A document searching apparatus for searching a document group having a link relation for a document, comprising:

a link importance assigning unit weighting the link relation and assigning link importance which indicates importance of the document based on the weighted link relation to the document, said link importance assigning unit comprising a similarity calculating unit calculating a URL similarity that is a similarity URLs that represent the location of the documents and that is an appearance of written characters of URLs and said link importance assigning unit calculates the link importance based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases; and

an accessing unit accessing the document based on the link importance.

28. A document searching method for searching a document group having a link relation for a document, comprising:

assigning a link importance as an importance of the document calculated with weighting the link relation to the document, comprising;

calculating a similarity that is a similarity of URLs that represent the location of the documents;

calculating the link importance based on an inverse URL similarity and the link relation of the document, so that a link weight increases as URL similarity decreases; and

accessing the document based on the link importance.