Method for effective indexing of partially dynamic documents

• US 20020138467A1
• Filed: 05/22/2002
• Published: 09/26/2002
• Est. Priority Date: 09/23/1997
• Status: Active Grant

• Current Assignee: Bampton Technologies, LLC (Intellectual Ventures LLC)
• Original Assignee: AT&T Corporation (AT&T, Inc.)
• Inventors: Krishnamurthy, Balachander, Jacobson, Guy, Srivastava, Divesh
• Granted Patent: US 6,606,639 B2
• Time in Patent Office: Days
• US Class Current: 707/1
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A method more efficiently indexes dynamic documents. The method adjusts the frequency with which dynamic documents are retrieved taking into account the extent to which the document varies between its most recent retrievals. Furthermore, the method selects portions of the document to be indexed based on the substance of the differences between recently retrieved copies.

Claims

1. A method for adjusting a frequency for indexing a dynamic document, comprising:

retrieving a first copy of a document;

allowing a defined time interval to elapse, said defined time interval being arranged to separate consecutive operations of retrieving a copy of a document to determine whether to index said document;

retrieving a second copy of said document after said defined time interval has elapsed;

characterizing a significance of a change between said first and second copies; and

adjusting said defined time interval depending on said significance.

2. The method of claim 1, wherein said significance depends on a number of differences between said first and second copies.

3. The method of claim 1, wherein said significance depends on a type of a content of said change.

4. The method of claim 1, wherein said significance depends on a sensitivity of said document.

5. The method of claim 1, wherein said adjusting comprises decreasing said defined time interval if said significance exceeds a predetermined threshold.

6. The method of claim 1, wherein said adjusting comprises increasing said defined time interval if said significance is less than a predetermined threshold.

7. The method of claim 1, wherein said adjusting reduces said defined time interval to a lower threshold value below which said defined time interval is not reduced.

8. A method for efficiently indexing dynamic documents, comprising:

(i) retrieving a copy of a document;

(ii) allowing a defined time interval to elapse;

(iii) retrieving another copy of said document following said defined time interval;

(iv) repeating (i)-(iii) an arbitrary number of times;

(v) comparing a most recently retrieved copy and an earlier copy retrieved at least two of said defined time intervals earlier; and

(vi) indexing said most recently retrieved copy based on a significance of a difference detected, by said comparing, between said most recently retrieved copy and said earlier copy.

9. The method of claim 8, further comprising adjusting said defined interval depending on said significance.

10. A method for efficiently indexing a changing document, comprising:

detecting at least one of a rate of change and a significance of a change to said document; and

adjusting a frequency of indexing of said document in accordance with said detecting.

11. The method of claim 10, wherein said frequency of indexing is increased if said rate of change increases.

12. The method of claim 10, wherein said frequency of indexing is increased if said significance exceeds a predetermined threshold.

13. The method of claim 10, wherein said frequency of indexing is decreased if said rate of change decreases.

14. The method of claim 10, wherein said frequency of indexing is decreased if said significance falls below a predetermined threshold.

16. The system of claim 15, wherein said frequency of indexing is increased if said rate of change increases.

17. The system of claim 15, wherein said frequency of indexing is increased if said significance exceeds a predetermined threshold.

18. The system of claim 15, wherein said detecting comprises:

retrieving a first copy of said document;

allowing a defined time interval to elapse, said defined time interval being arranged to separate consecutive operations of retrieving a copy of said document to determine whether to index said document;

retrieving a second copy of said document after said defined time interval has elapsed; and

characterizing a significance of a change between said first and second copies.

19. The system of claim 18, wherein said significance depends on a number of differences between said first and second copies.

20. The system of claim 18, wherein said significance depends on a type of a content of said change.

15. A system comprising:

an indexing server linked to at least one other server comprising at least one partially dynamic document, wherein said indexing server is configured to implement a process for efficiently indexing said partially dynamic document, said process comprising;

detecting at least one of a rate of change and a significance of a change to said partially dynamic document; and

adjusting a frequency of indexing of said partially dynamic document in accordance with said detecting.