Index replication using crawl modification information

• US 20070208716A1
• Filed: 02/23/2007
• Published: 09/06/2007
• Est. Priority Date: 03/01/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Krishnaprasad, Muralidhar, Bhavsar, Meeten, Chang, Thomas H., Delgado, Joaquin A., Yang, Chi-Ming
• Granted Patent: US 7,945,533 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Systems, methodologies, media, and other embodiments associated with index replication using crawl modification information are described. One exemplary system embodiment includes an enterprise search system comprising a target search system comprising an index logic that uses modified crawl information related to items associated with sources to maintain an index that supports searching of the items; and, a crawl search system comprising a pipeline processor configured to receive modified crawl information related to the items and to propagate the modified crawl information to the target system.

Claims

1. An enterprise search system, comprising:

a target search system comprising an index logic that uses modified crawl information related to items associated with sources to maintain an index that supports searching of the items; and

,

a crawl search system comprising a pipeline processor configured to receive modified crawl information related to the items and to propagate the modified crawl information to the target system.

2. The enterprise search system of claim 1, where the index functionally replicates a primary index of the crawl search system.

3. The enterprise search system of claim 2, where the index and the primary index employ different storage mechanisms.

4. The enterprise search system of claim 1, further comprising a plurality of target search systems, where the index of each target system independently functionally replicates a primary index of the crawl search system.

5. The enterprise search system of claim 1, further comprising a plurality of target search systems, where the indexes of the target systems collectively functionally replicate a primary index of the crawl search system.

6. The enterprise search system of claim 1, where the items comprise at least one of documents, files, web pages, emails spread sheets and/or databases.

7. The enterprise search system of claim 1, where the modified crawl information comprising at least one of modified content, metadata and/or security information.

8. The enterprise search system of claim 1, further comprising a crawler logic configured to access the items to determine a modification to the items, the crawler logic further configured to provide modified crawl information to the pipeline processor.

9. An enterprise search system, comprising:

a plurality of target search systems, each target search system comprising an index logic that uses modified crawl information related to items associated with sources to maintain an index that supports searching of the items;

a crawl search system comprising;

a pipeline processor configured to receive modified crawl information related to the items and to propagate the modified crawl information to the plurality of target systems; and

,

a crawler logic configured to access the items to determine a modification to the items, the crawler logic further configured to provide modified crawl information to the pipeline processor.

10. The enterprise search system of claim 9, where each of the plurality of target search systems is configured to process only a particular type of item.

11. The enterprise search system of claim 9, where each of the plurality of target search systems is configured to process only items associated with a particular source.

12. The enterprise search system of claim 9, where each of the plurality of target search systems is configured to process only items associated with one or more entities.

13. The enterprise search system of claim 9, where the index of each target system independently functionally replicates a primary index of the crawl search system.

14. The enterprise search system of claim 9, where the indexes of the target systems collectively functionally replicate a primary index of the crawl search system.

15. A distributed enterprise crawl system, comprising:

a first search system and a second search system, each search system comprising;

an index logic that uses modified crawl information related to items associated with sources to maintain an index that supports searching of the items;

a pipeline processor configured to receive modified crawl information related to the items and to propagate the modified crawl information to the other search system; and

,

a crawler logic configured to access the items to determine a modification to the items, the crawler logic further configured to provide modified crawl information to the pipeline processor and to the index of the particular search system,

where the first search system is configured to access items associated with a first source and the second search system is configured to access items associated with a second source.

16. The distributed enterprise crawl system of claim 15, where the index of each search system independently functionally replicates the index of the other search system.

17. A method for replicating search information, comprising:

crawling items associate with sources;

identifying changes to the items;

broadcasting information regarding the identified changes to a plurality of target search systems; and

,

independently updating an index associated with each target search system based on the identified changes.

18. The method of claim 17, where each of the plurality of target search systems is configured to process only a particular type of item.

19. The method of claim 17, where each of the plurality of target search systems is configured to process only items associated with a particular source.

20. The method of claim 17, where each of the plurality of target search systems is configured to process only items associated with one or more entities.

21. The method of claim 17, where the index of each target system independently functionally replicates a primary index of a crawl search system.

22. The method of claim 17, where the indexes of the target systems collectively functionally replicate a primary index of a crawl search system.

23. A computer-readable medium for providing processor executable instructions that when executed cause a computer to perform a method for replicating search information in a computing system comprising a plurality of search systems and a plurality of crawlers, the method comprising:

distributing crawling assignments of a plurality of information sources between the plurality of crawlers;

crawling and identifying, by each of the crawlers, changes to items from one or more assigned information sources;

broadcasting, by each of the crawlers, the identified changes to the plurality of search systems; and

independently updating, by each of the search systems, an index to the plurality of information sources using the identified changes broadcasted from each of the crawlers.