System and method for secure full-text indexing

• US 20060200446A1
• Filed: 03/03/2005
• Published: 09/07/2006
• Est. Priority Date: 03/03/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Samsonov, Yevgeniy A., Peltonen, Kyle G., Gursky, Dennis
• Granted Patent: US 7,392,253 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Documents are indexed in accordance with a process that separates the filtering and word breaking portions of the process so that they are performed under a restricted security setting. When a document is requested by an indexer, the document is retrieved and then passed to the higher security process. The document is then filtered one or more filters and tokenized by one or more wordbreakers under the restricted security setting before being passed to the indexer. The restricted security setting prevents security vulnerabilities from being exploited during the filtering and word breaking processes.

Claims

1. A computer-implemented method for secure full-text indexing of a document, comprising:

receiving a document identifier from an indexer;

retrieving a document corresponding to the document identifier;

processing the document under a restricted security setting; and

forwarding the processed document to the indexer.

2. The computer-implemented method of claim 1, further comprising cross-checking the document identifier with the indexer before retrieving the document.

3. The computer-implemented method of claim 2, wherein cross-checking the document identifier further comprises comparing the document identifier received to the document identifier sent by the indexer.

4. The computer-implemented method of claim 2, wherein cross-checking the processing of the document under the restricted security setting is temporarily shut down when the cross-check of the document identifier fails.

5. The computer-implemented method of claim 1, further comprising intermittently shutting down processing under the restricted security setting such that a window of opportunity for breaching security is limited.

6. The computer-implemented method of claim 1, wherein processing the document under a restricted security setting further comprises processing the document using a filter.

7. The computer-implemented method of claim 1, wherein processing the document under a restricted security setting further comprises processing the document using a wordbreaker.

8. The computer-implemented method of claim 1, wherein document identifiers are sent by the indexer and documents are retrieved in batches.

9. The computer-implemented method of claim 1, wherein the document is retrieved according to a second process with a second security setting that is different from the restricted security setting.

10. The computer-implemented method of claim 9, wherein the indexer is operated under a third security setting that is different from the second security setting and the restricted security setting.

11. The computer-implemented method of claim 1, wherein the document is retrieved by a protocol handler.

12. The computer-implemented method of claim 11, wherein the protocol handler receives the document request directly from the indexer.

13. The computer-implemented method of claim 11, wherein the protocol handler receives the document request after it is propagated through a process corresponding to the restricted security setting.

14. A system for secure full-text indexing of a document, comprising:

an indexer that is arranged to build an index, where entries in the index correspond to words of the document;

a protocol handler that is arranged to retrieve a document from a data source upon a document request received from the indexer; and

a restricted process that is arranged to process the document under a restricted security setting and forward the processed document to the indexer.

15. The system of claim 14, wherein the restricted process further comprises a filter that converts the document to pure text.

16. The system of claim 15, wherein the restricted process further comprises a wordbreaker that tokenizes the pure text into words.

17. The system of claim 14, wherein the restricted process is further arranged to be intermittently shut down to prevent hijacking of the restricted process.

18. The system of claim 14, wherein the protocol handler cross-checks the documented identifier with the indexer before retrieving the document from the data source.

19. A computer-readable medium containing computer-executable instructions for secure full-text indexing of a document, comprising:

receiving a document identifier from an indexer;

cross-checking the document identifier with the indexer;

retrieving a document corresponding to the document identifier;

processing the document under a restricted security setting;

forwarding the processed document to the indexer; and

intermittently shutting down processing under the restricted security setting such that a window of opportunity for breaching security is limited.

20. The computer-readable medium of claim 19, wherein processing the document under a restricted security setting further comprises processing the document using at least one of a filter and a wordbreaker.