Crawling secure data sources

• US 8,601,028 B2
• Filed: 06/28/2012
• Issued: 12/03/2013
• Est. Priority Date: 03/01/2006
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Liao, Ciya, Chang, Thomas, Krishnaprasad, Muralidhar, Bhavsar, Meeten
• Primary Examiner(s): Woo, Isaac M
• Application Number:
  US13/536,488
• Publication Number:
  US 20120272304A1
• Time in Patent Office: 523 Days
• Field of Search: 707600-899
• US Class Current: 707/791
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/0815 providing single-sign-on or federations
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/102 Entity profiles

Abstract

It is desirable to provide a secure search mechanism to provide for searching over any and all content, such as across an enterprise. A secure search, however, requires access to the secure content repositories holding the data to be searched. In some cases the credentials required to crawl a repository may be extremely sensitive, or the user may be reluctant or unwilling to store user identification information in memory or on disk for any longer than is absolutely necessary. An approach is provided that allows a user or an administrator to provide security credentials to be stored and used only during a crawl, and to erase the credentials from the system when the crawl is complete.

Claims

1. A method of crawling secure documents in an enterprise system, comprising:

specifying in metadata for a secure data source an indication that a temporary storage option is selected for the secure data source, the temporary storage option causing security credentials to be stored temporarily;

examining, by a processor, the metadata for the secure data source at crawl time to determine that the temporary storage option is selected for the secure data source;

prompting for and receiving security credentials;

in response to determining that the temporary storage option is selected, writing, by a processor, the security credentials to a temporary storage;

reading the security credentials from the temporary storage;

crawling, by a processor, a set of documents provided by the secure data source, wherein each document of the set of documents is associated with a set of security attributes;

wherein crawling further comprises presenting the security credentials to the secure data source;

indexing, by a processor, one or more documents of the set of documents, wherein each document of the one or more documents is associated with a set of security attributes that allow access to said each document based on the security credentials; and

deleting, by a processor, the security credentials in response to completing use of the security credentials.

2. The method of claim 1, wherein the security credentials comprise a password.

3. The method of claim 1, further comprising obtaining security attributes for each of the secure data sources.

4. The method of claim 3, wherein the obtaining of the security attributes for each of the secure data sources includes obtaining the security attributes from a federation broker.

5. The method of claim 3, further comprising pushing the security attributes into a text index.

6. The method of claim 1, wherein the temporary storage option is set by an administrator.

7. The method of claim 1, wherein deleting the security credentials further comprises a callback at the end of the crawl.

8. The method of claim 1, further comprising deleting the security credentials in response to an interruption of the crawl process or at system restart.

9. The method of claim 1, wherein multiple crawls are initiated, and further comprising retaining the security credentials until no longer needed for any of the multiple crawls.

10. A non-transitory computer-readable storage medium storing instructions, the instructions which when executed by one or more processors, cause the one or more processors to crawl secure documents in an enterprise system, the instructions comprising instructions for:

specifying in metadata for a secure data source an indication that a temporary storage option is selected for the secure data source, the temporary storage option causing security credentials to be stored temporarily;

examining the metadata for the secure data source at crawl time to determine that the temporary storage option is selected for the secure data source;

prompting for and receiving security credentials;

in response to determining that the temporary storage option is selected, writing the security credentials to a temporary storage;

reading the security credentials from the temporary storage;

crawling a set of documents provided by the secure data source, wherein each document of the set of documents is associated with a set of security attributes;

wherein crawling further comprises presenting the security credentials to the secure data source;

indexing one or more documents of the set of documents, each document of the one or more documents associated with a set of security attributes that allow access to the one or more documents based on the security credentials; and

deleting the security credentials in response to completing use of the security credentials.

11. The non-transitory computer-readable storage medium of claim 10, wherein the security credentials comprise a password.

12. The non-transitory computer-readable storage medium of claim 10, the instructions further comprising instructions for obtaining security attributes for each of the secure data sources.

13. The non-transitory computer-readable storage medium of claim 12, wherein the instructions for obtaining of the security attributes for each of the secure data sources includes instructions for obtaining the security attributes from a federation broker.

14. The non-transitory computer-readable storage medium of claim 12, the instructions further comprising instructions for pushing the security attributes into a text index.

15. The non-transitory computer-readable storage medium of claim 10, wherein the temporary storage option is set by an administrator.

16. The non-transitory computer-readable storage medium of claim 10, wherein deleting the security credentials further comprises a callback at the end of the crawl.

17. The non-transitory computer-readable storage medium of claim 10, the instructions further comprising for deleting the security credentials in response to an interruption of the crawl process or at system restart.

18. The non-transitory computer-readable storage medium of claim 10, wherein multiple crawls are initiated, and the instructions further comprising instructions for retaining the security credentials until no longer needed for any of the multiple crawls.