Multi-user search system with methodology for bypassing instant indexing

• US 10,579,609 B2
• Filed: 06/30/2017
• Issued: 03/03/2020
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Goel, Samir, Chastagnol, Franck, Agrawal, Abhishek
• Primary Examiner(s): Patel, Manglesh M
• Assistant Examiner(s): Hasty, Nicholas
• Application Number:
  US15/640,121
• Publication Number:
  US 20170300520A1
• Time in Patent Office: 977 Days
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2329 using versioning
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/24564 Applying rules; Deductive queries
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/328 Management therefor
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/101 Access control lists [ACL]

Abstract

Multi-user computer search system with methodology for bypassing instant indexing of documents. In some embodiments, for example, a system for bypassing instant indexing includes a token store storing a set of token for a current version of a document and a tokenizer server configured to tokenize a new version of the document and to generate a set of tokens for the new version of the document. The system further includes an instant indexer server configured to determine tokens to index the document by based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document, to determine whether the new version of the document is a bypass document version or non-bypass document version, and to generate an index mutation for the new version of the document including a specification of whether the generated index mutation is a bypass index mutation or a non-bypass index mutation based on whether the new version of the document is a bypass document version or non-bypass document version. The system also includes an index mutation server configured to provide the generated index mutation to an index server, if the generated index mutation is a non-bypass index mutation, or not provide the generated index mutation to the index server, if the generated index mutation is a bypass index mutation.

Claims

1. A computing system comprising:

one or more processors;

storage media;

a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords;

an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier; and

instructions stored in the storage media for execution by the one or more processors and configured for;

detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location;

based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.

2. The computing system of claim 1, further comprising instructions stored in the storage media for execution by the one or more processors and configured for:

detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;

wherein the file system file is not contained in the file system folder;

wherein the distributed index indexes a pre-update version of the document; and

sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.

3. The computing system of claim 1, wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.

4. The computing system of claim 1, wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.

5. The computing system of claim 1, wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.

6. The computing system of claim 1, wherein the bypassing the instant indexing mechanism includes:

(a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.

7. The computing system of claim 6, further comprising instructions stored in the storage media for execution by the one or more processors and configured for:

storing the respective index mutation in the index mutation journal for each new respective version of the set of documents with a bypass flag set in the respective index mutation; and

determining not to send the respective index mutation to the plurality of index servers for each new respective version of the set of documents based on the bypass flag set in the respective index mutation.

8. A method performed by a computing system comprising one or more processors, the computing system operatively coupled to a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords, the computing system coupled to an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier, the method comprising:

detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location; and

based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.

9. The method of claim 8, further comprising:

detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;

wherein the file system file is not contained in the file system folder;

wherein the distributed index indexes a pre-update version of the document; and

sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.

10. The method of claim 8, wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.

11. The method of claim 8, wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.

12. The method of claim 8, wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.

13. The method of claim 8 wherein the bypassing the instant indexing mechanism includes:

(a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.

14. The method of claim 13, further comprising:

storing the respective index mutation in the index mutation journal for each new respective version of the set of documents with a bypass flag set in the respective index mutation; and

determining not to send the respective index mutation to the plurality of index servers for each new respective version of the set of documents based on the bypass flag set in the respective index mutation.

15. One or more non-transitory computer-readable media storing instructions for execution by a computing system comprising one or more processors, the computing system operatively coupled to a plurality of index servers comprising a distributed index, the distributed index configured to index documents by keywords, the computing system coupled to an instant indexing mechanism configured to update the distributed index by sending index mutations to the plurality of index servers when new versions of documents are stored with a content management service, wherein a new version of a document is identified by the content management service by a respective document version identifier, the instructions, when executed by the computing system, are capable of causing the computing system to perform:

detecting that a file system folder, containing a set of file system files corresponding to a set of documents stored with the content management service, is moved from a first file system location to a second file system location; and

based, at least in part, on the detecting that the file system folder is moved, storing a new respective version of each document in the set of documents with the content management service, and bypassing the instant indexing mechanism for each new respective version of the set of documents stored with the content management service, the bypassing the instant indexing mechanism including not sending a respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.

16. The one or more non-transitory computer-readable media of claim 15, wherein the instructions, when executed by the computing system, are further capable of causing the computing system to perform:

detecting that a file system file, corresponding to a document stored with the content management service, is updated at a computing device;

wherein the file system file is not contained in the file system folder;

wherein the distributed index indexes a pre-update version of the document; and

sending, to an index server of the plurality of index servers, an index mutation for a post-update version of the document, the index mutation reflecting one or more differences between the pre-update version of the document and the post-update version of the document.

17. The one or more non-transitory computer-readable media of claim 15, wherein each new respective version of the set of documents corresponds to deleting a respective file system file, of the set of file system files, from the first file system location.

18. The one or more non-transitory computer-readable media of claim 15, wherein each new respective version of the set of documents corresponds to adding a respective file system file, of the set of file system files, at the second file system location.

19. The one or more non-transitory computer-readable media of claim 15, wherein the bypassing the instant indexing mechanism includes not storing, in an index mutation journal, the respective index mutation for each new respective version of the set of documents stored with the content management service.

20. The one or more non-transitory computer-readable media of claim 15, wherein the bypassing the instant indexing mechanism includes:

(a) storing, in an index mutation journal, the respective index mutation for each new version of the set of documents stored with the content management service, but does not include (b) sending the respective index mutation to the plurality of index servers for each new respective version of the set of documents stored with the content management service.