Synchronization protocol for multi-premises hosting of digital content items

• US 9,479,567 B1
• Filed: 12/22/2015
• Issued: 10/25/2016
• Est. Priority Date: 10/29/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Rude, Christopher, Koorapati, Nipunn, von Muhlen, Marcio, Bunger, Nils
• Primary Examiner(s): KATSIKIS, KOSTAS J
• Application Number:
  US14/979,234
• Time in Patent Office: 308 Days
• Field of Search: 709/203, 709/217, 709/204, 707/610, 707/612, 707/622, 707/626, 707/633
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 47/196 Integration of transport layer protocols, e.g. TCP and UDP
• CPC Class Codes: H04L 61/457 containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A synchronization protocol for multi-premises hosting of digital content items. In one embodiments, for example, a method, comprising: detecting creation or modification of a content item; determining a content item namespace to which the content item belongs; obtaining an identifier of a block server to which the content item namespace is assigned; and uploading any content item blocks of the content item to the block server that are not already stored at the block server.

Claims

1. A method, comprising:

at a computing device comprising one or more processors and storage media storing one or more computer programs executed by the one or more processors to perform the method, performing operations comprising;

detecting creation or modification of a content item;

determining a content item namespace to which the content item belongs;

obtaining an identifier of a block server to which the content item namespace is assigned;

uploading any content item blocks of the content item to the block server that are not already stored at the block server;

determining a current client journal cursor value, the current client journal cursor value representing how up-to-date the content item namespace mounted at the computing device is with respect to any newer content item changes in the content item namespace that are committed to an online content management service and uploaded to the block server to which the content item namespace is assigned;

providing, to a server, the current client journal cursor value in association with an identifier of the content item namespace and an identifier of the block server to which the content item namespace is assigned;

receiving, from the server, a list of one or more server journal entries, each server journal entry in the list identifying one or more content item blocks of a content item represented by the server journal entry, each server journal entry in the list having a server journal cursor value that is greater than the current client journal cursor value of the computing device; and

for each server journal entry in the list of one or more server journal entries, downloading, from the block server, any content item blocks of the content item represented by the server journal entry that are not already stored at the computing device.

2. The method of claim 1, wherein the operation of detecting the content item comprises the operation of receiving a notification from an operating system of the computing device when a file in a file system corresponding to the content item is added or modified.

3. The method of claim 1, wherein the operation of determining the content item namespace to which the new content item belongs is based on a file system path to the content item in a file system of the computing device.

4. The method of claim 1, wherein the operation of obtaining the block server identifier comprises the operation of accessing a mapping stored at the computing device that associates each identifier of one or more identifiers of one or more content item namespaces mounted at the computing device with an identifier of a block server to which the identified content item namespace is assigned.

5. The method of claim 1, wherein the operations further comprise:

determining a first list of one or more content item block hashes of one or more content item blocks that constitute the content item;

providing the first list to a server;

obtaining a second list from the server, the second list identifying one or more content item block hashes in the first list corresponding to one or more content item blocks that are not already stored at the block server; and

uploading the one or more content item blocks that are not already stored at the block server.

6. The method of claim 1, wherein:

the content item is a first content item;

the block server is a first block server;

the content item namespace is a first content item namespace;

the operations further comprise;

detecting creation or modification of a second content item that is not the first content item;

determining a second content item namespace, that is not the first content item namespace, to which the second content item belongs;

obtaining an identifier of a second block server, that is not the first block server, to which the second content item namespace is assigned; and

uploading any content item blocks of the second content item that are not already stored at the second block server.

7. The method of claim 1, the operations further comprising resolving the identifier of the block server to a network address of the block server.

8. The method of claim 1, wherein the block server is connected to the same local area network as the computing device.

9. The method of claim 1, wherein the block server is operated by an online content management service.

10. A computing device, comprising:

one or more processors; and

storage media storing one or more computer programs comprising instructions configured for;

detecting creation or modification of a content item;

determining a content item namespace to which the content item belongs;

obtaining an identifier of a block server to which the content item namespace is assigned;

uploading any content item blocks of the content item to the block server that are not already stored at the block server;

determining a current client journal cursor value, the current client journal cursor value representing how up-to-date the content item namespace mounted at the computing device is with respect to any newer content item changes in the content item namespace that are committed to an online content management service and uploaded to the block server to which the content item namespace is assigned;

providing, to a server, the current client journal cursor value in association with an identifier of the content item namespace and an identifier of the block server to which the content item namespace is assigned;

receiving, from the server, a list of one or more server journal entries, each server journal entry in the list identifying one or more content item blocks of a content item represented by the server journal entry, each server journal entry in the list having a server journal cursor value that is greater than the current client journal cursor value of the computing device; and

for each server journal entry in the list of one or more server journal entries, downloading, from the block server, any content item blocks of the content item represented by the server journal entry that are not already stored at the computing device.

11. The computing device of claim 10, wherein the block server is operated by an online content management service.

12. The computing device of claim 10, wherein the instructions configured for detecting the content item comprise instructions configured for receiving a notification from an operating system of the computing device when a file in a file system corresponding to the content item is added or modified.

13. The computing device of claim 10, wherein the instructions configured for determining the content item namespace to which the new content item belongs is based on a file system path to the content item in a file system of the computing device.

14. The computing device of claim 10, wherein the instructions configured for obtaining the block server identifier comprise instructions configured for accessing a mapping stored at the computing device that associates each identifier of one or more identifiers of one or more content item namespaces mounted at the computing device with an identifier of a block server to which the identified content item namespace is assigned.

15. The computing device of claim 10, wherein the instructions are further configured for:

determining a first list of one or more content item block hashes of one or more content item blocks that constitute the content item;

providing the first list to a server;

obtaining a second list from the server, the second list identifying one or more content item block hashes in the first list corresponding to one or more content item blocks that are not already stored at the block server; and

uploading the one or more content item blocks that are not already stored at the block server.

16. The computing device of claim 10, wherein:

the content item is a first content item;

the block server is a first block server;

the content item namespace is a first content item namespace;

the instructions are further configured for;

detecting creation or modification of a second content item that is not the first content item;

determining a second content item namespace, that is not the first content item namespace, to which the second content item belongs;

obtaining an identifier of a second block server, that is not the first block server, to which the second content item namespace is assigned; and

uploading any content item blocks of the second content item that are not already stored at the second block server.

17. The computing device of claim 10, the instructions are further configured for resolving the identifier of the block server to a network address of the block server.

18. The computing device of claim 10, wherein the block server is connected to the same local area network as the computing device.