Synchronization protocol for multi-premises hosting of digital content items

• US 10,685,038 B2
• Filed: 10/24/2016
• Issued: 06/16/2020
• Est. Priority Date: 10/29/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Koorapati, Nipunn, Rude, Christopher, von Muhlen, Marcio, Bunger, Nils
• Primary Examiner(s): Katsikis, Kostas J
• Application Number:
  US15/332,782
• Publication Number:
  US 20170126782A1
• Time in Patent Office: 1,331 Days
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 47/196 Integration of transport layer protocols, e.g. TCP and UDP
• CPC Class Codes: H04L 61/457 containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A synchronization protocol for multi-premises hosting of digital content items. In one embodiments, for example, a method, comprising: detecting creation or modification of a content item; determining a content item namespace to which the content item belongs; obtaining an identifier of a block server to which the content item namespace is assigned; and uploading any content item blocks of the content item to the block server that are not already stored at the block server.

Claims

1. A method, comprising:

detecting creation or modification of a content item;

obtaining a first set of identifiers of content item blocks of the created or modified content item;

determining a content item namespace to which the content item belongs;

sending, to a metadata server of a content management service, a user account identifier;

based, at least in part, on sending the user account identifier, obtaining from the metadata server, data representing assignments of the content item namespace to a plurality of candidate content item block servers;

selecting, from among the plurality of candidate content item block servers to which the content item namespace is assigned, a target content item block server;

wherein the selecting the target content item block server is based, at least in part, on determining that an identifier of the content item namespace is associated with an identifier of the target content item block server in the data obtained from the metadata server representing the assignments of the content item namespace to the plurality of candidate content item block servers;

sending the first set of identifiers to a metadata server;

based, at least in part, on the selecting the target content item block server and the sending the first set of identifiers to the metadata server, receiving a second set of one or more identifiers from the metadata server, the second set of one or more identifiers identifying one or more content item blocks of the content item that are not stored at the target content item block server;

uploading, to the target content item block server, the identified one or more content item blocks of the content item that are not stored at the target content item block server; and

wherein the method is performed by a computing system having one or more processors and storage media storing one or more programs, the one or more programs including instructions configured to perform the method, the instructions executed by the one or more processors to perform the method.

2. The method of claim 1, wherein:

the content item is a first content item;

the target content item block server is a first target content item block server; and

the method further comprises;

detecting creation or modification of a second content item that is not the first content item;

determining that the second content item belongs to the content item namespace;

selecting a second target content item block server, that is not the first target content item block server, based, at least in part, on the content item namespace; and

uploading, to the second target content item block server, one or more content item blocks of the second content item.

3. The method of claim 1, wherein the detecting creation or modification of the content item is based, at least in part, on detecting when a file corresponding to the content item is created or modified in a file system, remote from the target content item block server, at which the content item namespace is mounted.

4. The method of claim 1, wherein the determining the content item namespace to which the content item belongs is based, at least in part, on a) a file system path to a file corresponding to the content item, and b) a second file system path to a mount point of the content item namespace in a file system local to the computing system storing the one or more content item blocks.

5. The method of claim 1, wherein:

the content item is a first content item;

the content item namespace is a first content item namespace; and

the method further comprises;

detecting creation or modification of a second content item that is not the first content item;

determining that the second content item belongs to a second content item namespace that is not the first content item namespace;

identifying that the second content item namespace is assigned to the target content item block server; and

uploading, to the target content item block server, one or more content item blocks of the second content item.

6. A method performed by a computing system having one or more processors and storage media storing one or more programs, the one or more programs including instructions configured to perform the method, the instructions executed by the one or more processors to perform the method, the method comprising:

sending a current client journal cursor value to a metadata server of a content management service, the current client journal cursor value associated with a content item namespace and a first content item block server to which the content item namespace is assigned;

based, at least in part, on the sending the current client journal cursor value, receiving, from the metadata server, a first set of content item block identifiers corresponding to at least one content item block that makes up a content item, wherein the content item belongs to the content item namespace;

based, at least in part, on the receiving the first set of content item block identifiers, determining one or more content item blocks of the content item that are not locally stored by the computing system;

sending, to the metadata server, a user account identifier associated with the content item;

based, at least in part, on the sending the user account identifier, obtaining, from the metadata server, data representing assignments of the content item namespace to a plurality of candidate content item block servers, the plurality of candidate content item block servers including the first content item block server and a second content item block server;

selecting, from among the plurality of candidate content item block servers to which the content item namespace is assigned, the second content item block server from which to download the one or more content item blocks of the content item that were not locally stored by the computing system;

wherein the selecting the second content item block server is based, at least in part, on determining that an identifier of the content item namespace is associated with an identifier of the second content item block server in the data obtained from the metadata server representing the assignments of the content item namespace to the plurality of candidate content item block servers; and

based, at least in part, on selecting the second content item block server, downloading, to the computing system from the second content item block server, the one or more content item blocks of the content item that were not locally stored by the computing system.

7. The method of claim 6, wherein the first content item block server and the second content item block server are the same content item block server.

8. The method of claim 6, wherein:

the first content item block server and the second content item block server are different content item block servers;

the method further comprises receiving an identifier of the second content item block server from the metadata server; and

the downloading the one or more content item blocks of the content item is based, at least in part, on the received second content item block server identifier.

9. The method of claim 6, further comprising sending, to the metadata server, an identifier of the content item namespace and an identifier of the first content item block server.

10. A method, comprising:

detecting a new content item belonging to a content item namespace;

determining a client at which the content item namespace is mounted;

notifying the client that an update to the content item namespace is available;

receiving, from the client, a current client journal cursor value, the current client journal cursor value associated with the content item namespace and a content item block server to which the content item namespace is assigned;

based, at least in part, on the received current client journal cursor value, obtaining a set of identifiers of content item blocks, of which the new content item is composed, stored at the content item block server;

sending the set of content item block identifiers to the client;

wherein the content item namespace is assigned to a plurality of content item block servers, the plurality of content item block servers to which the content item namespace is assigned including the content item block server; and

wherein the method is performed by a computing system having one or more processors and storage media storing one or more programs, the one or more programs including instructions configured to perform the method, the instructions executed by the one or more processors to perform the method.

11. The method of claim 10, further comprising:

storing data identifying content item blocks stored at the content item block server; and

wherein the obtaining the set of identifiers of the content item blocks of the new content item stored at the content item block server is based, at least in part, on the data identifying content item blocks stored at the content item block server.

12. The method of claim 10, further comprising:

receiving, from the client, an identifier of the content item namespace and an identifier of the content item block server; and

wherein the obtaining the set of content item block identifiers is based, at least in part, on the received content item namespace identifier and the content item block server identifier from the client.

13. The method of claim 10, further comprising:

storing data representing assignments of content item namespaces to content item block servers; and

determining that the content item namespace is assigned to the content item block server based, at least in part, on the data representing assignments of content item namespaces to content item block servers; and

wherein the obtaining the set of content item block identifiers is based, at least in part, on the determining that the content item namespace is assigned to the content item block server.

14. A method, comprising:

receiving, from a client, (A) an identifier of a content item namespace and (B) a first set of identifiers of first content item blocks, the first content item blocks making up a first version of a content item stored at the client, the content item belonging to the content item namespace;

based, at least in part, on the receiving the identifier of the content item namespace and the first set of identifiers of first content item blocks from the client, selecting, from among a plurality of candidate content item block servers to which the content item namespace is assigned, a particular content item block server to which the content item namespace is assigned;

based, at least in part, on the first set of identifiers, determining a second set of identifiers of second content item blocks, the second content item blocks making up a second version of the content item stored at the particular content item block server;

sending, to the client, an indication of the selection of the particular content item block server and one or more content item block identifiers that are in the first set that are not in the second set; and

wherein the method is performed by a computing system having one or more processors and storage media storing one or more programs, the one or more programs including instructions configured to perform the method, the instructions executed by the one or more processors to perform the method.

15. The method of claim 14, further comprising:

storing data identifying content item blocks stored at the particular content item block server; and

wherein the determining the second set of identifiers of the second content item blocks is based, at least in part, on the data identifying content item blocks stored at the particular content item block server.

16. The method of claim 15, further comprising:

updating the data identifying content item blocks stored at the particular content item block server such that the updated data identifying content item blocks stored at the particular content item block server identifies one or more content item blocks in the first set, that are not in the second set.

17. The method of claim 14, further comprising:

based, at least in part, on the selecting the particular content item block server to which the content item namespace is assigned, sending an identifier of the particular content item block server to the client.

18. The method of claim 14, further comprising:

determining a server journal cursor value associated with the content item namespace and the particular content item block server, the server journal cursor value corresponding to the content item; and

sending the server journal cursor value to the client.

19. The method of claim 14, further comprising:

receiving an identifier of a user account from the client; and

based, at least in part, on the received user account identifier, verifying the user account has permission to access the content item namespace.

20. The method of claim 14, further comprising:

receiving an identifier of the particular content item block server from the client; and

wherein the selecting the particular content item block server to which the content item namespace is assigned is based, at least in part, on the received particular content item block server identifier from the client.

21. The method of claim 14, further comprising:

storing data representing assignments of content item namespaces to content item block servers; and

wherein the selecting the particular content item block server to which the content item namespace is assigned is based, at least in part, on the data representing the assignments.