Peer-to-peer synchronization protocol for multi-premises hosting of digital content items

• US 9,571,573 B1
• Filed: 12/22/2015
• Issued: 02/14/2017
• Est. Priority Date: 10/29/2015
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: Dropbox, Inc.
• Inventors: Koorapati, Nipunn, Rude, Christopher, von Muhlen, Marcio, Bunger, Nils
• Primary Examiner(s): Hussain, Tauqir
• Application Number:
  US14/979,256
• Time in Patent Office: 420 Days
• Field of Search: 709/223
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 47/196 Integration of transport layer protocols, e.g. TCP and UDP
• CPC Class Codes: H04L 61/457 containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A peer-to-peer synchronization protocol for multi-premises hosting of digital content items. In one embodiment, for example, a method comprises the operations of: receiving, from a peer computing device, a peer download advertisement comprising an identifier of a content item namespace in association with a peer client journal cursor value; receiving, from a server, one or more server journal entries, each identifying one or more content item blocks of a content item represented by the server journal entry and each having a server journal cursor value; and for at least one server journal entry, downloading, from the peer computing device, at least one content item block of the content item represented by the server journal entry that is not already stored at the computing device.

Claims

1. A method, comprising:

at a computing device comprising one or more processors and memory storing one or more computer programs executed by the one or more processors to perform the method, performing operations comprising;

receiving, from a peer computing device, a peer download advertisement, the peer download advertisement comprising an identifier of a content item namespace in association with a peer client journal cursor value;

wherein the computing device receives the peer download advertisement from the peer computing device over a data network either (a) in one or more user datagram packets (UDP) sent over the data network by the peer computing device, or (b) over a transmission control protocol (TCP) connection established over the data network between the computing device and the peer computing device;

receiving, from a server, one or more server journal entries, each server journal entry, of the one or more server journal entries, identifying one or more content item blocks of a content item represented by the server journal entry, each server journal entry of the one or more server journal entries having a server journal cursor value; and

for at least one server journal entry of the one or more server journal entries, determining, based on the one or more identified content item blocks of the content item represented by the server journal entry, that at least one content item block of the content item represented by the server journal entry is not stored at the computing device, and downloading, from the peer computing device, at least one content item block of the content item represented by the server journal entry that is not already stored at the computing device.

2. The method of claim 1, the operations further comprising:

determining if the server journal cursor value of the at least one server journal entry is less than the peer client journal cursor value; and

downloading, from the peer computing device, the at least one content item block of the content item represented by the at least one server journal entry that is not already stored at the computing device, if the server journal cursor value of the at least one server journal entry is less than the peer client journal cursor value.

3. The method of claim 1, wherein the operation of receiving the peer download advertisement comprises the operation of receiving a user datagram packet (UDP) comprising the peer download advertisement sent by the peer computing device.

4. The method of claim 1, the operations further comprising:

establishing a network connection with the peer computing device; and

prior to downloading the at least one content item block of the content item represented by the at least one server journal entry, using the network connection to determine if the at least one content item block of the content item represented by the at least one server journal entry is available for download from the peer computing device.

5. The method of claim 1, wherein the one or more server journal entries comprises a plurality of server journal entries, and wherein the operations further comprise:

for at least one other server journal entry of the plurality of server journal entries, downloading, from a block server, at least one content item block of the content item represented by the other server journal entry that is not already stored at the computing device.

6. The method of claim 5, wherein the block server is an off-premises block server operated by an online content management service.

7. The method of claim 1, wherein:

the peer computing device is a first peer computing device;

the peer download advertisement is a first peer download advertisement;

the content item namespace is a first content item namespace;

the peer client journal cursor value is a first peer client journal cursor value;

the one or more server journal entries is a plurality of server journal entries;

the operations further comprise;

receiving, from a second peer computing device that is not the first peer computing device, a second peer download advertisement that is not the first peer download advertisement, the second peer download advertisement comprising an identifier of a second content item namespace, that is not the first content item namespace, in association with a second peer client journal cursor value that is not the first peer client journal cursor value; and

for at least one other server journal entry of the plurality of server journal entries, downloading, from the second peer computing device, at least one content item block of the content item represented by the other server journal entry that is not already stored at the computing device.

8. The method of claim 1, the operations further comprising:

storing a peer devices table at the computing device; and

updating the peer devices table with information from the peer download advertisement.

9. The method of claim 1, wherein the peer download advertisement comprises a plurality of advertised content item namespaces, each of the plurality of advertised content item namespaces associated in the peer download advertisement with an identifier of the advertised content item namespace.

10. The method of claim 1, wherein the peer download advertisement comprises one or more advertised content item namespaces, each of the one or more advertised content item namespaces associated in the peer download advertisement with a client journal cursor value reflecting how up-to-date the peer computing device is with respect to any newer content item changes to the associated advertised content item namespace committed to an online content management service.

11. A computing device, comprising:

one or more processors; and

storage media storing one or more computer programs comprising instructions configured for;

receiving, from a peer computing device, a peer download advertisement, the peer download advertisement comprising an identifier of a content item namespace in association with a peer client journal cursor value;

wherein the computing device receives the peer download advertisement from the peer computing device over a data network either (a) in one or more user datagram packets (UDP) sent over the data network by the peer computing device, or (b) over a transmission control protocol (TCP) connection established over the data network between the computing device and the peer computing device;

receiving, from a server, one or more server journal entries, each server journal entry, of the one or more server journal entries, identifying one or more content item blocks of a content item represented by the server journal entry, each server journal entry of the one or more server journal entries having a server journal cursor value; and

for at least one server journal entry of the one or more server journal entries, determining, based on the one or more identified content item blocks of the content item represented by the server journal entry, that at least one content item block of the content item represented by the server journal entry is not stored at the computing device, and downloading, from the peer computing device, at least one content item block of the content item represented by the server journal entry that is not already stored at the computing device.

12. The computing device of claim 11, the instructions further configured for:

determining if the server journal cursor value of the at least one server journal entry is less than the peer client journal cursor value; and

downloading, from the peer computing device, the at least one content item block of the content item represented by the at least one server journal entry that is not already stored at the computing device, if the server journal cursor value of the at least one server journal entry is less than the peer client journal cursor value.

13. The computing device of claim 11, wherein the instructions configured for receiving the peer download advertisement comprise instructions configured for receiving a user datagram packet (UDP) comprising the peer download advertisement sent by the peer computing device.

14. The computing device of claim 11, the instructions further configured for:

establishing a network connection with the peer computing device; and

prior to downloading the at least one content item block of the content item represented by the at least one server journal entry, using the network connection to determine if the at least one content item block of the content item represented by the at least one server journal entry is available for download from the peer computing device.

15. The computing device of claim 11, wherein the one or more server journal entries comprises a plurality of server journal entries, and wherein the instructions are further configured for:

for at least one other server journal entry of the plurality of server journal entries, downloading, from a block server, at least one content item block of the content item represented by the other server journal entry that is not already stored at the computing device.

16. The computing device of claim 15, wherein the block server is an off-premises block server operated by an online content management service.

17. The computing device of claim 11, wherein:

the peer computing device is a first peer computing device;

the peer download advertisement is a first peer download advertisement;

the content item namespace is a first content item namespace;

the peer client journal cursor value is a first peer client journal cursor value;

the one or more server journal entries is a plurality of server journal entries;

the instructions are further configured for;

receiving, from a second peer computing device that is not the first peer computing device, a second peer download advertisement that is not the first peer download advertisement, the second peer download advertisement comprising an identifier of a second content item namespace, that is not the first content item namespace, in association with a second peer client journal cursor value that is not the first peer client journal cursor value; and

for at least one other server journal entry of the plurality of server journal entries, downloading, from the second peer computing device, at least one content item block of the content item represented by the other server journal entry that is not already stored at the computing device.

18. The computing device of claim 11, the instructions further configured for:

storing a peer devices table at the computing device; and

updating the peer devices table with information from the peer download advertisement.

19. The computing device of claim 11, wherein the peer download advertisement comprises a plurality of advertised content item namespaces, each of the plurality of advertised content item namespaces associated in the peer download advertisement with an identifier of the advertised content item namespace.

20. The computing device of claim 11, wherein the peer download advertisement comprises one or more advertised content item namespaces, each of the one or more advertised content item namespaces associated in the peer download advertisement with a client journal cursor value reflecting how up-to-date the peer computing device is with respect to any newer content item changes to the associated advertised content item namespace committed to an online content management service.