Data files synchronization with cloud storage service

• US 10,574,753 B2
• Filed: 03/20/2017
• Issued: 02/25/2020
• Est. Priority Date: 12/22/2008
• Status: Active Grant

• Current Assignee: CTERA Networks Limited
• Original Assignee: CTERA Networks Limited
• Inventors: Brand, Aron
• Primary Examiner(s): Vu, Viet D
• Application Number:
  US15/463,433
• Publication Number:
  US 20170195417A1
• Time in Patent Office: 1,072 Days
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H04L 9/0618 Block ciphers, i.e. encrypting groups of characters of a plain text message using fixed encryption transformation
• CPC Class Codes: H04L 9/3242 involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC

Abstract

A method for synchronizing data files between a client device and a cloud storage service (CSS). The method comprises marking each data file that has been changed since a last synchronization between the client device and the CSS; dividing each marked data file into a plurality of blocks; assigning each block with a unique code; determining, based on the assigned the unique codes, a set of modified blocks from among the plurality of blocks; and synchronizing the set of modified blocks such that the CSS and the client device include a consistent version of each data file that has changed.

Claims

1. A method for synchronizing data files between a client device and a cloud storage service (CSS), comprising:

marking each data file that has been changed since a last synchronization between the client device and the CSS, the marking being done in response to a determination that the file has changed that is made without use of communication between the client device and the CSS;

dividing each marked data file into a plurality of blocks;

assigning each block with a unique code;

determining, based on the assigned the unique codes, a set of modified blocks from among the plurality of blocks;

synchronizing the set of modified blocks such that the CSS and the client device include a consistent version of each data file that has changed; and

pipelining execution of;

dividing the file into the plurality of blocks, and assigning each block with the unique code.

2. The method of claim 1, wherein determining the set of modified blocks further comprises:

sending a list of the unique codes to the CSS; and

receiving, from the CSS, a subset of the list of unique codes, wherein the subset includes codes of the list of unique codes that are missing from the CSS.

3. The method of claim 2 wherein the list of the unique codes is included in a file map designating at least a sequence of block codes of all of the plurality of blocks in any one of:

a data file and a byte range of the data file.

4. The method of claim 1, further comprising at least one of

compressing the modified blocks;

and

encrypting the modified blocks.

5. The method of claim 1, wherein assigning each block with a unique code further comprises:

calculating a message digest function of the block.

6. The method of claim 1, wherein transmitting the modified blocks further comprises:

using a cloud transport protocol (CTP).

7. The method of claim 1, wherein synchronizing the modified blocks is performed in at least one of:

from the client device to the CSS, and from the CSS to the client device.

8. The method of claim 1, wherein the synchronization of data files is between a plurality of client devices and the CSS, wherein least one data file is shared among the plurality of client devices.

9. The method of claim 1, wherein the client device is at least a network attached storage.

10. The method of claim 9, wherein the client device and the CSS are geographically distributed from each other.

12. The method of claim 1, wherein the client device is at least a network attached storage.

13. The method of claim 9, wherein the client device and the CSS are geographically distributed from each other.

11. A non-transitory computer readable medium having stored thereon instructions for causing one or more processors to perform a process for synchronizing data files between a client device and a cloud storage service (CSS), the process comprising:

marking each data file that has been changed since a last synchronization between the client device and the CSS, the marking being done in response to a determination that the file has changed that is made without use of communication between the client device and the CSS;

between the client device and the CSS;

dividing each marked data file into a plurality of blocks;

assigning each block with a unique code;

determining, based on the assigned the unique codes, a set of modified blocks from among the plurality of blocks;

synchronizing the set of modified blocks such that the CSS and the client device include a consistent version of each data file that has changed; and

pipelining execution of;

dividing the file into the plurality of blocks, and assigning each block with the unique code.

14. A system for storing data files in a cloud storage service (CSS), comprising:

storage configured to store a plurality of data files;

a processor; and

a memory, the memory containing instructions that, when executed by the processor, configure the system to;

for each data file of the plurality of data files locally stored in the storage;

mark each data file that has been changed since a last synchronization between the client device and the CSS, the marking being done in response to a determination that the file has changed that is made without use of communication between the client device and the CSS;

divide the data file into a plurality of blocks;

assign each block of the plurality of blocks with a unique code;

determine, based on the unique codes, at least one block that is not stored in a same device group in the CSS;

transmit the at least one block to the CSS to be stored in the CSS, thereby maintaining at least one consistent version of the data file; and

pipelining execution of;

dividing the data file into the plurality of blocks, and assigning each block of the plurality of blocks with the unique code.