Data synchronization based on file system activities

• US 9,276,980 B2
• Filed: 10/02/2013
• Issued: 03/01/2016
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Razer (Asia-Pacific) Pte. Ltd. (Razer Incorporated)
• Original Assignee: Nextbit Systems, Inc. (Razer Incorporated)
• Inventors: Chan, Michael A., Quan, Justin, Fleming, Michael K.
• Primary Examiner(s): Arjomandi, Noosha
• Application Number:
  US14/044,605
• Publication Number:
  US 20140101103A1
• Time in Patent Office: 881 Days
• Field of Search: 707/634
• US Class Current: 1/1
• CPC Class Codes: G06F 16/125 characterised by the use of retention policies retention policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 9/5038 considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration scheduling strategies G06F9/4881 and subgroups
• CPC Class Codes: H04B 7/26 at least one of which is mobile
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/55 Push-based network services
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Technology is disclosed herein for data synchronization based on file system activities. According to at least one embodiment, a computing device monitors one or more file system operations for a file stored in the computing device. The computing device records the file system operations for the file sequentially into a log data structure. The computing device synchronizes the file with a server by sending the log data structure to the server. The server can apply the file system operations recorded in the log data to an instance of the file stored in the server such that data of the file stored in the computing device are consistent with data in the instance of the file stored in the server after the file system operations are applied.

Claims

1. A computer implemented method for data synchronization based on file system activities, the method comprising:

monitoring, at a computing device, one or more file system operations for a file stored in the computing device;

recording, at the computing device, the file system operations to the file sequentially into a log data structure; and

synchronizing the file with a synchronization device by sending the log data structure to the synchronization device;

wherein the file system operations recorded in the log data structure are suitable to be applied to an instance of the file stored in the synchronization device such that data of the file stored in the computing device are consistent with data in the instance of the file stored in the synchronization device after the file system operations being applied;

wherein the log data structure includes an offset distance and a data checksum of new content data for a data write operation, and the data checksum is suitable to be used to determine whether another data write operation received by the synchronization device is the same operation as the data write operation.

2. The method of claim 1, wherein the log data structure is suitable for determining whether data of the file stored in the computing device are consistent with data in the instance of the file stored in the synchronization device.

3. The method of claim 1, wherein the synchronization device is a storage server or another computing device.

4. The method of claim 1, further comprising:

generating, at the computing device, a file checksum from data of the file; and

sending the file checksum to the synchronization device, wherein the file checksum is suitable to be compared with a file checksum generated from data of the instance of the file stored in the synchronization device.

5. The method of claim 1, wherein the file system operations are suitable to be applied to the instance of the file stored in the synchronization device to generate a snapshot of a version of the file.

6. The method of claim 1, wherein the file system operations are recorded in the log data structure based on an ordered sequence of the file system operations being applied to the file stored in the computing device.

7. The method of claim 1, wherein the file system operations recorded in the log data structure include at least one of a data write operation, a data removal operation, a file delete operation, a file creation operation, a metadata change operation, a file rename operation, a file truncate operation.

8. The method of claim 1, wherein the log data structure includes an offset distance and a data chunk including new content data for a data write operation.

9. The method of claim 1, wherein the log data structure includes an offset distance and a data removal length for a data removal operation.

10. The method of claim 1, wherein the log data structure include a new file name for a metadata change operation.

11. The method of claim 1, wherein file system operations to multiple files stored in the computing device are monitored and recorded into a log data structure.

12. The method of claim 1, wherein file system operations to multiple files stored in the computing device are monitored, and the file system operations to each of the files are recorded into one of multiple log data structures.

13. The method of claim 1, wherein the log data structure is part of a metadata structure for the file.

14. The method of claim 1, wherein the log data structure is a file or a database.

15. The method of claim 1, further comprising:

compressing the log data structure at the computing device.

16. A method for synchronizing data based on file system activities in a client-server distributed storage system, the server comprising:

receiving, from a first computing device at a server, a log data structure recording one or more file system operations, wherein the file system operations are generated by the first computing device to be applied to a version of the data container stored in the first computing device;

determining, at the server, whether a version of the data container stored in the server is the same as the version of the data container stored in the first computing device;

synchronizing the data container by applying the file system operations to the version of the data container stored in the server, when the version of the data container stored in the server is not the same as the version of the data container stored in the first computing device;

wherein a data write operation recorded in the log data structure includes an offset distance and a data checksum generated from new content data for the data write operation.

17. The method of claim 16, wherein the log data structure recording file system operations to be applied to multiple data containers stored in the server, and the log data structure includes data container identifications for the file system operations.

18. The method of claim 16, further comprising:

receiving, from the first computing device, a first checksum generated from the version of the data container stored in the first computing device;

generating, at the server, a second checksum from the version of the data container stored in the server; and

wherein the step of determining comprises;

comparing the first checksum with the second checksum.

19. The method of claim 16, further comprising:

receiving, from a second computing device, a log data structure recording file system operations for a version of the data container stored in the second computing device;

determining, at the server, whether there is a conflict between the log data structures from the first and second computing devices;

consolidating the log data structures from the first and second computing devices if there is no conflict between the log data structures from the first and second computing devices.

20. The method of claim 19, wherein the step of determining whether there is a conflict comprises:

comparing a first data checksum included in the log data structure from the first computing device with a second data checksum included in the log data structure from the second computing device;

wherein the first data checksum is generated from new content data for a data write operation recorded in the log data structure from the first computing device, and wherein the second data checksum is generated from new content data for a data write operation recorded in the log data structure from the second computing device.

21. The method of claim 20, further comprising:

receiving, from the first computing device, the new content data associated with the first data checksum; and

sending an instruction, to the second computing device, not to send the new content data associated with the second data checksum, if the second data checksum equals to the first checksum.

22. The method of claim 16, wherein a data write operation recorded in the log data structure includes an offset distance and a data chunk including new content data for the data write operation.

23. The method of claim 16, further comprising:

generating, at the server, a snapshot of a new version of the data container by applying the file system operations to the version of the data container stored in the server.

24. A storage server, comprising:

a processor;

a storage component configured to store a server version of a file;

a networking component configured to receive from a first computing device a log data structure recording one or more file system operations, wherein the file system operations are generated by the first computing device to be applied to a device version of the file stored in the first computing device;

a synchronization module which, when executed by the processor, apply the file system operations recorded in the log data structure to the server version of the file such that after the applying the file system operations, data of the file stored in the storage component of the storage server are the same as data of the file stored in the first computing device;

wherein the file system operations recorded in the log data structure include a data write operation having an offset distance and a data checksum generated from new content data for the data write operation; and

wherein the networking component is further configured to receive the new content data for the data write operation.

25. The storage server of claim 24, wherein the log data structure represents a difference between data of the file stored in the first computing device and data of the file stored in the server.

26. The storage server of claim 24, wherein the file system operations recorded in the log data structure include a data write operation having an offset distance and a data chunk including new content data for the data write operation, or a data removal operation having an offset distance and a data removal length for a data removal operation.

27. The storage server of claim 24, wherein the network component is further configured to receive another log data structure from a second computing device;

and wherein the synchronization module is further configured to determine whether the other log data structure is redundant by comparing the file system operations in the log data structure with the file system operations in the other log data structure.

28. The storage server of claim 24, wherein the synchronization module is further configured to determine whether all devices connected to the server that store the file contain the newest version of the file as stored in the server;

and wherein the synchronization module is further configured to remove the log data structure if all devices connected to the server that store the file contain the newest version of the file as stored in the server.

29. The storage server of claim 24, wherein the networking component is further configured to send the log data structure to a second computing device such that the file stored in the second computing device can be synchronized with the file stored in the server by applying the file system operations of the log data structure to the file stored in the second computing device.

30. The storage server of claim 24, wherein the synchronization module is further configured to determine whether to send to the second computing device for a file synchronization purpose the log data structure or the file stored in the server, depending on the sizes of the log data structure and the file stored in the server.