Method and system for data synchronization, and apparatus thereof

• US 8,635,266 B2
• Filed: 07/28/2008
• Issued: 01/21/2014
• Est. Priority Date: 01/26/2006
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Kang, Jiao, Guo, Xiangzhou, Tian, Linyi
• Primary Examiner(s): Serrao, Ranodhi
• Assistant Examiner(s): Hussain, Farrukh
• Application Number:
  US12/180,626
• Publication Number:
  US 20090157802A1
• Time in Patent Office: 2,003 Days
• Field of Search: 709/201, 709/203, 709/217, 709/219, 709/248, 707/610, 707/611
• US Class Current: 709/203
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Embodiments of the present disclosure disclose a data synchronization method, system and apparatuses thereof. The data synchronization method includes the following processes: sending, by a client, a data characteristic verification message carrying one or multiple data characteristics for data in a client database, to a server; comparing, by the server, the one or multiple data characteristics carried in the data characteristic verification message with the data characteristics for the data in a server database; determining, by the server, the data to be synchronized based on a result of the comparing and sending the client a data characteristic verification response carrying information indicating the determined data to be synchronized; and determining, by the client, the data to be synchronized according to the information in the data characteristic verification response, and synchronizing the determined data to be synchronized with the server. The method, system and apparatus thereof provided by the embodiments of the present disclosure can reduce the amount of data to be transmitted in data synchronization, and improve the efficiency of data synchronization.

Claims

1. In a server configured to synchronize data with a client database, a data synchronization method thereof, comprising:

receiving, from a client, a data characteristic verification message that includes a plurality of Local Unique Identifiers (LUIDs) with corresponding data fingerprints that each represent portions of data in the client database for synchronization with a server database, wherein each of the corresponding data fingerprints uniquely identify contents of the portions of data in that different data has different data fingerprints and identical data has identical data fingerprints;

based on the plurality of LUIDs, comparing one or more of the corresponding data fingerprints included within the data characteristic verification message with one or more data fingerprints stored on the server database;

based on the comparison, determining at least one of the portions of data on the client database that needs to be synchronized with the server database; and

based on the determination, sending the client a data characteristic verification response that includes at least one of the plurality of LUIDs corresponding to the at least one of the portions of data on the client database that needs to be synchronized, which indicates to the client what data to synchronize with the server database.

2. The method according to claim 1, further comprising:

receiving the data fingerprints for the data in the server database from the client, and storing the received data fingerprints in the server, wherein the data fingerprints for the data is obtained, by the client, through computation based on a predetermined data fingerprint algorithm.

3. The method according to claim 2, wherein the receiving, by the server, the data fingerprints for the data in the server database from the client, and storing the received data fingerprints in the server comprise one of the following processes:

if the client and the server negotiate to employ a one-party computation smart sync, receiving, by the server, the data fingerprints sent from the client in advance, and storing the received data fingerprints, or keeping, by the server, the data fingerprints sent from the client in the last data synchronization in one-party computation smart sync;

otherwise, generating, by the server, the data fingerprints through the computation based on the data fingerprint algorithm determined in advance or determined by the client and the server in the negotiation; and

arranging, by the client and the server in advance, an agreement on employing one-party computation smart sync, wherein the agreement covers all data, data of all types in the client database and the server database, data of specified types, or data of specified size in the client database and server database.

4. The method according to claim 2, further comprising:

receiving status information of the last data synchronization and a sync type to be initiated this time from the client, wherein the sync type is determined according to the status information of the data in the client database;

determining, by the server, whether the received sync type is adopted according to the received status information of the last data synchronization in the client database, and sending the determined sync type to the client; and

receiving the data characteristic verification message carrying one or multiple data fingerprints for data in the client database from the client when the client determines that the received sync type is the one-party computation smart sync.

5. The method according to claim 4, wherein the determining by the server whether the received sync type is accepted according to the received status information of the last data synchronization in the client database further comprises:

determining to employ the one-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization and the mapping table and the data fingerprints in the server database are not lost;

determining to employ the one-party computation smart sync, when the server detects that the mapping table in the server database is lost, or the modification status information in the server database is lost, and the data fingerprints in the server database are not lost.

6. The method according to claim 5, wherein the status information of the last data synchronization comprises Anchor information;

and

the detecting by the server that the last data synchronization failed comprises;

comparing the Anchor from the client with the Anchor which was sent by the client in the last data synchronization and stored in the server database;

if the Anchor from the client is identical with the Anchor in the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

7. The method according to claim 5, wherein the status information of the last data synchronization comprises the data fingerprints generated for the client database through computation based on the predetermined data fingerprint algorithm;

and

the detecting the last data synchronization failed comprises;

comparing the data fingerprints of the client database from the client with the data fingerprints of the client database which were sent by the client in the last data synchronization and stored in the server database;

if the data fingerprints from the client are identical with the data fingerprints in the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

8. The method according to claim 7, wherein the generating data fingerprints for the client database through computation based on the predetermined data fingerprint algorithm comprises one of the following processes:

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database or the LUIDs of the data in the directories as inputted elements;

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database, or the data fingerprint information of the data in the directory as the inputted elements; and

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints or the LUIDs of all data in the client database as inputted elements.

9. The method according to claim 2, wherein

if the server modifies a piece of data in the server database, the server inserts an identification bit into the data fingerprints of the data, and stores the data fingerprints, and

the determining the data to be synchronized based on a result of the comparing comprises:

determining that the data corresponding to the received data fingerprints need to be synchronized if the server database includes no data fingerprint identical with the received data fingerprints;

determining whether the identical data fingerprints in the server database includes an identification bit, if the server database includes data fingerprints identical with the received data fingerprints;

determining that the data corresponding to the received data fingerprints need to be synchronized, if the data fingerprints in the server database include the identification bit, otherwise, determining that the data corresponding to the identical data fingerprints does not need to be synchronized.

10. The method according to claim 2, further comprising:

receiving a data synchronization completing confirmation message from the server, wherein the data synchronization completing confirmation message comprises the data synchronization status information indicating a completion of the data synchronization; and

storing the data synchronization status information in the data synchronization completing confirmation message in the server database and returning a response message.

11. The method according to claim 1, further comprising:

obtaining, by the client, one or multiple data fingerprints through a real-time computation based on a predetermined data fingerprint algorithm;

or obtaining, by the client, the one or multiple data fingerprints in advance through the computation based on the predetermined data fingerprint algorithm, and storing the one or multiple data fingerprints in the client; and

setting, by the server, the data fingerprints for the data in the server database through the computation based on the predetermined data fingerprint algorithm.

12. The method according to claim 1, further comprising:

negotiating, by the client and the server, that a two-party computation smart sync is employed, and the data fingerprints are set through computation based on the predetermined data fingerprint algorithm; and

arranging, by the client and the server in advance, an agreement on employing the two-party computation smart sync, wherein the agreement covers all data in the client database and server database, or data of specified types in the client database and server database.

13. The method according to claim 1, wherein the data fingerprint algorithms used in the client and the server are the same data fingerprint algorithm, and the data fingerprint algorithm is determined in advance, or determined after a negotiation between the client and the server.

14. The method according to claim 11, further comprising:

receiving, status information of the last data synchronization and a sync type to be initiated this time, from the server, wherein the sync type is determined according to status of the client database;

determining whether the received sync type is adopted according to the received status information of the last data synchronization, and sending the determined sync type to the client; and

receiving the data characteristic verification message carrying one or multiple data fingerprints for data in the client database from the client when the client determines that the received sync type is the two-party computation smart sync.

15. The method according to claim 14,

the determining the sync type comprises:

determining to employ the two-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization, or the mapping table in the server database is lost, or the data fingerprints in the server database are lost.

16. The method according to claim 15, wherein the status information of the last data synchronization, which is sent by the client, comprises Anchor information;

and

the detecting that the last data synchronization failed comprises;

comparing the Anchor from the client with the Anchor which was sent by the client in the last data synchronization and stored in the server database or generated by the server for the server database, if the Anchor from the client is identical with the Anchor on the server, determining that the last data synchronization succeeded, otherwise, determining that the last data synchronization failed.

17. The method according to claim 15, wherein the status information of the last data synchronization, which is sent by the client, comprises data fingerprints generated for the client database through the computation based on the predetermined data fingerprint algorithm;

and

the detecting that the last data synchronization failed comprises;

comparing the data fingerprints of the client database from the client with the data fingerprints of the server database;

if the data fingerprints of the client database are identical with the data fingerprints of the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

18. The method according to claim 17, wherein the generating data fingerprints for the client database through computation based on the predetermined data fingerprint algorithm comprises one of the following processes:

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database or the LUIDs of the data in the directories as inputted elements;

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database, or the data fingerprint information of the data in the directory as the inputted elements; and

generating data fingerprints of the client database through the computation based on the predetermined data fingerprint algorithm with the data fingerprints or the LUIDs of all data in the client database as inputted elements.

19. The method according to claim 1, wherein the information indicating the determined data to be synchronized comprises:

LUIDs of the data to be synchronized, or the data fingerprints of the data to be synchronized in the client database, or status codes corresponding to the data fingerprints of the data to be synchronized in the client database.

20. The method according to claim 1, wherein the data includes directories, files or data items.

21. The method according to claim 20, wherein when the data includes the directory, the data fingerprints of the directory are set through computation based on the predetermined data fingerprint algorithm with the data fingerprints of the data in the directory or the LUIDs of the data under the directory as inputted elements;

when the data includes the file or the data item, the data fingerprints of the file or data item are set through computation based on the predetermined data fingerprint algorithm with contents of the file or the data item as inputted elements.

22. The method according to claim 1, wherein the data fingerprint is obtained by computing the data based on a preset data fingerprint algorithm or a data fingerprint algorithm negotiated by the client and the server, and when the data is modified a new data fingerprint is generated and the data fingerprint is updated by using the new data fingerprint.

23. The method according to any one of claim 2, wherein the predetermined data fingerprint algorithm includes one of the following algorithm:

a digest algorithm, a cyclic redundancy check algorithm, a normal hash algorithm, and a timestamp algorithm.

24. A computer program product for use in a server used to synchronize a client database, wherein the computer program product comprising one or more computer executable instructions stored on a non-transitory computer readable medium such that when executed by a processor cause the server to perform the following:

receiving, from a client, a data characteristic verification message that includes a plurality of Local Unique Identifiers (LUIDs) with corresponding data fingerprints that each represent portions of data in the client database for synchronization with a server database, wherein each of the corresponding data fingerprints uniquely identify contents of the portions of data in that different data has different data fingerprints and identical data has identical data fingerprints;

based on the plurality of LUIDs, comparing one or more of the corresponding data fingerprints included within the data characteristic verification message with one or more data fingerprints stored on the server database;

based on the comparison, determining at least one of the portions of data on the client database that needs to be synchronized with the server database; and

based on the determination, sending the client a data characteristic verification response that includes at least one of the plurality of LUIDs corresponding to the at least one of the portions of data on the client database that needs to be synchronized, which indicates to the client what data to synchronize with the server database.