METHOD AND SYSTEM FOR DATA SYNCHRONIZATION, AND APPARATUS THEREOF

• US 20090157802A1
• Filed: 07/28/2008
• Published: 06/18/2009
• Est. Priority Date: 01/26/2006
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Tian, Linyi, Kang, Jiao, Guo, Xiangzhou
• Granted Patent: US 8,635,266 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Embodiments of the present disclosure disclose a data synchronization method, system and apparatuses thereof. The data synchronization method includes the following processes: sending, by a client, a data characteristic verification message carrying one or multiple data characteristics for data in a client database, to a server; comparing, by the server, the one or multiple data characteristics carried in the data characteristic verification message with the data characteristics for the data in a server database; determining, by the server, the data to be synchronized based on a result of the comparing and sending the client a data characteristic verification response carrying information indicating the determined data to be synchronized; and determining, by the client, the data to be synchronized according to the information in the data characteristic verification response, and synchronizing the determined data to be synchronized with the server. The method, system and apparatus thereof provided by the embodiments of the present disclosure can reduce the amount of data to be transmitted in data synchronization, and improve the efficiency of data synchronization.

Claims

1. A data synchronization method, comprising:

sending, by a client, a data characteristic verification message carrying one or multiple data characteristics for data in a client database, to a server;

comparing, by the server, the one or multiple data characteristics carried in the data characteristic verification message with the data characteristics for the data in a server database;

determining, by the server, the data to be synchronized based on a result of the comparing and sending the client a data characteristic verification response carrying information indicating the determined data to be synchronized; and

determining, by the client, the data to be synchronized according to the information in the data characteristic verification response, and synchronizing the determined data to be synchronized with the server.

2. The method according to claim 1, further comprising:

obtaining, by the client, one or multiple data characteristics through a real-time computation based on a predetermined data characteristic algorithm;

or obtaining, by the client, the one or multiple data characteristics in advance through the computation based on the predetermined data characteristic algorithm, and storing the one or multiple data characteristics in the client; and

receiving, by the server, the data characteristics for the data in the server database from the client, and storing the received data characteristics in the server.

3. The method according to claim 2, wherein the receiving, by the server, the data characteristics for the data in the server database from the client, and storing the received data characteristics in the server comprise one of the following processes:

if the client and the server negotiate to employ a one-party computation smart sync, receiving, by the server, the data characteristics sent from the client in advance, and storing the received data characteristics, or keeping, by the server, the data characteristics sent from the client in a last data synchronization in the one-party computation smart sync;

otherwise, generating, by the server, the data characteristics through the computation based on the data characteristic algorithm determined in advance or determined by the client and the server in the negotiation; and

arranging, by the client and the server in advance, an agreement on employing the one-party computation smart sync, wherein the agreement covers all data, data of all types in the client database and server database, data of specified types, or data of specified size in the client database and server database.

4. The method according to claim 2, further comprising:

determining, by the client, a sync type to be initiated according to status information of the data in the client database, and reporting to the server the status information of the last data synchronization and the sync type to be initiated this time;

determining, by the server, whether the received sync type is adopted according to the received status information of the last data synchronization in the client database, and sending the determined sync type to the client; and

determining, by the client, whether the received sync type is the one-party computation smart sync, if the received sync type is the one-party computation smart sync, sending, by the client, the data characteristic verification message carrying one or multiple data characteristics for data in the client database, to the server;

otherwise, initiating, by the client, the sync determined in a negotiation between by the client and the server, or restarting the negotiation between the client and the server.

5. The method according to claim 4, wherein the determining by the client a sync type to be initiated comprises:

determining to employ a slow sync if data characteristic generation method of the data in the client database is changed;

determining to employ a one-party computation smart sync if Local Unique IDs, LUIDs, of the data in the client database are out of order or/and regenerated, or if a Change Log in the client database is lost;

determining to employ a sync type chosen by a user using the client, if the data characteristic generation method of the data in the client database is unchanged, the data LUIDs in the client database are not out of order or regenerated, and the Change Log in the client database is not lost.

6. The method according to claim 4, wherein the determining by the server whether the received sync type is accepted according to the received status information of the last data synchronization in the client database further comprises:

determining to employ the one-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization and the mapping table and the data characteristics in the server database are not lost;

determining to employ the one-party computation smart sync, when the server detects that the mapping table in the server database is lost, or/and the modification status information in the server database is lost, and the data characteristics in the server database are not lost;

determining to employ a slow sync, when the server detects that the mapping table and the data characteristics in the server database are lost;

determining to employ the slow sync, when the server detects that the data characteristic generation method of the data in the client database is changed.

7. The method according to claim 6, wherein the status information of the last data synchronization comprises an Anchor information;

and

the detecting by the server that the last data synchronization failed comprises;

comparing, by the server, the Anchor from the client with the Anchor which was sent by the client in the last data synchronization and stored in the server database;

if the Anchor from the client is identical with the Anchor in the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

8. The method according to claim 6, wherein the status information of the last data synchronization comprises the data characteristics generated for the client database through computation based on the predetermined data characteristic algorithm;

and

the detecting by the server the last data synchronization failed comprises;

comparing, by the server, the data characteristics of the client database from the client with the data characteristics of the client database which were sent by the client in the last data synchronization and stored in the server database;

if the data characteristics from the client are identical with the data characteristics in the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

9. The method according to claim 8, wherein the generating data characteristics for the client database through computation based on the predetermined data characteristic algorithm comprises one of the following processes:

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics of all directories in the client database and/or the LUIDs of the data in the directories as inputted elements;

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics of all directories in the client database, and/or the data characteristic information of the data in the directory as the inputted elements; and

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics and/or the LUIDs of all data in the client database as inputted elements.

10. The method according to claim 2, wherein

if a LUIDs of the data characteristics in the client database are out of order or/and regenerated while the modification status information in the client data base is not lost,

the data characteristics in the client database comprise the data characteristics of the to-be-synchronized data with changed modification status information, or the data characteristics of all to-be-synchronized data;

and

the comparing comprises;

comparing, by the server, all the data characteristics from the client with the data characteristics in the server database;

or

if the LUIDs of the data characteristics in the client database are not out of order or regenerated while the modification status information in the client data base is lost,

the data characteristics in the client database comprise the data characteristics of all to-be-synchronized data identified with LUIDs in the client database; and

the comparing comprises;

determining, by the server, corresponding data in the server database according to the one or multiple LUIDs of the data characteristics in the data characteristic verification message, and comparing the one or multiple data characteristics in the data characteristic verification message with the data characteristics of the determined corresponding data in the server database;

or

if the LUIDs of the data characteristics in the client database are out of order or regenerated while the modification status information in the client data base is lost,

the data characteristics in the client database comprises the data characteristics of all to-be-synchronized data in the client database; and

the comparing comprises;

comparing, by the server, the data characteristics from the client with all data characteristics in the server database.

11. The method according to claim 10, wherein if the server modifies a piece of data in the server database, the server deletes the data characteristics of the data, and

the comparing by the server the data characteristics from the client with the data characteristics in the server database comprises:

determining, by the server, whether the server database includes data characteristics identical with the received data characteristics; and

if the server database includes data characteristics identical with the received data characteristics, determining that the data corresponding to the identical data characteristics does not need to be synchronized;

otherwise, determining that the data corresponding to the received data characteristics need to be synchronized;

or

if the server modifies the piece of data in the server database, the server inserts an identification bit into the data characteristics of the data, and stores the data characteristics, and

the comparing by the server the data characteristics from the client with the data characteristics in the server database comprises;

determining, by the server, whether the server database includes data characteristics identical with the received data characteristics;

if the server database includes no data characteristic identical with the received data characteristics, determining that the data corresponding to the received data characteristics need to be synchronized;

if the server database includes data characteristics identical with the received data characteristics, determining whether the identical data characteristics in the server database includes an identification bit, if the data characteristics in the server database include the identification bit, determining that the data corresponding to the received data characteristics need to be synchronized, otherwise, determining that the data corresponding to the identical data characteristics does not need to be synchronized.

12. The method according to claim 10, wherein the synchronizing the determined data to be synchronized with the server comprises:

determining, by the client, the data to be sent to the server according to the information in the data characteristic verification response, and sending the determined data to the server so that the server can synchronize the data in the server database with the data received by the server; and

sending, by the server, the data to be sent to the client according to comparison results, so that the client synchronizes the data in the client database with the data received by the client.

13. The method according to claim 12, wherein the data to be sent to the client comprises one of the following data:

the data in the server database corresponding to the data characteristics that are different from the received data characteristics;

the data in the server database corresponding to the data characteristics that are identical with the received data characteristics, and including identification bit(s); and

the data in the server database corresponding to the data characteristics different from the received data characteristics, and to be sent to the client after conflict detection and resolution.

14. The method according to claim 12, further comprising:

sending, by the client, the data synchronization status information indicating a completion of the data synchronization to the server in a data synchronization completing confirmation message; and

storing, by the server, the data synchronization status information in the data synchronization completing confirmation message in the server database and returning a response message.

15. The method according to claim 1, further comprising:

obtaining, by the client, one or multiple data characteristics through a real-time computation based on a predetermined data characteristic algorithm;

or obtaining, by the client, the one or multiple data characteristics in advance through the computation based on the predetermined data characteristic algorithm, and storing the one or multiple data characteristics in the client; and

setting, by the server, the data characteristics for the data in the server database through the computation based on the predetermined data characteristic algorithm.

16. The method according to claim 15, wherein the obtaining by the server the data characteristics for the data in the server database through the computation based on the predetermined data characteristic algorithm comprises one of the following processes:

negotiating, by the client and the server, whether a two-party computation smart sync is employed;

if the two-party computation smart sync is employed, the data characteristics are set through computation based on the predetermined data characteristic algorithm;

otherwise, the data characteristics are sent by the client and stored by the server; and

arranging, by the client and the server in advance, an agreement on employing the two-party computation smart sync, wherein the agreement covers all data in the client database and server database, or data of specified types in the client database and server database.

17. The method according to claim 15, wherein the data characteristic algorithms used in the client and the server are the same data characteristic algorithm, and the data characteristic algorithm is determined in advance, or determined after a negotiation between the client and the server.

18. The method according to claim 17, wherein the negotiation on the data characteristic algorithm between the client and the server comprises:

sending, by the client, a data package carrying one or multiple data characteristic algorithm identifications to the server;

determining, by the server, the data characteristic algorithms corresponding to the data characteristic algorithm identifications in the data package, and choosing a data characteristic algorithm based on status of the server from the data characteristic algorithms; and

returning, from the server, a data package carrying the identification of the chosen data characteristic algorithm to the client; and

determining, by the client, according to the data characteristic algorithm identification in the returned data package, the data characteristic algorithm for setting the data characteristics for the data in the client database.

19. The method according to claim 17, wherein the negotiation on the data characteristic algorithm between the client and the server comprises:

sending, by the client, a data package carrying the identification of a designated data characteristic algorithm to the server;

determining, by the server, the data characteristic algorithm corresponding to the data characteristic algorithm identification in the received data package, and returning the data package carrying the identification of the data characteristic algorithm designated by the client and confirmed by the server to the client.

20. The method according to claim 15, further comprising:

determining, by the client, a sync type to be initiated according to status of the client database, reporting to the server status information of the last data synchronization and the sync type to be initiated this time;

determining, by the server, whether the received sync type is adopted according to the received status information of the last data synchronization, and sending the determined sync type to the client; and

determining, by the client, whether the received sync type is a two-party computation smart sync, if the received sync type is the two-party computation smart sync, sending, by the client, the data characteristic verification message carrying one or multiple data characteristics for data in the client database, to the server;

otherwise, initiating, by the client, the sync determined in a negotiation between the client and the server, or restarting the negotiation between the client and the server.

21. The method according to claim 20, wherein the determining by the client the sync type to be initiated comprises:

determining, by the client, to employ the two-party computation smart sync if LUIDs of the data in the client database are out of order or/and regenerated or/and modification status information is lost; and

the determining by the server the sync type comprises;

determining, by the server, to employ the two-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization, or the mapping table in the server database is lost, or the data characteristics in the server database are lost.

22. The method according to claim 21, wherein the status information of the last data synchronization, which is sent by the client, comprises an Anchor information;

and

the detecting by the server that the last data synchronization failed comprises;

comparing, by the server, the Anchor from the client with the Anchor which was sent by the client in the last data synchronization and stored in the server database or generated by the server for the server database, if the Anchor from the client is identical with the Anchor on the server, determining that the last data synchronization succeeded, otherwise, determining that the last data synchronization failed.

23. The method according to claim 21, wherein the status information of the last data synchronization, which is sent by the client, comprises data characteristics generated for the client database through the computation based on the predetermined data characteristic algorithm;

and

the detecting by the server that the last data synchronization failed comprises;

comparing, by the server, the data characteristics of the client database from the client with the data characteristics of the server database;

if the data characteristics of the client database are identical with the data characteristics of the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

24. The method according to claim 23, wherein the generating data characteristics for the client database through computation based on the predetermined data characteristic algorithm comprises one of the following processes:

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics of all directories in the client database or a LUIDs of the data in the directories as inputted elements;

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics of all directories in the client database, or the data characteristic information of the data in the directory as the inputted elements; and

generating data characteristics of the client database through the computation based on the predetermined data characteristic algorithm with the data characteristics or the LUIDs of all data in the client database as inputted elements.

25. The method according to claim 15, wherein

if a LUIDs of the data characteristics in the client data base are out of order or regenerated while the modification status information in the client data base is not lost,

the data characteristics in the client database comprise the data characteristics of the to-be-synchronized data with changed modification status information, or the data characteristics of all to-be-synchronized data;

and

the comparing comprises;

comparing, by the server, all the data characteristics from the client with the data characteristics in the server database;

or

if the LUIDs of the data characteristics are not out of order or regenerated while the modification status information in the client data base is lost,

the data characteristics in the client database comprises the data characteristics of all to-be-synchronized data identified with LUIDs in the client database; and

the comparing comprises;

determining, by the server, corresponding data in the server database according to the one or multiple LUIDs of the data characteristics in the data characteristic verification message, and comparing the one or multiple data characteristics in the data characteristic verification message with the data characteristics of the determined corresponding data in the server database;

or

if the LUIDs of the data characteristics in the client data base are out of order or regenerated while the modification status information in the client data base is lost,

the data characteristics in the client database comprises the data characteristics of all to-be-synchronized data in the client database; and

the comparing comprises;

comparing, by the server, the data characteristics from the client with all data characteristics in the server database.

26. The method according to claim 25, wherein the synchronizing the determined data to be synchronized with the server comprises:

determining, by the client, the data to be sent to the server according to the information in the data characteristic verification response, and sending the determined data to the server so that the server can synchronize the data in the server database with the data received by the server; and

sending, by the server, the data to be sent to the client to the client according to comparison results, so that the client synchronizes the data in the client database with the data received by the client.

27. The method according to claim 26, wherein the data to be sent to the client comprises one type of the following data:

the data in the server database corresponding to the data characteristics that are different from the received data characteristics;

the data in the server database corresponding to the data characteristics that are identical with the received data characteristics, and including identification bit(s); and

the data in the server database corresponding to the data characteristics different from the received data characteristics, and to be sent to the client after conflict detection.

28. The method according to claim 26, further comprising:

sending, by the client, the data synchronization status information indicating a completion of the data synchronization to the server in a data synchronization completing confirmation message; and

comparing, by the server, the data synchronization status information in the data synchronization completing confirmation message with the data synchronization status information of corresponding data in the server database, wherein the data synchronization status information of corresponding data in the server database is sent by the client and stored by the server or is generated by the server through the computation;

if the data synchronization status information in the data synchronization completing confirmation message is identical with the data synchronization status information of corresponding data in the server database, returning a response by the server, otherwise, sending, by a client, the data characteristic verification message carrying one or multiple data characteristics set for data in the client database, to the server.

29. The method according to claim 1, wherein the one or multiple data characteristics for data in the client database comprise the data characteristics of the data with a present level in the client database;

and

the method further comprises;

determining, by the client, according to the information in the data characteristic verification response, the level of the data corresponding to the data characteristics that are not identical with the data characteristics in the server database;

setting the data with one level lower than the level of the data corresponding to the data characteristics that are not identical with the data characteristics in the server database as the data with the present level, returning to the process of sending by the client the data characteristic verification message carrying one or multiple data characteristics for data in the client database to the server, until the data with the lowest level in the client database have been processed, and proceeding to the process of determining by the client the data to be synchronized according to the information in the data characteristic verification response, and synchronizing the determined data to be synchronized with the server.

30. The method according to claim 1, wherein the information indicating the determined data to be synchronized comprises:

LUIDs of the data to be synchronized, or the data characteristics of the data to be synchronized in the client database, or status codes corresponding to the data characteristics of the data to be synchronized in the client database.

31. The method according to claim 1, wherein the data includes directories, files or data items.

32. The method according to claim 31, wherein when the data includes the directory, the data characteristics of the directory are set through computation based on the predetermined data characteristic algorithm with the data characteristics of the data in the directory or/and a LUIDs of the data under the directory as inputted elements;

when the data includes the file or the data item, the data characteristics of the file or data item are set through computation based on the predetermined data characteristic algorithm with contents of the file or the data item as inputted elements.

33. The method according to claim 1, wherein the data characteristics include data fingerprints.

34. The method according to claim 2, wherein the predetermined data characteristic algorithm includes an algorithm selected from the group consisting of:

a digest algorithm, a cyclic redundancy check algorithm, a normal hash algorithm, and a timestamp algorithm.

35. A data synchronization system, comprising:

a client, configured to be connected to a database for the client; and

a server, configured to synchronize data with the data on the client, and configured to be connected to the database for the server;

wherein

each of the client and the server comprises a synchronization module, a data processing module connected to the synchronization module and the database, and a data characteristic information processing module connected to the synchronization module,

the data processing module is configured to send the data in the database to the synchronization module, or receive a data synchronization message from the synchronization module, and modify or add data in the database based on synchronization data in the data synchronization message;

the data characteristic information processing module is configured to set up data characteristics for the data in the synchronization module, and send the data characteristics to the synchronization module; and

the synchronization module connected to the server or the client, is configured to compare the data characteristics of the data sent from the server or the client with the data characteristics set up by the data characteristic information processing module for corresponding data, send result of the comparison to the server or the client, receive the data synchronization message carrying the synchronization data from the server or the client, and forward the data synchronization message to the data processing module;

or the synchronization module is configured to send the data characteristic set up by the data characteristic information processing module to the server or the client, receive the result of the comparison from the server or the client, and send data synchronization message carrying synchronization data to the server or the client according to the result of the comparison, so that the server or the client synchronizes the data on the server or the client.

36. The data synchronization system according to claim 35, wherein the synchronization module is further connected to the server or the client by connecting the synchronization module in the server or the client.

37. The data synchronization system according to claim 35, wherein the client and the server are peers of each other in a data synchronization network.

38. A data synchronization client, comprising:

a database for the client, a synchronization module, a data characteristic information processing module connected to the synchronization module, and a data processing module connected to the synchronization module and the database for the client, wherein

the data processing module is configured to send data in the database to the synchronization module, or receive a data synchronization message from the synchronization module, and modify or add data in the database based on the synchronization data in the data synchronization message;

the data characteristic information processing module is configured to set up data characteristics for the data in the synchronization module, and send the data characteristics to the synchronization module; and

the synchronization module is configured to compare the data characteristics of the data sent from a server with the data characteristics set up by the data characteristic information processing module for corresponding data, send a result of the comparison to the server, receive the data synchronization message carrying the synchronization data from the server, and forward the data synchronization message to the data processing module;

or the synchronization module is configured to send the data characteristic set up by the data characteristic information processing module to the server, receive the result of the comparison from the server, and send data synchronization message carrying synchronization data to the server according to the result of the comparison so that the server can synchronize the data on the server.

39. The client according to claim 38, wherein the synchronization module is further configured to interact with a synchronization module in the server to exchange information.

40. A data synchronization server, comprising:

a database for the server, a synchronization module, a data characteristic information processing module connected to the synchronization module, and a data processing module connected to the synchronization module and the database for the server, wherein

the data processing module is configured to send data in the database to the synchronization module, or receive a data synchronization message from the synchronization module, and modify or add data in the database based on the synchronization data in the data synchronization message;

the data characteristic information processing module is configured to set up data characteristics for the data in the synchronization module, and send the data characteristics to the synchronization module; and

the synchronization module is configured to compare the data characteristics of the data sent from a client with the data characteristics set up by the data characteristic information processing module for corresponding data, send a result of the comparison to the client, receive the data synchronization message carrying the synchronization data from the client, and forward the data synchronization message to the data processing module;

or the synchronization module is configured to send the data characteristic set up by the data characteristic information processing module to the client, receive the result of the comparison from the client, and send the data synchronization message carrying synchronization data to the client according to the result of the comparison, so that the client synchronizes the data on the client.

41. The server according to claim 40, wherein the synchronization module is further configured to interact with a synchronization module in the client to exchange information.