Method and system for data synchronization, and apparatus thereof

• US 9,524,327 B2
• Filed: 12/16/2013
• Issued: 12/20/2016
• Est. Priority Date: 01/26/2006
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Kang, Jiao, Guo, Xiangzhou, Tian, Linyi
• Primary Examiner(s): Ashraf, Waseem
• Assistant Examiner(s): Hussain, Farrukh
• Application Number:
  US14/107,938
• Publication Number:
  US 20140108338A1
• Time in Patent Office: 1,100 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Embodiments of the present disclosure disclose a data synchronization method, system and apparatuses thereof. The data synchronization method includes the following processes: sending, by a client, a data characteristic verification message carrying one or multiple data characteristics for data in a client database, to a server; comparing, by the server, the one or multiple data characteristics carried in the data characteristic verification message with the data characteristics for the data in a server database; determining, by the server, the data to be synchronized based on a result of the comparing and sending the client a data characteristic verification response carrying information indicating the determined data to be synchronized. The method, system and apparatus thereof provided by the embodiments of the present disclosure can reduce the amount of data to be transmitted in data synchronization, and improve the efficiency of data synchronization.

Claims

1. A data synchronization method, comprising:

receiving, by a server, a data characteristic verification message that includes a plurality of Identifiers (IDs) with corresponding data fingerprints that each represent portions of data in a client database for synchronization with a server database, wherein each of the corresponding data fingerprints uniquely identify contents of the portions of data;

comparing, by the server, one or more of the corresponding data fingerprints included within the data characteristic verification message with one or more data fingerprints stored on the server database, wherein the data fingerprints stored on the server database represent the portions of data having the same IDs as the portions of data in the data characteristic verification message;

in accordance with the comparing, determining, by the server, at least one of the portions of data on the client database that needs to be synchronized with the server database; and

in accordance with the determining, sending, by the server, to the client a data characteristic verification response that includes at least one of the plurality of IDs corresponding to the at least one of the portions of data on the client database that needs to be synchronized, which indicates to the client what data to synchronize with the server database.

2. The method according to claim 1, further comprising:

receiving, by the server, the data fingerprints for the data in the server database from the client, and storing the received data fingerprints in the server, wherein the data fingerprints for the data is obtained, by the client, through computation in accordance with a predetermined data fingerprint algorithm.

3. The method according to claim 2, wherein the receiving, by the server, the data fingerprints for the data in the server database from the client, and storing the received data fingerprints in the server comprise one of the following processes:

if the client and the server negotiate to employ a one-party computation smart sync, receiving, by the server, the data fingerprints sent from the client in advance, and storing the received data fingerprints, or keeping, by the server, the data fingerprints sent from the client in a last data synchronization in one-party computation smart sync;

otherwise, generating, by the server, the data fingerprints through the computation in accordance with the data fingerprint algorithm determined in advance or determined by the client and the server in the negotiation; and

arranging, by the client and the server in advance, an agreement on employing one-party computation smart sync, wherein the agreement covers all data, data of all types in the client database and the server database, data of specified types, or data of specified size in the client database and server database.

4. The method according to claim 2, further comprising:

receiving status information of a last data synchronization and a sync type to be initiated this time from the client, wherein the sync type is determined according to the status information of the data in the client database;

determining, by the server, whether the received sync type is adopted according to the received status information of the last data synchronization in the client database, and sending the determined sync type to the client; and

receiving the data characteristic verification message carrying one or multiple data fingerprints for data in the client database from the client when the client determines that the received sync type is a one-party computation smart sync.

5. The method according to claim 4, wherein the determining by the server whether the received sync type is accepted according to the received status information of the last data synchronization in the client database further comprises:

determining to employ the one-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization and a mapping table and the data fingerprints in the server database are not lost; and

determining to employ the one-party computation smart sync, when the server detects that the mapping table in the server database is lost, or modification status information in the server database is lost, and the data fingerprints in the server database are not lost.

6. The method according to claim 5, wherein the status information of the last data synchronization comprises the data fingerprints generated for the client database through computation in accordance with the predetermined data fingerprint algorithm;

and

the detecting by the server the last data synchronization failed comprises;

comparing, by the server, the data fingerprints of the client database from the client with the data fingerprints of the client database which were sent by the client in the last data synchronization and stored in the server database; and

if the data fingerprints from the client are identical with the data fingerprints in the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

7. The method according to claim 6, wherein the generating data fingerprints for the client database through computation in accordance with the predetermined data fingerprint algorithm comprises one of the following processes:

generating data fingerprints of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database or LUIDs of the data in the directories as inputted elements;

generating data fingerprints of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database, or data fingerprint information of the data in the directory as the inputted elements; and

generating data fingerprint s of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprint s or the LUIDs of all data in the client database as inputted elements.

8. The method according to claim 2, wherein if the server modifies a piece of data in the server database, the server inserts an identification bit into the data fingerprints of the data, and stores the data fingerprints, and

the determining the data to be synchronized in accordance with a result of the comparing comprises:

determining that the data corresponding to the received data fingerprints need to be synchronized if the server database includes no data fingerprint identical with the received data fingerprints;

determining whether the identical data fingerprint s in the server database includes an identification bit, if the server database includes data fingerprint s identical with the received data fingerprints; and

determining that the data corresponding to the received data fingerprints need to be synchronized, if the data fingerprint s in the server database include the identification bit, otherwise, determining that the data corresponding to the identical data fingerprint s does not need to be synchronized.

9. The method according to claim 2, further comprising:

receiving, by the server, a data synchronization completing confirmation message from the server, wherein the data synchronization completing confirmation message comprises data synchronization status information indicating a completion of the data synchronization; and

storing, by the server, the data synchronization status information in the data synchronization completing confirmation message in the server database and returning a response message.

10. The method according to claim 1, further comprising:

obtaining, by the client, one or multiple data fingerprints through a real-time computation in accordance with a predetermined data fingerprint algorithm;

or obtaining, by the client, the one or multiple data fingerprints in advance through the computation in accordance with the predetermined data fingerprint algorithm, and storing the one or multiple data fingerprints in the client; and

setting, by the server, the data fingerprints for the data in the server database through the computation in accordance with the predetermined data fingerprint algorithm.

11. The method according to claim 1, further comprising:

negotiating, by the client and the server, that a two-party computation smart sync is employed, and the data fingerprints are set through computation in accordance with a predetermined data fingerprint algorithm; and

arranging, by the client and the server in advance, an agreement on employing the two-party computation smart sync, wherein the agreement covers all data in the client database and server database, or data of specified types in the client database and server database.

12. The method according to claim 1, wherein data fingerprint algorithms used in the client and the server are the same data fingerprint algorithm, and the data fingerprint algorithm is determined in advance, or determined after a negotiation between the client and the server.

13. The method according to claim 10, further comprising:

receiving, by the server, status information of a last data synchronization and a sync type to be initiated this time, from the client, wherein the sync type is determined according to status of the client database;

determining, by the server, whether the received sync type is adopted according to the received status information of the last data synchronization, and sending the determined sync type to the client; and

wherein the receiving, by a server, a data characteristic verification message comprises;

receiving, by a server, a data characteristic verification message that includes a plurality of Identifiers (IDs) with corresponding data fingerprints that each represent portions of data in a client database for synchronization with a server database, when the client determines that the received sync type is a two-party computation smart sync.

14. The method according to claim 13, the determining by the server the sync type comprises:

determining, by the server, to employ the two-party computation smart sync, when the server detects that the last data synchronization failed according to the status information of the last data synchronization, or a mapping table in the server database is lost, or the data fingerprints in the server database are lost.

15. The method according to claim 14, wherein the status information of the last data synchronization, which is sent by the client, comprises data fingerprints generated for the client database through the computation in accordance with the predetermined data fingerprint algorithm;

and

the detecting by the server that the last data synchronization failed comprises;

comparing, by the server, the data fingerprints of the client database from the client with the data fingerprints of the server database;

if the data fingerprints of the client database are identical with the data fingerprints of the server database, determining that the last data synchronization succeeded;

otherwise, determining that the last data synchronization failed.

16. The method according to claim 15, wherein the generating data fingerprints for the client database through computation in accordance with the predetermined data fingerprint algorithm comprises one of the following processes:

generating data fingerprints of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database or the IDs of the data in the directories as inputted elements;

generating data fingerprints of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprints of all directories in the client database, or data fingerprint information of the data in the directory as the inputted elements; and

generating data fingerprints of the client database through the computation in accordance with the predetermined data fingerprint algorithm with the data fingerprints or the IDs of all data in the client database as inputted elements.

17. The method according to claim 1, wherein information indicating the determined data to be synchronized comprises:

IDs of the data to be synchronized, or the data fingerprints of the data to be synchronized in the client database, or status codes corresponding to the data fingerprints of the data to be synchronized in the client database.

18. The method according to claim 1, wherein the data includes directories, files or data items;

when the data includes the directory, the data fingerprints of the directory are set through computation in accordance with a predetermined data fingerprint algorithm with the data fingerprints of the data in the directory or the IDs of the data under the directory as inputted elements; and

when the data includes the file or the data item, the data fingerprints of the file or data item are set through computation in accordance with the predetermined data fingerprint algorithm with contents of the file or the data item as inputted elements.

19. The method according to claim 1, wherein the data fingerprint is obtained by computing the data in accordance with a preset data fingerprint algorithm or a data fingerprint algorithm negotiated by the client and the server, and when the data is modified a new data fingerprint is generated and the data fingerprint is updated by using the new data fingerprint.

20. The method according to claim 2, wherein the predetermined data fingerprint algorithm includes one of the following algorithm:

a digest algorithm, a cyclic redundancy check algorithm, a normal hash algorithm, and a timestamp algorithm.

21. A computer program product for use in a server used to synchronize a client database, wherein the computer program product comprises one or more computer executable instructions stored on a non-transitory computer readable medium such that when executed by a processor cause the server to perform the following:

receiving, by a server, a data characteristic verification message that includes a plurality of Identifiers (IDs) with corresponding data fingerprints that each represent portions of data in a client database for synchronization with a server database, wherein each of the corresponding data fingerprints uniquely identify contents of the portions of data;

comparing, by the server, one or more of the corresponding data fingerprints included within the data characteristic verification message with one or more data fingerprints stored on the server database, wherein the data fingerprints stored on the server database represent the portions of data having the same IDs as the portions of data in the data characteristic verification message;

in accordance with the comparing, determining, by the server, at least one of the portions of data on the client database that needs to be synchronized with the server database; and

in accordance with the determination, sending, by the server, to the client a data characteristic verification response that includes at least one of the plurality of IDs corresponding to the at least one of the portions of data on the client database that needs to be synchronized, which indicates to the client what data to synchronize with the server database.