System and method for replicating files in a computer network

• US 20050177603A1
• Filed: 01/05/2005
• Published: 08/11/2005
• Est. Priority Date: 02/06/2004
• Status: Active Grant

• Current Assignee: Aviall Incorporated (The Boeing Co.)
• Original Assignee: Aviall Incorporated (The Boeing Co.)
• Inventors: Shavit, Tsachi Chuck
• Granted Patent: US 7,685,384 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques

Abstract

A system and method for performing real-time replication of data across a network is provided. A mirroring engine receives a write request from a host application operating on a source computer. The mirroring engine compares data in the write request with corresponding data stored in memory. If data in the write request differs from stored data, the mirroring engine processes the write request. Processing involves computing a data signature across data in the write request and associating the signature with a transaction number and a status byte. The transaction number is used to uniquely identify the data signature and can be used to ensure that the signature is properly handled if it is received, for example, out of order. The status byte contains information used for handling the data signature and transaction number as well as information identifying how the data signature was computed. In an embodiment, the status byte may contain a bit for identifying if the data signature was computed across an entire block of data or if the signature was computed over only a portion of a data block, such as for example, only data bytes that have changed as compared with those in the stored data. Processing may further include compressing and encrypting the data signature, transaction number and status byte prior to transmission over a network. Embodiments may further employ a meta-file for storing data signatures, transaction numbers, status bytes, and other information useful for performing real-time data replication over a network.

Claims

1. A method for replicating data comprising the steps of:

operating an application in conjunction with a primary file server, said application further outputting a write operation;

receiving said write operation and comparing said data to stored data associated with a primary file;

generating an index referencing a changed data byte within said file, wherein said changed data byte occurs when a data byte within said data does not substantially match a corresponding stored data byte in said stored data;

providing said index to a meta-file for use in replicating said changed data byte; and

providing a mirroring engine operatively associated with said meta-file and utilizing a transaction number to uniquely identify said changed data byte.

2. The method of claim 1 further comprising the step of:

grouping said changed data byte with a plurality of other changed data bytes producing a data block.

3. The method of claim 2 wherein said mirroring engine is associated with queuing at least a portion of said data block, compressing at least a portion of said data block and transmitting at least a portion of said data block to a remote server using a network.

4. The method of claim 3 wherein said mirroring engine is further associated with encrypting at least a portion of said data block.

5. The method of claim 1 wherein said meta-file comprises a meta-header portion, a status portion and a signature portion.

6. The method of claim 5 wherein said meta-header portion includes a file-status value for use in determining if said meta-file was properly closed.

7. The method of claim 5 wherein said signature portion represents said changed byte.

8. The method of claim 7 wherein said signature portion is determined using a hash function.

9. The method of claim 8 wherein said hash function is a cyclic redundancy check function.

10. The method of claim 9 wherein said data block comprises a determined number of bytes.

11. The method of claim 5 wherein said status portion is comprised of substantially eight bits and at least a portion of said bits are used to represent information associated with a member selected from the group consisting of a dirty bit, an absolute signature of said data block and a cumulative signature of said data block.

12. The method of claim 11 wherein said absolute signature is obtained by operating a hash function against the entire data block.

13. The method of claim 11 wherein said cumulative signature is obtained by operating a hash function against a portion of said data block containing said changed data byte.

14. The method of claim 5 wherein said signature portion comprises a block signature portion and a transaction number portion, said block signature portion representing said changed byte and said transaction number portion uniquely identifying said data block and a time when said data block was written relative to other write operations associated with said data block.

15. A method for replicating data across a communications network comprising the steps of:

opening a meta-file and determining if said meta-file was properly closed, said meta-file including a block signature representing a data block associated with a source computer, and further including a transaction number for facilitating identification of said data block;

establishing data communication with a remote computer having replicated data residing thereon using said communications network;

receiving a remote meta-file file from said remote computer, said remote meta-file including a remote block signature and a remote transaction number; and

comparing said meta-file to said remote meta-file for determining if said data was properly replicated from said source computer to said remote computer.

16. The method of claim 15 further comprising the steps of:

building an array including at least said block signature if said meta-file was properly closed; and

re-computing said block signature if said meta-file was not properly closed.

17. The method of claim 16 further comprising the step of:

setting a dirty bit in said block signature if said meta-file was properly closed.

18. The method of claim 17 further comprising the step of:

setting said block signature to an absolute value if said block signature is non-dirty and a data signature portion of said block signature has a cumulative value.

19. The method of claim 18 wherein said comparing step further includes comparing said remote block signature to said block signature if said block signature is non-dirty.

20. The method of claim 19 further comprising the steps of:

setting said block signature to dirty if said transaction number and said remote transaction numbers do not match; and

transmitting said block signature to said remote computer if said signature is dirty.

21. The method of claim 20 further comprising the steps of:

writing said block signature to a computer-readable storage device operatively associated with said remote computer;

updating said transaction number using a transaction number counter to produce an updated transaction number; and

sending an updated remote block signature and said updated transaction number to said source computer using said communications network.

22. The method of claim 21 further comprising the steps of:

setting said updated remote block signature to non-dirty; and

enabling data writes on said source computer.

23. A method for replicating data across a communications network comprising the steps of:

receiving a data packet over said network, said packet having one or more primary file changes respectively represented by changed data, said packet further having a data signature representing a changed data byte within a data block associated with a source computer, and further having a transaction number associated therewith for uniquely identifying said changed byte;

computing a remote data signature for a remote data block residing in a memory, said remote data block further associated with a remote computer; and

determining if said data signature and said remote data signature match, wherein a match indicates that the content of said data block is substantially identical to the content of said remote data block.

24. The method of claim 23 wherein said data packet is associated with a meta-file and said remote data signature is associated with a remote meta-file.

25. The method of claim 24 further comprising the step of:

updating said remote meta-file if said data signature does not match said remote data signature, said updating further including copying said remote data signature into a signature portion of said remote meta-file, said updating further including incrementing said transaction number to produce an incremented transaction number and copying said incremented transaction number into a transaction portion of said remote meta-file.

26. The method of claim 25 further comprising the step of:

comparing said transaction number to a remote transaction number associated with said remote data block.

27. The method of claim 26 further comprising the step of:

generating an error message if said transaction number does not match said remote transaction number.

28. The method of claim 26 further comprising the step of:

generating an error message if said block signature does not match said remote data signature.

29. The method of claim 26 further comprising the step of:

sending an error message to said source computer, said error message causing said source computer to perform an error recovery function.

30. The method of claim 26 further comprising the steps of:

decrypting said data packet; and

decompressing said data packet.

31. A system for replicating data across a network, the system comprising:

a primary server comprising;

a communication interface for providing a data packet comprising at least a portion of a new data block to said network;

a storage device having stored data thereon, said stored data including a plurality of stored data blocks; and

a processor executing machine-readable instructions for receiving said new data block from a host application, said processor further comparing said new data block to a corresponding portion of said stored data, said processor further interacting with a meta-data file after processing said new data block using a hash function; and

a remote server communicatively coupled to said network for receiving said data packet over said network.

32. The system of claim 31 wherein said new data block is associated with said meta-file.

33. The system of claim 31 wherein said meta-file comprises a meta-header portion, a signature portion and a status portion.

34. The system of claim 33 wherein said signature portion further comprises:

a data signature representing said new data block; and

a transaction number for uniquely identifying said new data block.

35. The system of claim 33 wherein said signature portion comprises:

a data signature representing a changed byte associated with said new data block, wherein said changed byte occurs when a data byte within said new data block does not substantially match a corresponding stored data byte within said stored data block; and

a transaction number for uniquely identifying the order of said changed data byte with respect to changes associated with other data bytes in a primary file.

36. The system of claim 34 wherein said data block is of a determined size.

37. The system of claim 35 wherein said data signature is generated by processing said new data block using said hash function.

38. The system of claim 37 wherein said remote server generates said transaction number.

39. The system of claim 37 wherein said primary server further comprises:

a queue for holding said data signature prior to providing said data packet to said network.

40. A computer-readable data transmission for facilitating data replication across a data communications network, said data transmission comprising:

a file open status indicator;

a header portion having information useful for determining if a meta-file was properly closed;

a block temporal signature portion having a data block signature and a transaction number; and

a status portion having information about said data signature.

41. The computer-readable data transmission of claim 40 further comprising:

a time stamp.

42. The computer-readable data transmission of claim 41 wherein said transaction number is generated using a remote server.

43. The computer-readable data transmission of claim 42 wherein said header portion and said data block signature are each of a respective first determined size, said transaction number is of a second determined size and said status portion is of a third determined size.

44. The computer-readable data transmission of claim 42 wherein said data block signature is generated using a hash function and wherein said header portion, said block temporal portion, and said status portion are compressed and encrypted prior to being transmitted across said data communications network.

45. The computer-readable data transmission of claim 44 wherein said data block signature is computed across changed data bytes within said data block, wherein a changed data byte occurs when a data byte within said data block does not substantially match a corresponding stored data byte within a corresponding stored data block associated with a source computer.

46. The computer-readable data transmission of claim 45 wherein said status portion includes a dirty bit if said data communications network is not available for conveying said data transmission from a source computer to a remote computer.

47. The computer-readable data transmission of claim 45 wherein said transaction number facilitates identification of the order of data writes associated with said source computer and further discourages writing changed data bytes onto said remote computer in an order differing from the order that said changed data bytes were written on said source computer.

48. The computer-readable data transmission of claim 47 wherein said changed data bytes are asynchronously written with respect to writing data on said source computer.

49. The computer-readable data transmission of claim 48 wherein a host application generating said changed bytes is not modified when cooperatively operating in conjunction with said remote computer.

50. The computer-readable data transmission of claim 49 wherein said transmission is associated with a plurality of changed data bytes associated with a plurality of files.

51. The computer-readable data transmission of claim 50 wherein said plurality of changed data bytes are associated with an ordering on said source computer and said order is further maintained when said changed data bytes are written to said remote computer.