Flexible remote data transfer and data synchronization

• US 20040039889A1
• Filed: 08/20/2002
• Published: 02/26/2004
• Est. Priority Date: 08/20/2002
• Status: Active Grant

• Current Assignee: Gpmicro Incorporated
• Original Assignee: Gruintine Pueche Inc.
• Inventors: Elder, Kevin, Hemmers, Oliver
• Granted Patent: US 7,047,377 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2079 Bidirectional techniques
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A flexible data transfer and data synchronization system is described. Local and geographically remote systems comprise a data processing network. Each remote and local system includes a data storage facility. The local system backs up, mirrors and/or synchronizes its data to one or several geographically remote systems. During normal operation the data is transferred between the local system and remote systems in an asynchronous manner. According to one aspect of the invention, should the local system be out of synchronization for any reason with the geographically remote system the two systems will automatically re-synchronize without interruption to local or remote system users. This includes the initial data transfer between the local and geographically remote system. Embodiments of the invention are based on the interception and queuing of file system events as they occur, rather than the interception and transmittal of actual data disk writes. In this way, embodiments of the invention it does not require temporary storage of the physical data disk writes themselves in a separate, possibly volatile buffer system.

Claims

1. A machine-readable medium having a set of instructions to cause a machine to perform a method for transferring data to a mirrored file system comprising:

receiving an indication of a write command to a data file on a first file system on a first server;

saving the indication of the write command as an entry in a queue file, the queue file configured to store one or more entries of processed file system commands to the first file system; and

sending a copy of the data file to a second server based on one of the entries in the queue file, wherein the copied data file is to be copied from the data file on the first file system.

2. The machine-readable medium of claim 1 wherein the sending further comprises searching the queue file to select the entry associated with the write command to the data file on the first file system to be copied and sent to the second server.

3. The machine-readable medium of claim 2 wherein the searching the queue file to select the entry of the write command to the data file on the first file system further comprises removing duplicate entries of the write command to the data file from the queue file.

4. The machine-readable medium of claim 2 wherein the selecting an entry in the queue file further comprises selecting the entry based on the size of the data file.

5. The machine-readable medium of claim 2 wherein the selecting an entry in the queue file further comprises selecting the entry based a first-in-first-out methodology.

6. The machine-readable medium of claim 1 further comprises sending the copy of the data file to the second server upon determining whether the second server is ready to receive the data file.

7. The machine-readable medium of claim 1 wherein the sending further comprises sending the copy of the data file via one of a plurality of communication threads.

8. A computer system comprising:

a Dynamically Loadable Mirroring Module (DLMM) to intercept file system commands to a file system including a write command for a data file in the file system;

a File System Event Logger (FSEL) to select the file system commands intercepted by the DLMM to be placed in a queue file; and

a Data Transfer Agent (DTA) to send the selected file system commands from the queue file to a remote file system, wherein the DTA is to generate a copy of the data file to be sent to the remote file system based on the write command entry in the queue file.

9. The system of claim 8 further comprising a Data Receive Agent (DRA) to receive the file system commands on the remote file system.

10. The system of claim 8 wherein the queue is to be prioritized based on the size of the data file to be sent.

11. The system of claim 8 wherein duplicate write command entries for the data file are removed from the queue file.

12. The system of claim 8 wherein the DTA sends the data file via one of a plurality of communication threads.

13. The system of claim 8 wherein the DTA sends the selected file system commands and the DRA receives the file system commands via an Internet communications connection.

14. The system of claim 13 wherein the communications connection is authenticated via an auth_string.

15. A computer system comprising:

a file system command intercepting means to intercept file system commands made to a file system including a write command to a data file in the file system, the file system means to generate an entry for each intercepted file system command describing the intercepted file system commands to a buffer;

a transfer means to transmit the entry associated with the file system commands from the buffer to a remote file system, wherein the transfer means is to generate a copy of the data file to be sent to the remote file system based on a write command entry in the buffer.

16. The system of claim 15 wherein the transfer means is to prioritize the transfer of the copy of the data file based on the size of the data file to be sent.

17. The system of claim 15 wherein duplicate write command entries associated with the data file are removed from the buffer.

18. The system of claim 15 wherein the transfer means sends the data file via one of a plurality of communication threads.

19. The system of claim 15 further comprising:

a selecting means to select the file system commands to be intercepted by the intercepting means to be placed in a buffer.