Systems and methods for backing up data files

• US 6,625,623 B1
• Filed: 12/16/1999
• Issued: 09/23/2003
• Est. Priority Date: 12/16/1999
• Status: Expired due to Term

• Current Assignee: Keepitsafe
• Original Assignee: LiveVault Corp. (Iron Mountain Incorporated)
• Inventors: Gonsalves, Manuel, Webb, John, Midgley, Christopher, Hess, Ted, Acott, John
• Primary Examiner(s): Corrielus, Jean M.
• Assistant Examiner(s): ALAM, SHAHID AL
• Application Number:
  US09/465,408
• Time in Patent Office: 1,377 Days
• Field of Search: 707/2, 707/8, 707/202, 707/203, 707/204, 707/10, 705/2, 705/3, 714/5, 714/6, 714/4, 715/511, 709/236, 711/162, 713/176
• US Class Current: 707/640
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

The invention provides systems and methods for continuous back up of data stored on a computer network. To this end the systems of the invention include a synchronization process that replicates selected source data files data stored on the network and to create a corresponding set of replicated data files, called the target data files, that are stored on a back up server. This synchronization process builds a baseline data structure of target data files. In parallel to this synchronization process, the system includes a dynamic replication process that includes a plurality of agents, each of which monitors a portion of the source data files to detect and capture, at the byte-level, changes to the source data files. Each agent may record the changes to a respective journal file, and as the dynamic replication process detects that the journal files contain data, the journal files are transferred or copied to the back up server so that the captured changes can be written to the appropriate ones of the target data files.

Claims

1. A system for backing-up at least one source data file, the system comprising:

a back-up server having storage for at least one backup file corresponding to the at least one source data file,

a data server having

storage for the at least one source data file, and

an agent for monitoring file access operations associated with the at least one source data file and for recording detected changes to the at least one source data file within at least one journal file corresponding to the at least one source data file,

a synchronization replication process for replicating the at least one source data file to generate at least one backup file corresponding to the at least one source data file, and

a dynamic replication process responsive to data within the at least one journal file for modifying the at least one back-up file based on the corresponding at least one journal file.

2. A system according to claim 1, wherein the agent monitors file access operations associated with the at least one source data file for determining whether the at least one source data file is open.

3. A system according to claim 1, wherein the agent monitors at least one of file input calls and file output calls to an operating system.

4. A system according to claim 1, wherein the agent monitors file access operations for recording byte-level modifications to the at least one source data file.

5. A system according to claim 1, wherein byte-level modifications to the at least one source data file are recorded in the corresponding at least one journal file.

6. A system according to claim 1, wherein the agent records at least one time of change associated with at least one of the detected changes in the at least one journal file.

7. A system according to claim 1, wherein the synchronization replication process generates an image signal representative of a state of at least one of the at least one source data file and the corresponding at least one back-up file.

8. A system according to claim 1, wherein the synchronization replication process generates an image signal representative of a state of a directory structure associated with at least one source data file and the corresponding at least one back-up file.

9. A system according to claim 1, wherein the synchronization replication process generates a block image signal representative of a state of at least one block of data within at least one source data file.

10. A system according to claim 1, wherein the dynamic replication process monitors the at least one journal file for changes to the corresponding at least one source data file.

11. A system according to claim 1, wherein the dynamic replication process writes the changes recorded in the at least one journal file to the corresponding at least one back-up file.

12. A system according to claim 1, wherein the dynamic replication process modifies at least one back-up file based on a time of change associated with the recorded changes in the corresponding at least one journal file.

13. A system according to claim 1, further comprising a policy manager for allowing a user to identify at least one source data file to be monitored by the synchronization replication process and the dynamic replication process.

14. A system for backing-up at least one source data file, the system comprising:

a back-up server having storage for at least one back-up file,

a data server having

storage for the at least one source data file, and

an agent for monitoring file access operations associated with the at least one source data file and for recording detected changes to the at least one source data file within at least one journal file corresponding to the at least one source data file,

a console process for allowing a user to select at least one source data file for back-up,

a synchronization replication process for replicating the at least one source data file to generate at least one back-up file corresponding to the at least one source data file, and

a dynamic replication process responsive to data within the at least one journal file for modifying the at least one back-up file based on the corresponding at least one journal file.

15. A system according to claim 14, wherein the console process allows the user to generate a profile representative of the at least one source data file.

16. A system according to claim 14, wherein the console process allows the user to control the timing of the synchronization replication process.

17. A system according to claim 14, wherein the console process allows the user to control the timing of the dynamic replication process.

18. A system according to claim 14, wherein the console process allows the user to select at least one source data file for processing by the synchronization replication process.

19. A system according to claim 14, wherein the console process allows the user to exclude at least one source data file from processing by the synchronization replication process.

20. A system according to claim 14, wherein the console process directs the synchronization replication process to replicate at least one source data file by performing at least one of

regenerating the at least one source data file, and

modifying the corresponding at least one back-up file to include the changes recorded in the corresponding at least one journal file.

21. A system according to claim 14, further including

a transactional process for allowing the user to identify at least two source data files as members of a transactional group and for directing the dynamic replication process to modify the back-up files corresponding to the members of the transactional group in a unitary operation.

22. A process for creating back-up files for source data files stored on a data server on a computer network, the process comprising:

selecting at least one of the source data files for back-up,

synchronizing the source data files on the data server with corresponding target data files stored on a back-up server by replicating each of the source data files as a corresponding target file on the back-up server,

for each source data file, monitoring file access operations associated with the source data file and recording detected changes to the source data file within a corresponding journal file, and

for each source data file, writing changes recorded in the corresponding journal file to the corresponding target data file.

23. A process according to claim 22, wherein selecting at least one of the source data files includes identifying at least two of the source data files as members of a transactional group, and identifying a status representative of the back-up state of each of the members.

24. A process according to claim 22, wherein synchronizing the source data files includes generating an image signal representative of a state of the source data files.

25. A process according to claim 24, wherein generating an image signal includes applying a cylic redundancy check (CRC) process to metadata associated with the source data files.

26. A process according to claim 25, wherein the metadata includes at least one of a size of a source data file, time information associated with a source data file, date information associated with a source data file, and an open status of a source data file.

27. A process according to claim 22, wherein synchronizing includes generating an image signal for a data structure representative of at least a portion of at least one directory maintained on the data server, and comparing the image signal to a corresponding image signal associated with the backup server to detect a change in the state of the data structure.

28. A process according to claim 22, wherein synchronizing includes generating an image signal for a source data file associated with a directory, and comparing the image signal to a corresponding image signal associated with the backup server to detect a change in the state of at least one of the source data files.

29. A process according to claim 22, wherein synchronizing includes generating a block image signal for at least one block of data associated with a source data file, and comparing the block image signal to a corresponding block image signal associated with the backup server to detect a change in the state of the source data file.

30. A process according to claim 29, including transmitting to the back-up server a journal file corresponding to the source data file, in response to detecting a change in the state of the source data file, thereby allowing changes recorded in the journal file to be written to the corresponding target data file.

31. A process according to claim 29, including transmitting to the back-up server the source data file, in response to detecting a change in the state of the source data file, thereby allowing the corresponding target data file to be replaced with the source data file.

32. A process according to claim 22, wherein recording detected changes within the journal file includes recording at least one time of change associated with at least one of the detected changes.

33. A process according to claim 22, wherein recording detected changes within the journal file includes recording in the journal file, in response to a predetermined event, at least one safe point for indicating at least one detected change can be written to the corresponding target data file.

34. A process according to claim 33, including monitoring access to the corresponding source data file to record at least one safe point in the journal file based on a closed file status associated with the source data file.

35. A process according to claim 33, including monitoring access to the corresponding source data file to record at least one safe point in the journal file at a predetermined period of time after a time of change associated with at least one detected change recorded in the journal file.

36. A process according to claim 33, further comprising recording the corresponding target data file to a long term storage medium in response to a safe point indicating at least one change can be written to the corresponding target data file.

37. A process according to claim 22, including detecting an open file status for a source data file and closing the source data file for synchronizing data contained in the source data file.

38. A process according to claim 22, further including monitoring file access operations associated with a source data file to identify the source data file as a file for back-up, and responsive thereto, generating a corresponding target data file.

39. A process according to claim 38, wherein monitoring the file access operations includes monitoring the file access operations to identify a file rename operation.

40. A method for backing-up at least one source data file, the method comprising:

synchronizing the at least one source data file and at least one back-up file corresponding to the at least one source data file,

monitoring file access operations associated with the at least one source data file,

based on changes detected in the at least one source data file, generating a journal file corresponding to the at least one source data file, and

modifying the corresponding at least one back-up file based on the corresponding at least one journal file.

41. A method according to claim 40, wherein synchronizing the at least one source data file and at least one back-up file corresponding to the at least one source data file comprises:

replicating the at least one source data file to generate the corresponding at least one back-up file.

42. A method according to claim 40, wherein synchronizing the at least one source data file and at least one back-up file corresponding to the at least one source data file comprises:

generating an image signal representative of a state associated with the at least one source data file.

43. A method according to claim 42, herein generating an image signal comprises:

applying a cyclic redundancy check (CRC) process to metadata associated with the at least one source data file.

44. A method according to claim 43, wherein the metadata include at least one of a file size, a closed file status, an open file status, and a write lock status associated with the at least one source data file.

45. A method according to claim 40, wherein synchronizing the at least one source data file and at least one back-up file corresponding to the at least one source data file comprises:

generating a block image signal representative of a state of at least one block of data associated with the at least one source data file.

46. A method according to claim 40, wherein synchronizing the at least one source data file and at least one back-up file corresponding to the at least one source data file comprises:

generating a source data file image signal representative of a state of the at least one source data file,

generating a back-up file image signal representative of a state of the corresponding at least one source data file, and

based on comparing the source data file image signal and the back-up file image signal, detecting a change in the state of the at least one source data file.

47. A method according to claim 40, further comprising:

identifying at least two source data files as members of a transactional group.

48. A method according to claim 47, wherein identifying at least two source data files as members of a transactional group comprises:

recording, in the journal files corresponding to the members of the transactional group, safepoints identifying the at least two source data files as members of the transactional group.

49. A method according to claim 47, wherein modifying the corresponding at least one back-up file based on the corresponding at least one journal file comprises:

modifying the back-up files corresponding to the members of the transactional group based on the corresponding journal files in a unitary operation.

50. A method according to claim 40, further comprising:

recording, in the corresponding at least one journal file, at least one safe point based on a a time of change associated with at last one of the changes detected in the at least one source data file.

51. A method according to claim 50, wherein recording at least one safe point based on a time of change comprises:

recording at least one safe point at a predetermined period of time after the time of change.

52. A method according to claim 40, further comprising:

allowing a user to control at least one of the timing of the synchronization replication process and the at least one source data file to be processed by the synchronization replication process.

53. A processor program for backing-up at least one source data file, the processor program being stored on a machine-readable medium and comprising instructions to cause a processor to:

synchronize the at least one source data file and at least one back-up file corresponding to the at least one source data file,

monitor file access operations associated with the at least one source data file,

based on changes detected in the at least one source data file, generate a journal file corresponding to the at least one source data file, and

modify the corresponding at least one back-up file based on the corresponding at least one journal file.