Management of data replication and storage apparatuses, methods and systems

• US 10,055,424 B2
• Filed: 01/31/2017
• Issued: 08/21/2018
• Est. Priority Date: 05/30/2014
• Status: Active Grant

• Current Assignee: Datto LLC
• Original Assignee: Datto, Inc. (Kaseya Limited)
• Inventors: Mortensen, Spencer, Giacobbi, Justin
• Primary Examiner(s): Le, Hung
• Application Number:
  US15/420,506
• Publication Number:
  US 20170139950A1
• Time in Patent Office: 567 Days
• Field of Search: None
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

The MANAGEMENT OF DATA REPLICATION AND STORAGE APPARATUSES, METHODS AND SYSTEMS (“BDR”) transform machine datasets using BDR components into replications over a network distributing redundant ready-to-use backup files. In some implementations, the disclosure provides a processor-implemented method for managing the replication and storage of data sets.

Claims

1. A processor implemented method for managing the replication and storage of data sets comprising:

sending a backup request according to a backup and snapshot schedule;

receiving at a host of a first file system a backup response comprising image enabled backup data reflecting the state and content of a computer device;

capturing a snapshot of the image enabled backup data, and storing the snapshot in the first file system;

validating according to a remote backup schedule a vector comprising a data set indicator corresponding to the computer device, and a mirroring relationship between the first file system and a second file system;

sending a task processing assistance request to a master server comprising a vector identifier, wherein the task processing assistance request is sent according to a first prioritization order comprising a user'"'"'s defined prioritized replication order and a second prioritization order comprising a prioritized actions'"'"' execution order; and

synchronizing the first file system with the second file system only after receiving a task processing assistance response from the master server validating the mirroring relationship.

2. The method of claim 1, wherein the said received image enabled backup data is selected from the group consisting of full backup data, differential backup data and incremental backup data.

3. The method of claim 1, wherein the image enabled backup data is stored in the first file system and the captured snapshot is associated with the time when the image enabled backup data was generated.

4. The method of claim 3, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

5. The method of claim 4, wherein the host of the first file system is different from a host of the second file system.

6. The method of claim 5, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

7. The method of claim 6, further comprising a step wherein said master server sends an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

8. The method of claim 7, further comprising a step wherein said master server receives an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

9. The method of claim 8, further comprising a step wherein said master server receives a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

10. The method of claim 9, further comprising a step wherein said master server sends an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

11. The method of claim 10, wherein a synchronization request generated by the host of the first file system is declined by the host of the second file system.

12. The method of claim 11, further comprising a step wherein said master server sends an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from a remote backup schedule.

13. The method of claim 3, further comprising generating a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and storing the rapid recovery file in the first file system.

14. The method of claim 13, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.

15. The method of claim 1, wherein the captured snapshot comprises an incremental relation mirroring data blocks that have been modified in the computer system from the time when a previously received image enabled backup data was generated to the time when the current image enabled backup data was generated, and associating the snapshot with the time when the current image enabled backup data was generated.

16. The method of claim 15, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

17. The method of claim 16, wherein the host of the first file system is different from a host of the second file system.

18. The method of claim 17, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

19. The method of claim 18, further comprising a step wherein said master server sends an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

20. The method of claim 19, further comprising a step wherein said master server receives an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

21. The method of claim 20, further comprising a step wherein said master server receives a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

22. The method of claim 21, further comprising a step wherein said master server sends an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

23. The method of claim 22, wherein a synchronization request generated by the host of the first file system is declined by the host of the second file system.

24. The method of claim 23, further comprising a step wherein said master server sends an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from the remote backup schedule.

25. The method of claim 15, further comprising generating a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and storing the rapid recovery file in the first file system.

26. The method of claim 25, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.

27. A system for managing the replication and storage of data sets comprising:

a memory storing computer-executable instructions; and

one or more processors configured to implement a host of a first file system, the one or more processors being further configured to execute the instructions to;

send a backup request according to a backup and snapshot schedule;

receive a backup response comprising image enabled backup data reflecting the state and content of a computer device;

capture a snapshot of the image enabled backup data, and store the snapshot in the first file system;

validate according to a remote backup schedule a vector comprising a data set indicator corresponding to the computer device, and a mirroring relationship between the first file system and a second file system;

send a task processing assistance request to a master server comprising a vector identifier, wherein the task processing assistance request is sent according to a first prioritization order comprising a user'"'"'s defined prioritized replication order and a second prioritization order comprising a prioritized actions'"'"' execution order; and

synchronize the first file system with the second file system only after receiving a task processing assistance response from the master server validating the mirroring relationship.

28. The system of claim 27, wherein the said received image enabled backup data is selected from the group consisting of full backup data, differential backup data and incremental backup data.

29. The system of claim 27, wherein the image enabled backup data is stored in the first file system and the captured snapshot is associated with the time when the image enabled backup data was generated.

30. The system of claim 29, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

31. The system of claim 30, wherein the host of the first file system is different from a host of the second file system.

32. The system of claim 31, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

33. The system of claim 32, wherein said master server is configured to send an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

34. The system of claim 33, wherein said master server is configured to receive an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

35. The system of claim 34, wherein said master server is configured to receive a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

36. The system of claim 35, wherein said master server is configured to send an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

37. The system of claim 36, wherein the second file system is configured to decline a synchronization request generated by the host of the first file system.

38. The system of claim 37, wherein said master server is configured to send an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from a remote backup schedule.

39. The system of claim 29, wherein the one or more processors are further configured to generate a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and store the rapid recovery file in the first file system.

40. The system of claim 39, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.

41. The system of claim 27, wherein the captured snapshot comprises an incremental relation mirroring data blocks that have been modified in the computer system from the time when a previously received image enabled backup data was generated to the time when the current image enabled backup data was generated, and wherein the one or more processors are further configured to associate the snapshot with the time when the current image enabled backup data was generated.

42. The system of claim 41, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

43. The system of claim 42, wherein the host of the first file system is different from a host of the second file system.

44. The system of claim 43, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

45. The system of claim 44, wherein said master server is configured to send an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

46. The system of claim 45, wherein said master server is configured to receive an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

47. The system of claim 46, wherein said master server is configured to receive a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

48. The system of claim 47, wherein said master server is configured to send an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

49. The system of claim 48, wherein the second file system is configured to decline a synchronization request generated by the host of the first file system.

50. The system of claim 49, wherein said master server is configured to send an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from the remote backup schedule.

51. The system of claim 41, wherein the one or more processors are further configured to execute the instructions to generate a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and store the rapid recovery file in the first file system.

52. The system of claim 51, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.

53. Non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors configured to implement a host of a first file system, are operable to cause the one or more processors to:

send a backup request according to a backup and snapshot schedule;

receive a backup response comprising image enabled backup data reflecting the state and content of a computer device;

capture a snapshot of the image enabled backup data, and store the snapshot in the first file system;

validate according to a remote backup schedule a vector comprising a data set indicator corresponding to the computer device, and a mirroring relationship between the first file system and a second file system;

send a task processing assistance request to a master server comprising a vector identifier, wherein the task processing assistance request is sent according to a first prioritization order comprising a user'"'"'s defined prioritized replication order and a second prioritization order comprising a prioritized actions'"'"' execution order; and

synchronize the first file system with the second file system only after receiving a task processing assistance response from the master server validating the mirroring relationship.

54. The non-transitory computer-readable media of claim 53, wherein the said received image enabled backup data is selected from the group consisting of full backup data, differential backup data and incremental backup data.

55. The non-transitory computer-readable media of claim 53, wherein the image enabled backup data is stored in the first file system and the captured snapshot is associated with the time when the image enabled backup data was generated.

56. The non-transitory computer-readable media of claim 55, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

57. The non-transitory computer-readable media of claim 56, wherein the host of the first file system is different from a host of the second file system.

58. The non-transitory computer-readable media of claim 57, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

59. The non-transitory computer-readable media of claim 58, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

60. The non-transitory computer-readable media of claim 59, wherein the computer-executable instructions are further operable to cause the master server to receive an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

61. The non-transitory computer-readable media of claim 60, wherein the computer-executable instructions are further operable to cause the master server to receive a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

62. The non-transitory computer-readable media of claim 61, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

63. The non-transitory computer-readable media of claim 62, wherein the computer-executable instructions are further operable to cause the host of the second file system to decline a synchronization request generated by the host of the first file system.

64. The non-transitory computer-readable media of claim 63, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from a remote backup schedule.

65. The non-transitory computer-readable media of claim 55, wherein the computer-executable instructions are further operable to cause the one or more processors to generate a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and to store the rapid recovery file in the first file system.

66. The non-transitory computer-readable media of claim 65, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.

67. The non-transitory computer-readable media of claim 53, wherein the captured snapshot comprises an incremental relation mirroring data blocks that have been modified in the computer system from the time when a previously received image enabled backup data was generated to the time when the current image enabled backup data was generated, and wherein the computer-executable instructions are further operable to cause the one or more processors to associate the snapshot with the time when the current image enabled backup data was generated.

68. The non-transitory computer-readable media of claim 67, wherein each snapshot can serve as a restore point for the computer device corresponding to its associated time.

69. The non-transitory computer-readable media of claim 68, wherein the host of the first file system is different from a host of the second file system.

70. The non-transitory computer-readable media of claim 69, wherein the task processing assistance request further comprises at least one new cryptographic key generated by the host of the first file system and a plurality of metrics related to the operational state of the host of the first file system.

71. The non-transitory computer-readable media of claim 70, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node processing request to the host of the second file system comprising at least one new cryptographic key generated by the host of the first file system.

72. The non-transitory computer-readable media of claim 71, wherein the computer-executable instructions are further operable to cause the master server to receive an assisting node processing response comprising a plurality of metrics related to the operational state of the host of the second file system.

73. The non-transitory computer-readable media of claim 72, wherein the computer-executable instructions are further operable to cause the master server to receive a network reconfiguration request comprising a command to delete a vector describing a relation between the first file system and the second file system.

74. The non-transitory computer-readable media of claim 73, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node reconfiguration request to the host of the second file system comprising at least one command to disable at least one cryptographic key corresponding to the host of the first file system.

75. The non-transitory computer-readable media of claim 74, wherein the computer-executable instructions are further operable to cause the host of the second file system to decline a synchronization request generated by the host of the first file system.

76. The non-transitory computer-readable media of claim 75, wherein the computer-executable instructions are further operable to cause the master server to send an assisting node reconfiguration request to the host of the first file system comprising at least one command to delete a vector from the remote backup schedule.

77. The non-transitory computer-readable media of claim 67, wherein the computer-executable instructions are further operable to cause the one or more processors to generate a rapid recovery file in a ready-to-run disk image format corresponding to the image enabled backup data received by the host of the first file system in the current backup response and to store the rapid recovery file in the first file system.

78. The non-transitory computer-readable media of claim 77, wherein the first and second file system can be synchronized at a different frequency than the frequency on which the backup responses are received by the host of the first file system.