Systems and/or methods for distributed data archiving

• US 20110238935A1
• Filed: 03/29/2010
• Published: 09/29/2011
• Est. Priority Date: 03/29/2010
• Status: Active Grant

• Current Assignee: Software AG
• Original Assignee: Software AG
• Inventors: Meehan, Michael C.
• Granted Patent: US 9,002,801 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Certain example embodiments of this invention relate to system and/or methods that pair a data extractor with a data accumulator, wherein these components may be located on any one or more computers in a network system. This distributed peer extract-accumulate approach is advantageous in that it reduces (and sometimes completely eliminates) the need for a “funnel” approach to data archiving, wherein all data is moved or backed up through a central computer or central computer system. In certain example embodiments, recall-accumulate, search, verify, and/or other archive-related activities may be performed in a similar peer-based and/or distributed manner. Certain example embodiments may in addition or in the alternative incorporate techniques for verifying the integrity of data in an archive system, and/or techniques for restoring/importing data from a non-consumable form.

Claims

1. An archival system, comprising:

a plurality of computers connected to a network;

at least one source data store and at least one target data store connected to the network; and

at least one archive service configured to coordinate a plurality of extract operations and a plurality of accumulate operations, each said extract operation being executed on one said computer in the plurality of computers to read data from one said source data store and each said accumulate operation being executed on one said computer in the plurality of computers to write data to one said target data store,

wherein each said extract operation is configured to run on any one predetermined computer in the plurality of computers and is paired with one said accumulate operation that is configured to run on any one predetermined computer in the plurality of computers.

2. The archival system of claim 1, wherein the at least one archive service is further configured to access rules (a) mapping the extract and accumulate operations to respective computers on which the operations are to be executed, and (b) storing pairings between peer extract and accumulate operations.

3. The archival system of claim 2, wherein the rules further identify at least one said source data store for each said extract operation and at least one said target data store for each said accumulate operation.

4. The archival system of claim 3, wherein the rules further include extract rules identifying data to be read from the at least one source data store and whether any further data is to be attached to the data to be read.

5. The archival system of claim 4, wherein the further data to be attached is a schema for the at least one source data store.

6. The archival system of claim 3, wherein the rules further include accumulate rules indicating how duplicate entries are to be handled and/or how long data is to be retained in each said target data store.

7. The archival system of claim 3, further comprising a configuration server in communication with the at least on archive service, the configuration server including the rules.

8. The archival system of claim 7, further comprising an administrative user interface in communication with configuration server and/or at least one archive service for setting the rules and/or monitoring progress of the operations.

9. The archival system of claim 3, wherein one said target data store is a vault that is accessible only in part by each said computer.

10. The archival system of claim 3, wherein the system is arranged such that a central hub or funnel is not used when data is written to the at least one target data store.

11. The archival system of claim 3, wherein the at least one archive service is further configured to coordinate at least one recall operation to retrieve data from the at least one target data store for access on one said computer.

12. The archival system of claim 11, wherein the at least one recall operation is further configured to place the retrieved data from the at least one target data store into at least one said source data store.

13. The archival system of claim 3, wherein the at least one archive service is further configured to coordinate at least one validation operation to verify integrity of data in the at least one target data store.

14. The archival system of claim 13, wherein the at least one validation operation is configured to run continuously for a predetermined amount of time.

15. The archival system of claim 13, wherein the at least one validation operation is configured to run periodically such that the at least one validation operation begins execution at predetermined times or time intervals.

16. The archival system of claim 13, wherein the at least one validation operation is configured to determine whether data exists, matches a checksum, and/or is recallable, based on predetermined rules accessible by the at least one validation operation.

17. The archival system of claim 16, wherein the at least one validation operation is further configured to raise an alarm if the at least one validation operation encounters a fault during execution.

18. The archival system of claim 3, wherein the at least one archive service is further configured to coordinate at least one importing operation to incorporate data from an otherwise non-consumable backup location into the at least one source data store and/or the at least one target data store.

19. The archival system of claim 18, wherein the at least one importing operation is configured to determine (a) what data exists in the data from the otherwise non-consumable backup location was backed up, and (b) how the data from the otherwise non-consumable backup location was backed up.

20. The archival system of claim 19, wherein the at least one importing operation implements rules for reacquiring the data from the otherwise non-consumable backup location, the rules for reacquiring the data being one or more user programmed rules, one or more predefined algorithms, and/or automatically generated by the at least one importing operation.

21. A computer-implemented method for tangibly storing data in an archival system, the method comprising:

providing a plurality of computers connected to a network;

providing at least one source data store and at least one target data store connected to the network;

providing at least one archive service to one or more of said computers, the at least one archive service being configured to interface with a plurality of extract operations and a plurality of accumulate operations, each said extract operation being configured to run on any one predetermined computer in the plurality of computers; and

pairing each said extract operation with one said accumulate operation, each said accumulate operation being configured to run on any one predetermined computer in the plurality of computers;

wherein each said extract operation is executable on one said computer in the plurality of computers to read data from one said source data store and each said accumulate operation is executable on one said computer in the plurality of computers to write data to one said target data store.

22. The method of claim 21, further comprising:

mapping the extract and accumulate operations to respective computers on which the operations are to be executed; and

storing pairings between peer extract and accumulate operations.

23. The method of claim 22, further comprising identifying at least one said source data store for each said extract operation and at least one said target data store for each said accumulate operation.

24. The method of claim 23, further comprising defining extract rules identifying (a) data to be read from the at least one source data store and (b) whether any further data is to be attached to the data to be read.

25. The method of claim 24, wherein the further data to be attached is a schema for the at least one source data store.

26. The method of claim 23, further comprising defining accumulate rules indicating (a) how duplicate entries are to be handled and/or (b) how long data is to be retained in each said target data store.

27. The method of claim 23, further comprising providing a configuration server in communication with the at least on archive service, the configuration server having access to rules describing the mapping, the pairing, the at least one said source data store identified for each said extract operation, and the at least one said target data store identified for each said accumulate operation.

28. The method of claim 27, further comprising providing an administrative user interface in communication with configuration server and/or at least one archive service for setting the rules and/or monitoring progress of the operations.

29. The method of claim 23, wherein one said target data store is a vault that is accessible only in part by each said computer.

30. The method of claim 23, wherein the operations are performed without resort to a central hub or funnel.

31. The method of claim 23, wherein the at least one archive service is further configured to coordinate at least one recall operation to retrieve data from the at least one target data store for access on one said computer.

32. The method of claim 31, wherein the at least one recall operation is further configured to place the retrieved data from the at least one target data store into at least one said source data store.

33. The method of claim 23, wherein the at least one archive service is configurable to coordinate at least one validation operation to verify integrity of data in the at least one target data store.

34. The method of claim 33, wherein the at least one validation operation is configurable to run continuously for a predetermined amount of time.

35. The method of claim 33, wherein the at least one validation operation is configurable to run periodically such that the at least one validation operation begins execution at predetermined times or time intervals.

36. The method of claim 33, wherein the at least one validation operation is configurable to determine whether data exists, matches a checksum, and/or is recallable, based on predetermined rules accessible by the at least one validation operation.

37. The method of claim 36, wherein the at least one validation operation is further configurable to raise an alarm if the at least one validation operation encounters a fault during execution.

38. The method of claim 23, wherein the at least one archive service is further configured to coordinate at least one importing operation to incorporate data from an otherwise non-consumable backup location into the at least one source data store and/or the at least one target data store.

39. The method of claim 38, wherein the at least one importing operation is configurable to determine (a) what data exists in the data from the otherwise non-consumable backup location was backed up, and (b) how the data from the otherwise non-consumable backup location was backed up.

40. The method of claim 39, wherein the at least one importing operation implements rules for reacquiring the data from the otherwise non-consumable backup location, the rules for reacquiring the data being one or more user programmed rules, one or more predefined algorithms, and/or automatically generated by the at least one importing operation.

41. A computer readable storage medium tangibly storing instructions that, when executed, cause a computer including at least one processor to perform the method of claim 21.