Transparent archiving

• US 20050149584A1
• Filed: 01/07/2004
• Published: 07/07/2005
• Est. Priority Date: 01/07/2004
• Status: Active Grant

• Current Assignee: Domo, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Wolfson, Charles Daniel, Bourbonnais, Serge
• Granted Patent: US 7,287,048 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/256 in federated or virtual databases
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

A method, apparatus, and article of manufacture implementing the method, transparently archives data. One or more data sources store data. A first data mover moves a first subset of data from the one or more data sources to a first federated archive in accordance with a first set of rules. In another aspect of the invention, a query is processed to retrieve a second subset of data, and a portion of the second subset of data is retrieved from the first federated archive. In yet another aspect of the invention, the processing of the query determines whether any data that would satisfy the query is stored in the first federated archive.

Claims

1. A method of transparently archiving data stored in one or more first data sources, comprising:

moving, by a first data mover, a first subset of data from the one or more first data sources to a first federated archive in accordance with a first set of rules.

2. The method of claim 1 further comprising:

moving, by the first data mover, a second subset of data from the first federated archive to a second federated archive in accordance with the first set of rules.

3. The method of claim 1 wherein the first federated archive is coupled to a second federated archive, further comprising:

moving, by a second data mover, at least a portion of the first subset of data to the second federated archive in accordance with a second set of rules.

4. The method of claim 1 further comprising:

processing a query to retrieve a second subset of data, wherein a first portion of the second subset of data is retrieved from the first federated archive.

5. The method of claim 4 wherein said processing comprises determining whether any subset of data that would satisfy the query is stored in the first federated archive.

6. The method of claim 4 further comprising:

creating a view over at least one of the one or more first data sources and the first federated archive;

wherein said processing comprises processing the query using the view to aggregate the second subset of data from the at least one of the one or more first data sources and the first federated archive.

7. The method of claim 4 wherein said processing comprises invoking a table function to retrieve the first portion of the second subset of data from the first federated archive.

8. The method of claim 1 further comprising:

receiving a query that specifies a time range from which to retrieve data; and

processing the query to retrieve data from the first federated archive based on the time range.

9. The method of claim 1 further comprising:

receiving a query to retrieve a second subset of data;

determining whether a portion of the second subset of data requested by the query is stored in the first federated archive; and

retrieving the portion of the second subset of data from the first federated archive.

10. The method of claim 9 further comprising:

updating meta-data indicating a location of the data, and

wherein said determining whether the portion of the second subset of data requested by the query is stored in the first federated archive is based on the meta-data.

11. The method of claim 1 further comprising:

updating at least one configuration parameter;

receiving a query; and

processing the query to retrieve a second subset of data from the first federated archive in accordance with the at least one configuration parameter.

12. The method of claim 1 wherein at least one rule from the set associates a predicate with a retention period on a table in the one or more first data sources and is combined with another rule to express a dependency between a different table in the one or more first data sources.

13. The method of claim 1 further comprising:

updating a value of a portion of the first subset of data in the first federated archive.

14. An apparatus for transparently archiving data, comprising:

a computer coupled to one or more first data sources and also to a first federated archive; and

one or more computer programs, to be executed by the computer, for

moving, by a first data mover, a first subset of data from the one or more first data sources to the first federated archive in accordance with a first set of rules.

15. The apparatus of claim 14 wherein the computer is coupled to a second federated archive, and said one or more computer programs are also for moving, by the first data mover, a second subset of data from the first federated archive to the second federated archive in accordance with the first set of rules.

16. The apparatus of claim 14 wherein the first federated archive is coupled to a second federated archive, and said one or more computer programs is also for:

moving, by a second data mover, at least a portion of the first subset of data to the second federated archive in accordance with a second set of rules.

17. The apparatus of claim 14 wherein said one or more computer programs is also for:

creating a view over at least one of the one or more first data sources and the first federated archive; and

processing a query to retrieve a second subset of data using the view, wherein a first portion of the second subset of data is retrieved from the first federated archive, and the view aggregates the second subset of data from the at least one of the one or more first data sources and the first federated archive.

18. The apparatus of claim 14 wherein the one or more computer programs are also for:

receiving a query to retrieve a second subset of data;

determining whether a portion of the second subset of data requested by the query is stored in the first federated archive; and

retrieving the portion of the second subset of data from the first federated archive.

19. The apparatus of claim 18 wherein the one or more computer program are also for:

updating meta-data indicating a location of the data, and

wherein said determining whether the portion of the second subset of data requested by the query is stored in the first federated archive is based on the meta-data.

20. An article of manufacture comprising a computer program usable medium embodying one or more instructions executable by a computer for performing a method of transparently archiving data stored in one or more first data sources, the method comprising:

moving, by a first data mover, a first subset of data from the one or more first data sources to a first federated archive in accordance with a first set of rules.

21. The article of manufacture of claim 20, said method further comprising:

moving, by the first data mover, a second subset of data from the first federated archive to a second federated archive in accordance with the first set of rules.

22. The article of manufacture of claim 20 wherein the first federated archive is coupled to a second federated archive, said method further comprising:

moving, by a second data mover, at least a portion of the first subset of data to the second federated archive in accordance with a second set of rules.

23. The article of manufacture of claim 20, said method further comprising:

processing a query to retrieve a second subset of data, wherein a first portion of the second subset of data is retrieved from the first federated archive.

24. The article of manufacture of claim 23 wherein said processing comprises determining whether any subset of data that would satisfy the query is stored in the first federated archive.

25. The article of manufacture of claim 23, said method further comprising:

creating a view over at least one of the one or more first data sources and the first federated archive,

wherein said processing comprises processing the query using the view to aggregate the second subset of data from the at least one of the one or more first data sources and the first federated archive.

26. The article of manufacture of claim 23 wherein said processing comprises invoking a table function to retrieve the first portion of the second subset of data from the first federated archive.

27. The article of manufacture of claim 20, said method further comprising:

receiving a query that specifies a time range from which to retrieve data; and

processing the query to retrieve data from the first federated archive based on the time range.

28. The article of manufacture of claim 20, said method further comprising:

receiving a query to retrieve a second subset of data;

determining whether a portion of the second subset of data requested by the query is stored in the first federated archive; and

retrieving the portion of the second subset of data from the first federated archvive.

29. The article of manufacture of claim 26, said method further comprising:

updating meta-data indicating a location of the data; and

wherein said determining whether the portion of the second subset of data requested by the query is stored in the first federated archive based on the meta-data.

30. The article of manufacture of claim 20, said method further comprising:

updating at least one configuration parameter;

receiving a query, and

processing the query to retrieve a second subset of data from the first federated archive in accordance with the at least one configuration parameter.

31. The article of manufacture of claim 20 wherein at least one rule from the first set of rules associates a predicate with a retention period on a table in the one or more first data sources and is combined with another rule to express a dependency between a different table in the one or more first data sources.

32. The article of manufacture of claim 20, said method further comprising:

updating a value of a portion of the first subset of data in the first federated archive.