Systems and methods for database archiving

• US 9,275,086 B2
• Filed: 03/06/2013
• Issued: 03/01/2016
• Est. Priority Date: 07/20/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Kumarasamy, Paramasivam, Vallabhaneni, Brahmaiah
• Primary Examiner(s): Fan, Shiow-Jy
• Application Number:
  US13/786,928
• Publication Number:
  US 20140025641A1
• Time in Patent Office: 1,091 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A data storage system according to certain aspects can archive database data associated with different database applications. The data storage system according to certain aspects may provide database archiving modules that include logic incorporating and/or based on the native schema and/or native commands specific to particular database applications. The database archiving modules according to certain aspects may determine the relationship between tables associated with corresponding database applications and archive selected database data based on the native schema and native commands.

Claims

1. A method of archiving data generated by one or more database applications in a networked data storage system, the method comprising:

using one or more computing devices comprising computer hardware;

receiving instructions to archive a logical subset of data in a stored database, the data organized in a plurality of tables, generated by a first database application residing on a first client computing device of a plurality of client computing devices, and stored in a first storage device associated with the first client computing device, the logical subset of data stored in one or more first tables of the plurality of tables, a first data agent associated with the database application installed on the first client computing device;

determining by the first data agent a relationship between the first tables and one or more other tables of the plurality of tables, based at least in part on a native schema of the first database application;

identifying by the first data agent one or more data items referenced by the logical subset, based on the determined relationship between the first tables and the other tables, the one or more data items stored in one or more second tables of the other tables, the second tables referenced by the first tables;

extracting by the first data agent the logical subset and the identified data items in the first storage device;

selecting a first secondary storage controller computer from a plurality of secondary storage controller computers, the plurality of secondary storage controller computers configured to perform storage operations for moving data between a primary storage subsystem and a secondary storage subsystem;

copying by the first secondary storage controller computer the extracted logical subset and identified data items to one or more secondary storage devices to create a secondary copy of the extracted logical subset and identified data items, said copying comprising copying additional extracted database data accumulated on the first storage device prior to said receiving the instructions, along with the extracted logical subset and identified data items, such that the additional extracted database data and the extracted logical subset and identified data items are copied to the one or more secondary storage devices using a fewer number of storage operations than individually copying the additional extracted database data and the extracted logical subset and identified data items; and

following said copying, pruning by the first data agent the logical subset and the identified data items from the first storage device, the logical subset being pruned prior to the identified data items,

wherein rules for said extracting, said copying, and said pruning are stored in a storage policy for moving data between the primary storage subsystem and the secondary storage subsystem,

wherein the logical subset includes at least a first data item and the identified data items include at least one data item referenced by the first data item.

2. The method of claim 1, wherein said extracting comprises using native commands of the first database application.

3. The method of claim 1, wherein said copying comprises using native commands of the first database application.

4. The method of claim 1, wherein said pruning comprises using native commands of the first database application.

5. The method of claim 1, wherein said identifying further comprises:

identifying a record in the first tables; and

identifying a record in the second tables; and

wherein said pruning further comprises;

deleting the record in the second tables before deleting the record in the first tables.

6. The method of claim 1, wherein said identifying comprises traversing the tables in the database using native commands of the first database application to identify the one or more data items.

7. The method of claim 1, wherein said copying comprises writing a plurality of the extracted logical subset and identified data items to a temporary location and copying all or a subset of the plurality of the extracted logical subset and identified data items from the temporary location to the one or more secondary storage devices.

8. The method of claim 1, wherein the database application comprises a relational database.

9. The method of claim 1, wherein the plurality of client computing devices comprises a second client computing device that is associated with a second storage device and has a second database application and a second data agent residing thereon, the second client computing device configured to:

receive second instructions to archive a second logical subset of data in a second stored database, the data organized in a second plurality of tables, generated by the second database application, and stored in the second storage device, the second logical subset of data stored in one or more third tables of the second plurality of tables;

determine by the second data agent a relationship between the third tables and one or more second other tables of the second plurality of tables, based at least in part on a native schema of the second database application;

identify by the second data agent one or more second data items referenced by the second logical subset, based on the determined relationship between the third tables and the second other tables, the one or more second data items stored in one or more fourth tables of the second other tables, the fourth tables referenced by the third tables;

extract by the second data agent the second logical subset and the identified second data items from the second stored database;

select a second secondary storage controller computer from the plurality of secondary storage controller computers;

copy by the second secondary storage controller computer the extracted second logical subset and identified second data items to the one or more secondary storage devices to create a secondary copy of the extracted second logical subset and identified second data items; and

following the copying of the extracted second logical subset and identified second data items, prune by the second data agent the second logical subset and the identified second data items from the second storage device, the second logical subset being pruned prior to the identified second data items.

10. The method of claim 9, wherein the first database application and the second database application are different and have different native schemas.

11. A data storage system configured to archive data generated by one or more database applications, comprising:

a plurality of client computing devices including a first client computing device that is associated with a first storage device and has a first database application and a first data agent residing thereon; and

a plurality of secondary storage controller computers comprising computer hardware configured to perform storage operations for moving data between a primary storage subsystem and a secondary storage subsystem;

the first client computing device configured to;

receive instructions to archive a logical subset of data in a stored database, the data organized in a plurality of tables, generated by the first database application, and stored in the first storage device, the logical subset of data stored in one or more first tables of the plurality of tables;

determine by the first data agent a relationship between the first tables and one or more other tables of the plurality of tables, based at least in part on a native schema of the first database application;

identify by the first data agent one or more data items referenced by the logical subset, based on the determined relationship between the first tables and the other tables, the one or more data items stored in one or more second tables of the other tables, the second tables referenced by the first tables;

extract by the first data agent the logical subset and the identified data items in the first storage device;

select a first secondary storage controller computer from the plurality of secondary storage controller computers;

copy by the first secondary storage controller computer the extracted logical subset and identified data items to one or more secondary storage devices to create a secondary copy of the extracted logical subset and identified data items, the copying of the extracted logical subset and identified data items comprising copying additional extracted database data accumulated on the first storage device prior to the receiving of the instructions, along with the extracted logical subset and identified data items, such that the additional extracted database data and the extracted logical subset and identified data items are copied to the one or more secondary storage devices using a fewer number of storage operations than individually copying the additional extracted database data and the extracted logical subset and identified data items; and

following the copying of the extracted logical subset and identified data items, prune by the first data agent the logical subset and the identified data items from the first storage device, the logical subset being pruned prior to the identified data items,

wherein rules for the extracting of the logical subset and the identified data items, the copying of the extracted logical subset and identified data items, and the pruning of the logical subset and the identified data items are stored in a storage policy for moving data between the primary storage subsystem and the secondary storage subsystem,

wherein the logical subset includes at least a first data item and the identified data items include at least one data item referenced by the first data item.

12. The data storage system of claim 11, wherein the extraction of the logical subset and the identified data items comprises using native commands of the first database application.

13. The data storage system of claim 11, wherein the copying of the extracted logical subset and identified data items comprises using native commands of the first database application.

14. The data storage system of claim 11, wherein the pruning of the logical subset and the identified data items comprises using native commands of the first database application.

15. The data storage system of claim 11, wherein the identification of the one or more data items further comprises:

identifying a record in the first tables; and

identifying a record in the second tables; and

wherein the pruning of the logical subset and the identified data items further comprises;

deleting the record in the second tables before deleting the record in the first tables.

16. The data storage system of claim 11, wherein the identification of the one or more data items comprises traversing the tables in the database using native commands of the first database application to identify the one or more data items.

17. The data storage system of claim 11, wherein said copying comprises writing a plurality of the extracted logical subset and identified data items to a temporary location and copying all or a subset of the plurality of the extracted logical subset and identified data items from the temporary location to the one or more secondary storage devices.

18. The data storage system of claim 11, wherein the database application comprises a relational database.

19. A non-transitory computer readable medium comprising instructions for archiving data generated by one or more database applications in a networked data storage system, that cause a processor to:

receive instructions to archive a logical subset of data in a stored database, the data organized in a plurality of tables, generated by a first database application residing on a first client computing device of a plurality of client computing devices, and stored in a first storage device associated with the first client computing device, the logical subset of data stored in one or more first tables of the plurality of tables, a first data agent associated with the database application installed on the first client computing device;

determining by the first data agent a relationship between the first tables and one or more other tables of the plurality of tables, based at least in part on a native schema of the first database application;

identify by the first data agent one or more data items referenced by the logical subset, based on the determined relationship between the first tables and the other tables, the one or more data items stored in one or more second tables of the other tables, the second tables referenced by the first tables;

extract by the first data agent the logical subset and the identified data items in the first storage device;

select a first secondary storage controller computer from a plurality of secondary storage controller computers, the plurality of secondary storage controller computers configured to perform storage operations for moving data between a primary storage subsystem and a secondary storage subsystem;

copy by the first secondary storage controller computer the extracted logical subset and identified data items to one or more secondary storage devices to create a secondary copy of the extracted logical subset and identified data items, the copying of the extracted logical subset and identified data items comprising copying additional extracted database data accumulated on the first storage device prior to receiving of the instructions, along with the extracted logical subset and identified data items, such that the additional extracted database data and the extracted logical subset and identified data items are copied to the one or more secondary storage devices using a fewer number of storage operations than individually copying the additional extracted database data and the extracted logical subset and identified data items; and

following the copying of the extracted logical subset and identified data items, prune by the first data agent the logical subset and the identified data items from the first storage device, the logical subset being pruned prior to the identified data items,

wherein rules for the extracting of the logical subset and the identified data items, the copying of the extracted logical subset and identified data items, and the pruning of the logical subset and the identified data items stored in a storage policy for moving data between the primary storage subsystem and the secondary storage subsystem,

wherein the logical subset includes at least a first data item and the identified data items include at least one data item referenced by the first data item.