System and method for performing auxillary storage operations

• US 7,490,207 B2
• Filed: 11/08/2005
• Issued: 02/10/2009
• Est. Priority Date: 11/08/2004
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Vijayan Retnamma, Manoj Kumar, Prahlad, Anand, Amarendran, Arun Prasad, Lu, Jun
• Primary Examiner(s): Sough; Hyung
• Assistant Examiner(s): Doan; Duc T
• Application Number:
  US11/269,119
• Publication Number:
  US 20060230244A1
• Time in Patent Office: 1,190 Days
• Field of Search: None
• US Class Current: 711/165
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

Systems and methods for protecting data in a tiered storage system are provided. The storage system comprises a management server, a media management component connected to the management server, a plurality of storage media connected to the media management component, and a data source connected to the media management component. Source data is copied from a source to a buffer to produce intermediate data. The intermediate data is copied to both a first and second medium to produce a primary and auxiliary copy, respectively. An auxiliary copy may be made from another auxiliary copy. An auxiliary copy may also be made from a primary copy right before the primary copy is pruned.

Claims

1. A method for storing data in a tiered storage system, the method comprising:

accessing a storage policy associated with a tiered data storage system, wherein the storage policy defines migration of data within the tiered data storage system, the storage policy defining;

copying source data from a data source to a primary copy stored in a first storage media destination in the tiered data storage system, wherein the first storage media destination is a first type of storage comprising one or more hard drive storage devices; and

simultaneously streaming at least a portion of the source data from the data source to an auxiliary copy stored in a second storage media destination in the tiered data storage system, wherein the second storage media destination is a second type of storage comprising one or more removable storage devices wherein the second type of storage is different then the first type of storage;

when the storage policy indicates that a storage operation is to be performed within the tiered data storage system, copying the source data according to the storage policy from the data source to a first buffer location to produce intermediate data, wherein the copying of the source data to produce the intermediate data comprises;

dividing the source data into a plurality of chunks;

encapsulating by a media management component, each of the chunks with a header portion that is based on the storage policy, wherein the header portion comprises metadata that indicates at least two distinct storage media destinations to which each respective chunk is to be copied, the two distinct storage media destinations comprising the first storage media destination and the second storage media destination, and wherein the header portion further indicates the first and second types of storage associated with the first and second storage media destinations, and wherein a media management component reads the header portion to determine the types of storage and performs read or write operations using the formatting required for the types of storage, and

transferring the plurality of chunks from the data source to the first buffer location;

copying the intermediate data, according to the header portions, to the first storage media destination to produce a primary copy of the source data on the first type of storage;

removing the header portion with the two distinct storage media destinations from each chunk in the intermediate data when copying the chunks to the first storage media destination; and

while the primary copy is still being produced, simultaneously streaming one or more of the chunks of the intermediate data, according to the header portions, to the second storage media destination to produce an auxiliary copy of the source data on the second type of storage, the second type of storage being a different type than the first type of storage; and

removing the header portion with the two distinct storage media destinations from each chunk in the intermediate data when copying the chunks to the second storage media destination.

2. The method of claim 1, comprising monitoring the copying of the intermediate data to the first and second storage media destinations.

3. The method of claim 2, comprising:

determining that an interruption occurred in the production of the primary copy or auxiliary copy;

completing the production of the uninterrupted copy; and

thereafter, when the interruption is resolved, completing the interrupted production of the primary or auxiliary copy.

4. The method of claim 1, wherein copying the source data further comprises using a data pipe.

5. The method of claim 1, wherein copying the intermediate data to the first and second storage media destinations comprises formatting the intermediate data in a first format based on the first type storage associated with the first storage media destination and formatting the intermediate data in a second format based on the second type of storage associated with the second storage media destination.

6. The method of claim 1, wherein the second type of storage comprises a tape.

7. The method of claim 1, wherein the storage policy comprises a user-defined policy.

8. The method of claim 1, wherein said copying the source data further comprises performing compression on the source data.

9. The method of claim 1, wherein said copying the source data further comprises performing encryption on the source data.

10. The method of claim 1, further comprising:

determining when the primary copy is about to be deleted; and

performing a copy of the primary data to a third storage media destination.

11. The method of claim 1, further comprising producing multiple auxiliary copies of the source data.

12. The method of claim 11, wherein producing multiple auxiliary copies comprises:

creating a single instance copy of each chunk; and

storing the header portion and a fingerprint of the data of each of the plurality of chunks on multiple storage media.

13. The method of claim 1, wherein the header portion further indicates a time to produce the primary copy or the auxiliary copy.

14. The method of claim 1, wherein the header portion is based at least in part on information contained in the storage policy.

15. A method which, when executed on a computer, stores data in a tiered storage system, the method comprising:

accessing a storage policy associated with at tiered data storage system, wherein the storage policy defines copying source data from a data source to at least two distinct storage media destinations in the tiered data storage system;

copying the source data according to said storage policy from the data source to a first buffer location to produce intermediate data, wherein said copying comprises;

dividing the source data into a plurality of portions, and

assigning by a media management component each of the plurality of portions a tag header, the tag header comprising the two distinct storage media destinations associated with the storage policy, the two distinct storage media destinations comprising a first storage media destination and a second storage media destination and wherein a media management component reads the tag header to determine types of storage associated with the first and second media destinations and to perform read or write operations using the formatting required for the types of storage, and copying the intermediate data, according to said tag headers, to the first storage media destination to produce a primary copy of the source data; and

while the primary copy is being produced, simultaneously streaming one or more portions of the intermediate data, according to said tag headers, to the second storage media destination to produce a secondary copy of the source data.

16. The method of claim 15, comprising monitoring the copying of the source data from the data source to the first location.

17. The method of claim 15, comprising monitoring the copying of the intermediate data to the first storage media destination.

18. The method of claim 17, comprising monitoring the copying of the intermediate data to the second storage media destination.

19. The method of claim 18, comprising:

determining that an interruption occurred in the production of the primary copy or secondary copy;

completing the production of the uninterrupted copy; and

thereafter, when the interruption is resolved, completing the interrupted production of the primary or secondary copy.

20. The method of claim 15, wherein copying the source data from the data source to the first location comprises using a data pipe.

21. The method of claim 15, wherein copying the intermediate data to the first and second storage media destinations comprises formatting the intermediate data in a first format based on a media type of the first storage media destination and formatting the intermediate data in a second format based on the a media type of the second storage media destination.

22. A method for performing multiple storage operations on data in a tiered storage system, the method comprising:

receiving storage policy information from a storage manager module, the storage policy information associated with a tiered data storage system, wherein the storage policy defines copying source data from a data source to distinct first and second storage media destinations;

copying the source data according to said storage policy information from the data source to a first buffer location to produce intermediate data, wherein said copying comprises,

obtaining the source data from the data store using at least one media agent module,

dividing the source data into a plurality of chunks,

associating by a media management component each of the plurality of chunks with a header portion, wherein each header portion indicates at least the following information;

(1) at least the first and second storage media destinations to which the corresponding one of the plurality of chunks is to be copied,

(2) first and second distinct types of storage associated with the first and second storage media destinations to which the corresponding one of the plurality of chunks is to be copied, and

(3) one or more times at which the corresponding one of the plurality of chunks is to be copied, and

reading the header portions with a media management component to determine first and second types of storage associated with the first and second storage media destinations and to perform read or write operations using the formatting required for the first and second types of storage;

copying the intermediate data, according to the information stored in said header portions, to the first storage media destination to produce a first copy of the source data on the first type of storage; and

while the first copy is being produced, simultaneously streaming copies of one or more chunks of the intermediate data, according to the information stored in said header portions, to the second storage media destination to produce a second copy of the source data on the second type of storage.

23. The method of claim 22, wherein said copying the intermediate data to at least one of the first storage media destination and the second storage media destination is performed with the media management component.

24. The method of claim 23, additionally comprising maintaining an index with the media management component indicating the location of the first copy on the first storage media destination.

25. The method of claim 24, additionally comprising maintaining a copy of the index on the first storage media destination.