Systems and methods for creating copies of data, such as archive copies

• US 8,909,881 B2
• Filed: 03/05/2013
• Issued: 12/09/2014
• Est. Priority Date: 11/28/2006
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Bunte, Alan, Prahlad, Anand, Brockway, Brian
• Primary Examiner(s): Alsip, Michael
• Assistant Examiner(s): BIRKHIMER, CHRISTOPHER D
• Application Number:
  US13/785,224
• Publication Number:
  US 20130179405A1
• Time in Patent Office: 644 Days
• Field of Search: None
• US Class Current: 711/161
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/325 Hash tables
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database

Abstract

A system and method of creating archive copies of data sets is described. In some examples, the system creates an archive copy from an original data set. In some examples, the system creates an archive copy when creating a recovery copy for a data set. In some examples, the system creates a copy without redundant data, and then encrypts the data set.

Claims

1. At least one data storage system that creates an archive copy of data originating from a file system of one or more client computers, comprising:

a hierarchical storage system having—

a storage manager server computer, and

at least one data store subsystem,

wherein the storage manager server computer includes at least one storage policy for directing storage operations,

wherein the one or more client computers are coupled for communication among the storage manager server computer and the data store subsystem,

wherein the data store subsystem is geographically separated from the one or more client computers, and

wherein the hierarchical storage system comprises;

an archive copy creation subsystem, wherein the archive copy creation subsystem creates an archive copy of at least a subset of data from the file system based on the storage policy of the storage manager server computer,

wherein the archive copy encapsulates a header file having cryptographically and substantially unique identifiers of each of multiple data files within the subset of data,

wherein the archive copy creation subsystem inserts the header file into a beginning of the archive copy prior to transfer of the archive copy to the data store subsystem,

wherein the archive copy creation subsystem includes;

a data selection component, wherein the data selection component selects the subset of data to be copied based at least in part on the storage policy,

 

wherein the selected subset of data is to be stored for a longer period of time than other data;

an indexing component, wherein the indexing component creates an index of content within the selected subset of data; and

a data adjustment component, communicatively coupled to the data selection component,

 

wherein the data adjustment component—

 

performs data compression of the selected subset of data;

 

performs data deduplication for the selected subset of data, or

 

performs both data compression and data deduplication for the selected subset of data.

2. The data storage system of claim 1, wherein the data originating from the file system corresponds to a primary copy data, wherein the selected subset of data is a subset of a secondary copy of the data, and wherein the storage manager server computer operates on the secondary copy.

3. The data storage system of claim 1, wherein the data selection component includes a user interface component capable of receiving a request from a user of the data storage system that identifies content to be searched in the selected subset of data, and wherein the archive copy creation subsystem searches the index of content in response to the received user request.

4. The data storage system of claim 1,

wherein the data selection component includes a user interface component capable of receiving a request from a user of the data storage system that identifies a subset of data to be selected,

wherein the subset of data is stored to magnetic tape after being processed by the data indexing component and the data adjustment component, and

wherein the header file having cryptographically identifiers of each of multiple data files is a single header file at the beginning of the magnetic tape that is read first when the magnetic tape is mounted and that can identify each of the multiple data files stored on the magnetic tape.

5. The data storage system of claim 1, wherein the archive copy creation subsystem further comprises:

an encryption component, wherein the encryption component encrypts the selected subset of data, and

wherein the data adjustment component, the data indexing component, and the encryption component operate as a data process flow for archiving the selected subset of data to magnetic tape storage.

6. The data storage system of claim 1,

wherein the indexing component creates an index of the cryptographically and substantially unique identifiers of data files within the subset of data, and

wherein the data adjustment component identifies at least some redundancies within the subset of data using the index of cryptographically and substantially unique identifiers and creates a single-instance copy of the subset of data without the identified redundancies.

18. The at least one data storage system of claim 1,

wherein the header file is stored at the beginning of sequential storage media with the archive copy, and

wherein the header file includes the cryptographically and substantially unique identifiers for all the multiple data files within the subset of data.

19. The at least one data storage system of claim 1, wherein the data store subsystem includes a magnetic tape, an optical disk, or a universal serial bus (USB) device.

20. The at least one data storage system of claim 3, wherein the request is an electronic discovery (e-discovery) request.

7. A method of archiving an original set of data created by a file system, the method comprising:

at a first time, identifying one or more redundant data objects within the original set of data and creating a copy of the original set of data that does not include the identified one or more redundant data objects,

wherein the identifying is based on cryptographically and substantially unique identifiers of data objects in the original set of data,

wherein the cryptographically and substantially unique identifiers are based on content of the data objects in the original set, and,

wherein identifying includes searching an index of the cryptographically and substantially unique identifiers;

appending at least one header file to the beginning of the copy of the original set of data to create an archive copy, before a second time that is after the first time, wherein the at least one header file includes the cryptographically and substantially unique identifiers of the data objects of the copy of the original set of data;

at the second time after the first time, encrypting or compressing the copy of the original set of data; and

at a third time after the first time and the second time, transferring the archive copy to a secondary storage device,

wherein the archive copy is stored in the secondary storage device in a geographical located separated from the file system,

wherein the archive copy is to be stored for a longer period of time than the original set of data,

wherein the archive copy is in a format that differs from a format of the original set of data, and

wherein data within the archive copy cannot be used by applications that created the original set of data without first decrypting, decompressing or converting the data within the archive copy.

8. The method of claim 7, wherein identifying one or more redundant data objects includes:

computing a hash value for each data object to be archived; and

comparing each of the hash values of the data objects.

9. The method of claim 7 further comprising:

at the first time, computing a hash value for each data object to be archived and storing the computed hash values in a table to be checked for determining data redundancies.

10. The method of claim 7, wherein the original set of data is a production copy of a set of data that is for primary use by users of the set of data.

11. The method of claim 7, wherein at the second time, both encrypting and compressing the copy of the original set of data.

12. The method of claim 7,

wherein the data storage medium is magnetic tape, and

wherein magnetic tape stores the archive copy along with the at least one header file, wherein the at least one header file is an initial index of the data stored in the archive copy.

13. The method of claim 7, further comprising:

at a fourth time, later than the third time, receiving a request from a user that identifies content to be searched in the archive copy, and

at a fifth time, later than the fourth time, searching a content index of data stored in the archive copy, and wherein the archive copy is stored on magnetic tape.

14. At least one non-transitory computer-readable medium storing instructions that, when executed by at least one data processing device, causes the creation of a copy of a production volume of data for archiving the production volume, comprising:

receiving or accessing two or more secondary copies of the production volume of the data,

wherein the two or more secondary copies include multiple instances of one or more data objects within the volume of data;

creating or accessing an index of cryptographically and substantially unique identifiers of each data object within the two or more secondary copies of the production volume;

identifying multiple instances of the one or more data objects using the index of cryptographically and substantially unique identifiers;

storing the data from the two or more secondary copies into an archive copy,

wherein the stored data includes only one instance for at least one data object having multiple instances,

wherein the stored data includes at least part of the index of cryptographically and substantially unique identifiers in a header file that is at a beginning of the archive copy;

encrypting or compressing the data stored within the archive copy,

wherein the archive copy is for storing data for a longer period of time than the two or more secondary copies; and

after encrypting or compressing, transferring the archive copy to a secondary storage device.

15. The at least one non-transitory computer-readable medium of claim 14,

wherein storing the data from the two or more secondary copies into the archive copy includes storing the one instance of the at least one data object in a first location within the archive copy and storing information related to other instances of the at least one data object in a second location within the archive copy,

wherein the archive copy is stored on magnetic tape, and

wherein the header file at the beginning of the magnetic tape is read first when the magnetic tape is mounted.

16. The at least one non-transitory computer-readable medium of claim 14, further comprising:

receiving a request from a user that identifies content to be searched in the archive copy,

searching a content index of data stored in the archive copy, and wherein the archive copy is stored on magnetic tape,

wherein the header file at the beginning of the magnetic tape is read first when the magnetic tape is mounted.

17. The at least one non-transitory computer-readable medium of claim 14, wherein encrypting or compressing the data includes both encrypting and compressing the data stored within the archive copy.