Storing a variable number of instances of data objects

• US 8,401,996 B2
• Filed: 12/30/2009
• Issued: 03/19/2013
• Est. Priority Date: 03/30/2009
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Muller, Marcus S., Attarde, Deepak R., Vijayan, Manoj Kumar
• Primary Examiner(s): Richardson, James E
• Application Number:
  US12/649,454
• Publication Number:
  US 20100250549A1
• Time in Patent Office: 1,175 Days
• Field of Search: 707/644, 707/645, 707/647, 707651-653, 707/655, 707/659, 707/660
• US Class Current: 707/651
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Described are systems and methods for storing a variable number of instances of data objects (e.g., 1, 2, 3, or up to N−1 instances, where N is the number of instances of the data object included in primary data) in secondary storage across a data storage network. In some examples, a system for storing a variable number of instances of data objects includes, one or more computing devices storing a set of data objects and multiple storage devices distinct from the one or more computing devices. Each of the multiple storage devices is configured to store at least a single instance of a data object. The system also includes a database configured to store information associated with the data objects. This information includes substantially unique identifiers for the data objects and, for each of the data objects, a number of instances of the data object stored on the multiple storage devices.

Claims

1. A method in a data storage system of storing a variable number of instances of data objects, wherein the method is performed by a computing system having a processor and memory, the method comprising:

accessing a set of data objects stored in a first set of one or more data stores,

wherein the set of data objects includes multiple data objects that are substantially similar, and

wherein the multiple data objects that are substantially similar includes a number N data objects;

analyzing a storage policy;

determining, by the computing system, a number X of instances of a first data object of the multiple data objects that are substantially similar that should be stored in a second set of one or more data stores,

wherein determining the number X of instances of the first data object is based on the storage policy;

wherein X is greater than one and less than N and N is greater than two, and

wherein the first and second sets of data stores are distinct from each other;

adjusting the number X of instances of the first data object,

wherein adjusting the number X includes;

determining a characteristic or priority of the first data object, wherein the characteristic or priority is based on at least two of;

an owner of the first data object;

a content of the first data object;

an association of the first data object with an organization unit;

a security descriptor of the first data object; and

the number N of data objects included in multiple data objects; and

based at least in part on the determined characteristic or priority of the first data object, increasing or decreasing the number X of instances of the first data object of the multiple data objects that should be stored in the second set of one or more data stores;

determining a number Y of instances of the first data object that are stored in the second set of one or more data stores,

wherein Y is greater than or equal to zero;

determining if the adjusted number X is greater than Y; and

when the adjusted number X is greater than Y, storing a number Z of instances of the first data object in the second set of one or more data stores,

wherein Z is equal to the adjusted number X minus Y.

2. The method of claim 1 wherein determining the number Y includes:

generating a substantially unique identifier for the first data object;

analyzing a data structure; and

based at least in part on the analyzing of the data structure, determining the number Y of instances of the first data object that are stored in the second set of one or more data stores.

3. The method of claim 1, further comprising:

receiving an indication to perform a storage operation on the set of data objects; and

performing the storage operation on the set of data objects, wherein performing the storage operation includes copying Z instances of the first data object to the second set of one or more data stores.

4. The method of claim 1, further comprising determining the second set of one or more data stores based at least in part upon one or more characteristics of at least one data store in the second set, wherein the one or more characteristics include at least one of:

a storage capacity of the at least one data store;

a power consumption of the at least one data store;

a time to store data on the at least one data store; and

a time to access data stored on the at least one data store.

5. The method of claim 1, further comprising:

monitoring the multiple data objects;

determining when the number N of data objects included in the multiple data objects exceeds a predetermined threshold; and

when the number N of data objects exceeds the predetermined threshold storing at least one instance of the first data object in the second set of data stores.

6. A system for storing multiple instances of a data object in a data storage network, wherein the data storage network includes multiple data storage devices coupled via a computer network, and wherein the computer network also couples to one or more computing devices storing a first set of N number of substantially similar data objects, the system comprising:

multiple data storage devices storing a second set of data objects,

wherein at least one of the multiple data storage devices is embodied in hardware, and

wherein the multiple data storage devices are distinct from, and coupled via the computer network to, the one or more computing devices;

a variable instance database configured to store information associated with the second set of data objects,

wherein the information includes, for each of the data objects of the second set;

a substantially unique identifier for the data object, and

a number of instances of the data object in the second set of data objects,

wherein at least one of the data objects of the second set has multiple instances in the second set of data objects;

a secondary storage computing device configured to;

receive an indication to perform a storage operation on the first set of N number of substantially similar data objects,

wherein N is a number greater than 3,

determine a substantially unique identifier for each of the N number of substantially similar data objects of the first set;

search for the substantially unique identifier in the variable instance database;

determine a number of instances of the N number of substantially similar data objects to store in the second set of data objects based on the received indication to perform the storage operation,

wherein the determined number is more than one and less than all of the N number substantially similar data objects;

adjust the number of instances of the N number of substantially similar data objects based on a characteristic or priority of the N number of substantially similar data objects,

wherein the characteristic or priority includes at least two of;

an owner of the N number of substantially similar data objects;

a content of the N number of substantially similar data objects;

an association of the N number of substantially similar data objects with an organization unit;

a security descriptor of the N number of substantially similar data objects; and

store the adjusted number of instances of the N number of substantially similar data objects in the second set of data objects.

7. The system of claim 6 wherein the one or more computing devices storing a first set of N number of substantially similar data objects are at a first hierarchical tier of the data storage network and the multiple data storage devices are at a second hierarchical tier of the data storage network.

8. The system of claim 6, further comprising a storage manager component configured to:

receive an indication to perform the storage operation on the N number of substantially similar data objects of the first set; and

send the indication to the secondary storage computing device.

9. The system of claim 6, further comprising:

a first storage manager component configured to;

receive an indication to perform the storage operation on the N number of data objects of the first set; and

send the indication to the secondary storage computing device; and

a second storage manager component configured to;

receive the indication to perform the storage operation on the N number of data objects of the first set; and

send the indication to the first storage manager component,

wherein the first storage manager component is hierarchically subordinate to the second storage manager component in the data storage network.

10. The system of claim 6, further comprising an indexing component configured to:

generate index data based at least partially upon the N number of data objects of the first set; and

store the generated index data in an index.

11. The system of claim 6 wherein the secondary storage computing device is further configured to generate the substantially unique identifier for each of the N number of substantially similar data objects of the first set.

12. The system of claim 6 wherein the substantially unique identifier for each of the N number of substantially similar data objects of the first set is generated by the one or more computing devices.

13. A method in a computing system of storing multiple instances of substantially similar data objects, wherein the method is performed by a computing system having a processor and memory, the method comprising:

receiving an indication, based on a storage policy, to perform a storage operation on a first set of data stored by one or more computing devices,

wherein the first set of data corresponds to a first tier of data storage for primary data,

wherein the first tier of data storage comprises disk media; and

for at least a first data object in the first set of data;

determining a priority of the first data object,

wherein the priority of the first data object is based on at least two of;

an owner of the first data object;

a content of the first data object;

an association of the first data object with an organization unit;

a security descriptor of the first data object;

determining, by the computing system, a first number of instances of the first data object that should be stored in a second set of data distinct from the first set of data,

wherein the first number is greater than one,

wherein the second set of data corresponds to a second tier of data storage for secondary data,

wherein the second tier of data storage comprises tape media,

wherein the first set of data is aged off of the first tier of data storage to the second tier of data storage, and

wherein the determined number that should be stored is based on the storage policy;

adjusting the first number of instances based on the priority of the first data object after said determining the first number of instances of the first data object;

generating a substantially unique identifier that represents the first data object;

based on the generated substantially unique identifier, determining a second number of instances of the first data object already stored in the second set of data; and

when the adjusted first number is greater than the second number, storing a third number of instances of the first data object in the second set of data,

wherein the third number is equal to the difference between the adjusted first number and the second number.

14. The method of claim 13 wherein the second set of data is stored on one or more storage devices, and wherein the method further comprises determining at least one storage device on which to store the at least one instance of the first data object, wherein storing the at least one instance of the first data object in the second set of data includes storing the at least one instance of the first data object on the at least one storage device.

15. The method of claim 13 wherein the second set of data is stored on one or more storage devices, and wherein the method further comprises determining at least one storage device on which to store the at least one instance of the first data object based upon one or more criteria, wherein the criteria include at least two of:

a power consumption of the at least one storage device;

a storage capacity of the at least one storage device;

a throughput of storage operations performed using the at least one storage device;

an availability of the at least one storage device; and

an access time for data stored on the at least one storage device.

16. The method of claim 13, further comprising, for at least the first data object in the first set of data:

determining a size of the first data object;

determining if the size of the first data object exceeds a threshold amount; and

if the size of the first data object exceeds the threshold amount, then decreasing the first number of instances of the first data object that should be stored in the second set of data.

17. The method of claim 13, further comprising, for at least the first data object in the first set of data:

determining a fourth number of instances of the first data object in the first set of data;

determining if the fourth number of instances of the first data object in the first set of data exceeds a threshold number; and

when the fourth number of instances of the first data object in the first set of data exceeds the threshold number, then storing at least one additional instance of the first data object in the second set of data.

18. The method of claim 13, further comprising, for at least the first data object in the first set of data:

determining if the priority of the first data object exceeds a predetermined priority; and

when the priority of the first data object exceeds the predetermined priority, then storing at least one additional instance of the first data object in the second set of data.