STORING A VARIABLE NUMBER OF INSTANCES OF DATA OBJECTS

• US 20100250549A1
• Filed: 12/30/2009
• Published: 09/30/2010
• Est. Priority Date: 03/30/2009
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Attarde, Deepak R., Vijayan, Manoj Kumar, Muller, Marcus S.
• Granted Patent: US 8,401,996 B2
• Time in Patent Office: Days
• US Class Current: 707/741
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Described are systems and methods for storing a variable number of instances of data objects (e.g., 1, 2, 3, or up to N-1 instances, where N is the number of instances of the data object included in primary data) in secondary storage across a data storage network. In some examples, a system for storing a variable number of instances of data objects includes, one or more computing devices storing a set of data objects and multiple storage devices distinct from the one or more computing devices. Each of the multiple storage devices is configured to store at least a single instance of a data object. The system also includes a database configured to store information associated with the data objects. This information includes substantially unique identifiers for the data objects and, for each of the data objects, a number of instances of the data object stored on the multiple storage devices.

Claims

1. A method in a data storage system of storing a variable number of instances of data objects, wherein the method is performed by a computing system having a processor and memory, the method comprising:

accessing a set of data objects stored in a first set of one or more data stores, wherein the set of data objects includes a plurality of data objects that are substantially similar, wherein the plurality includes a number N data objects;

determining, by the computing system, a number X of instances of a first data object of the plurality of data objects that should be stored in a second set of one or more data stores, where X is greater than one and less than N, wherein the intersection of the first and second sets of data stores is empty;

determining a number Y of instances of the first data object that are stored in the second set of one or more data stores, where Y is greater than or equal to zero;

determining if X is greater than Y; and

when X is greater than Y, storing a number Z of instances of the first data object in the second set of one or more data stores, where Z is equal to X minus Y.

2. The method of claim 1 wherein determining the number X includes:

analyzing a storage policy; and

based at least in part on the analyzing of the storage policy, determining the number X of instances of the first data object of the plurality of data objects that should be stored in the second set of one or more data stores.

3. The method of claim 1 wherein determining the number Y includes:

generating a substantially unique identifier for the first data object;

analyzing a data structure; and

based at least in part on the analyzing of the data structure, determining the number Y of instances of the first data object that are stored in the second set of one or more data stores.

4. The method of claim 1, further comprising:

receiving an indication to perform a storage operation on the set of data objects; and

performing the storage operation on the set of data objects, wherein performing the storage operation includes copying Z instances of the first data object to the second set of one or more data stores.

5. The method of claim 1 wherein determining the number X includes:

determining a characteristic of the first data object, wherein the characteristic includes at least one of;

an owner of the first data object;

a content of the first data object;

a value of the first data object; and

the number N of data objects included in the plurality of data objects; and

based at least in part on the determined characteristic of the first data object, determining the number X of instances of the first data object of the plurality of data objects that should be stored in the second set of one or more data stores.

6. The method of claim 1, further comprising determining the second set of one or more data stores based at least in part upon one or more characteristics of at least one data store in the second set, wherein the one or more characteristics include at least one of:

a storage capacity of the at least one data store;

a power consumption of the at least one data store;

a time to store data on the at least one data store; and

a time to access data stored on the at least one data store.

10. The method of claim 1 wherein Z is greater than one, wherein the second set of one or more data stores includes at least a first data store and a second data store, and wherein storing Z instances of the first data object in the second set of one or more data stores includes:

storing a first instance of the first data object in a first data store; and

storing a second instance of the first data object in a second data store.

11. The method of claim 1, further comprising:

monitoring the plurality of data objects;

determining when the number N of data objects included in the plurality of data objects exceeds a predetermined threshold; and

when the number N of data objects exceeds the predetermined threshold. storing at least one instance of the first data object in the second set of data stores.

7-9. -9. (canceled)

12. A system for storing multiple instances of a data object in a data storage network, wherein the data storage network includes multiple data storage devices coupled via a computer network, and wherein the computer network also couples to one or more computing devices storing a first set of N number of substantially similar data objects, the system comprising:

multiple data storage devices storing a second set of data objects, wherein the multiple data storage devices are distinct from, and coupled via the computer network to, the one or more computing devices;

a variable instance database configured to store information associated with the second set of data objects, wherein the information includes, for each of the data objects of the second set;

a substantially unique identifier for the data object, and

a number of instances of the data object in the second set of data objects, wherein at least one of the data objects of the second set has multiple instances in the second set of data objects;

a secondary storage computing device configured to;

receive an indication to perform a storage operation on the first set of N number of substantially similar data objects, where N is a number greater than 3,

determine a substantially unique identifier for each of the N number of substantially similar data objects of the first set;

search for the substantially unique identifier in the variable instance database;

determine the number of instances of the N number of substantially similar data objects to store in the second set of data objects; and

store the determined number of instances of the N number of substantially similar data objects in the second set of data objects.

13. The system of claim 12 wherein the one or more computing devices storing a first set of N number of substantially similar data objects are at a first hierarchical tier of the data storage network and the multiple data storage devices are at a second hierarchical tier of the data storage network.

14. The system of claim 12 wherein the secondary storage computing device is further configured to receive an indication that specifies at least one of the multiple data storage devices on which to store an instance of the N number of substantially similar data objects.

15. The system of claim 12 wherein the secondary storage computing device is further configured to receive an indication that specifies that at least one of the multiple storage devices on which to store an instance of the N number of substantially similar data objects is to be dynamically determined.

16. The system of claim 12, further comprising a storage manager component configured to:

receive an indication to perform the storage operation on the N number of substantially similar data objects of the first set; and

send the indication to the secondary storage computing device.

17. The system of claim 12, further comprising:

a first storage manager component configured to;

receive an indication to perform the storage operation on the N number of data objects of the first set; and

send the indication to the secondary storage computing device; and

a second storage manager component configured to;

receive the indication to perform the storage operation on the N number of data objects of the first set; and

send the indication to the first storage manager component,

wherein the first storage manager component is hierarchically subordinate to the second storage manager component in the data storage network.

18. The system of claim 12, further comprising an indexing component configured to:

generate index data based at least partially upon the N number of data objects of the first set; and

store the generated index data in an index.

19. The system of claim 12 wherein the secondary storage computing device is further configured to generate the substantially unique identifier for each of the N number of substantially similar data objects of the first set.

20. The system of claim 12 wherein the substantially unique identifier for each of the N number of substantially similar data objects of the first set is generated by the one or more computing devices.

21. A method in a computing system of storing multiple instances of substantially similar data objects, wherein the method is performed by a computing system having a processor and memory, the method comprising:

receiving an indication to perform a storage operation on a first set of data stored by one or more computing devices; and

for at least a first data object in the first set of data;

determining, by the computing system, a first number of instances of the first data object that should be stored in a second set of data distinct from the first set of data, where the first number is greater than one;

generating a substantially unique identifier that represents the first data object;

based on the generated substantially unique identifier, determining a second number of instances of the first data object already stored in the second set of data; and

when the first number is greater than the second number, storing at least one instance of the first data object in the second set of data.

22. The method of claim 21 wherein the second set of data is stored on one or more storage devices, and wherein the method further comprises determining at least one storage device on which to store the at least one instance of the first data object, wherein storing the at least one instance of the first data object in the second set of data includes storing the at least one instance of the first data object on the at least one storage device.

24. The method of claim 21 wherein the second set of data is stored on one or more storage devices, and wherein the method further comprises determining at least one storage device on which to store the at least one instance of the first data object based upon one or more criteria, wherein the criteria include at least one of:

a power consumption of the at least one storage device;

a storage capacity of the at least one storage device;

a throughput of storage operations performed using the at least one storage device;

an availability of the at least one storage device; and

an access time for data stored on the at least one storage device.

26. The method of claim 21, further comprising, for least the first data object in the first set of data:

determining a size of the first data object;

determining if the size of the first data object exceeds a threshold amount; and

if the size of the first data object exceeds the threshold amount, then decreasing the first number of instances of the first data object that should be stored in the second set of data.

27. The method of claim 21, further comprising, for at least the first data object in the first set of data:

determining a third number of instances of the first data object in the first set of data;

determining if the third number of instances of the first data object in the first set of data exceeds a threshold number; and

when the third number of instances of the first data object in the first set of data exceeds the threshold number, then storing at least one additional instance of the first data object in the second set of data.

28. The method of claim 21, further comprising, for at least the first data object in the first set of data:

determining a priority of the first data object;

determining if the priority of the first data object exceeds a predetermined priority; and

when the priority of the first data object exceeds the predetermined priority, then storing at least one additional instance of the first data object in the second set of data.

23. (canceled)

25. (canceled)

29. A computing system for storing multiple instances of a data object, the computing system comprising:

means for receiving an indication to perform a storage operation on a set of data objects stored by one or more computing devices;

means for determining a first number of instances of a first data object in the set of data objects that should be stored on one or more storage devices distinct from the one or more computing devices, wherein the first number is greater than one;

means for generating a substantially unique identifier that represents the first data object;

means for determining, based on the generated substantially unique identifier, a second number of instances of the first data object stored on the one or more storage devices; and

means for storing a third number of instances of the first data object on the one or more storage devices, where the third number is determined with reference to an amount by which the first number exceeds the second number.

30-38. -38. (canceled)

39. A system for storing multiple instances of a data object in a data storage network, wherein the data storage network includes at least two different hierarchical tiers of data storage devices coupled via a computer network, and wherein the computer network also couples to a computing device storing a primary copy of data that includes a first set of N number of the same data objects, the system comprising:

multiple data storage devices, wherein each of the multiple storage devices is distinct from, and coupled via the computer network to, the computing device, wherein at least two of the multiple storage devices are arranged into two different hierarchical tiers;

a variable instance database configured to store information associated with secondary copies of data that includes a second set of reduced instanced data objects, wherein the information includes, for each of the reduced instanced data objects of the second set;

a substantially unique identifier generated for the reduced instanced data object, and

a number of instances to store for the reduced instanced data object, wherein at least one of the reduced instanced data objects of the second set has multiple instances stored among the multiple storage devices;

a secondary storage computing device configured to;

receive an indication to perform a storage operation on the primary copy of data, including the first set of N number of the same data objects, where N is a number greater than 3,

wherein the indication specifies that fewer than N instances but greater than one instance of the N number of data objects of the first set are to be stored, and

wherein each of the data objects of the first set are to be stored on a distinct one of the multiple storage devices;

generate a substantially unique identifier for each of the N number of data objects of the first set;

access the variable instance database and for the N number of data objects of the first set, use the generated substantially unique identifier to determine the fewer than N instances but greater than one instance to store for the N number of data objects; and

for each one of the N number of data objects of the first set, perform the storage operation on the one data object, wherein performing the storage operation includes storing the fewer than N instances but greater than one instance of the data object on the multiple storage devices.

40. The system of claim 39 wherein the indication to perform the storage operation on the N number of data objects of the first set further specifies a number selected by an administrator, wherein storing the fewer than N instances but greater than one instance of the data object on the multiple storage devices includes storing one copy of the data object on storage devices associated with two different hierarchical tiers, and wherein the secondary storage computing device is further configured to identify the N data objects of the first set before generating the substantially unique identifier for each of the N number of data objects of the first set.

42. The system of claim 39 wherein the indication to perform the storage operation on the N number of data objects of the first set further specifies that the multiple storage devices on which to store instances of the data object are to be dynamically determined.

44. The system of claim 39, further comprising:

a first storage manager component configured to;

receive an indication to perform a storage operation on the N number of data objects of the first set;

process the indication; and

send the processed indication to perform the storage operation to the secondary storage computing device; and

a second storage manager component configured to;

receive the indication to perform the storage operation on the N number of data objects of the first set;

process the indication; and

send the processed indication to perform the storage operation to the first storage manager component,

wherein the first storage manager component is hierarchically subordinate to the second storage manager component.

41. (canceled)

43. (canceled)

45. (canceled)