COLLABORATIVE BACKUP IN A NETWORKED STORAGE SYSTEM

• US 20130339298A1
• Filed: 06/12/2013
• Published: 12/19/2013
• Est. Priority Date: 06/13/2012
• Status: Abandoned Application

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Muller, Marcus S., Ngo, David
• Application Number:
  US13/916,385
• Publication Number:
  US 20130339298A1
• Time in Patent Office: Days
• US Class Current: 707/640
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/1847 specifically adapted to static storage, e.g. adapted to flash memory or SSD
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 : Synchronous replication
• CPC Class Codes: G06F 16/90 : Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/95 : Retrieval from the web
• CPC Class Codes: G06F 2201/80 : Database-specific techniques

Abstract

A storage system according to certain embodiments includes a client-side signature repository that includes information representative of a set of data blocks stored in primary storage. During copy or backup operations, the system can use the client-side signature repository to identify data blocks located in primary storage that are new or that have changed. The system can also use the client-side signature repository to identify multiple locations within primary storage where different instances of the data blocks are located. Accordingly, during a copy or backup operation of one client computing device, the system can source a data block that is to be copied to secondary storage from another client computing device that includes a second instance of the data block.

Claims

1. A method of generating a secondary copy data set for a client computing device by collaboratively sourcing data to be used in the secondary copy data set from at least one other client computing device, the method comprising:

for each respective client computing device of a plurality of client computing devices;

monitoring storage of a plurality of files formed by data blocks generated by one or more software applications running on the respective client computing device,

wherein the files are stored in a data store associated with the respective client computing device;

maintaining, by a signature repository agent executing on one or more processors, a global mapping indicating which data blocks are stored in the data stores associated with each of the plurality of client computing devices, wherein separate copies of at least some of the data blocks reside in the data stores of multiple ones of the plurality of client computing devices;

in response to instructions to create a secondary copy in secondary storage of at least a subset of the plurality of files stored in the data store of a first client computing device of the plurality of client computing devices,

querying, by the signature repository agent, the global mapping to identify at least a first group of data blocks in the subset of the plurality of files that are stored in the data store associated with a second client computing device of the plurality of client computing devices;

retrieving the first group of data blocks from the data store associated with the second client computing device; and

retrieving at least some of the remaining data blocks in the first portion from the data store associated with the first client computing device.

2. The method of claim 1, further comprising communicating the retrieved data blocks to the secondary storage, wherein the secondary copy is created using at least the retrieved data blocks.

3. The method of claim 1, wherein the signature repository agent resides on a computing device that is separate from each of the plurality of client computing devices.

4. The method of claim 3, wherein the global mapping is maintained in one or more storage devices that are separate from the data stores of the plurality of client computing devices.

5. The method of claim 3, further comprising maintaining signatures in the global mapping which correspond to the data blocks that are stored in the data store associated with each of the plurality of client computing devices.

6. The method of claim 5, further comprising including in the global mapping, for each data block referenced in the global mapping:

a client computing device identifier indicating which client computing device stores the referenced data block in its associated data store; and

a storage location for the referenced data block within the client computing device that stores the referenced block.

7. The method of claim 1, further comprising:

accessing priority information associated with the first client computing device and the second client computing device;

wherein said retrieving the first group of data blocks comprises retrieving the first group of data blocks from the data store associated with the second client computing device only if the priority information indicates that it is preferable to source the data from the data store associated with the second client computing device rather than from the data store associated with the first client computing device.

8. The method of claim 7, wherein the priority information comprises one or more of user-defined priority, processing load information, and an amount of available resources.

9. The method of claim 1, wherein the global mapping includes only metadata and does not include copies of the data blocks referenced in the global mapping.

10. The method of claim 1, wherein the secondary storage comprises deduplicated data.

11. The method of claim 1, wherein the data stores of plurality of client computing devices comprises deduplicated data.

12. A storage system for generating a secondary copy data set for a client computing device by collaboratively sourcing data to be used in the secondary copy data set from at least one other client computing device, the system comprising:

a global mapping stored in one or more storage devices and indicating which data blocks are stored in data stores associated with a plurality of client computing devices, wherein the data blocks are generated by one or more software applications running on the plurality of client computing devices;

a signature repository agent executing on a computing device and configured to;

maintain the global mapping indicating which data blocks are stored in the data stores associated with each of the plurality of client computing devices, wherein separate copies of at least some of the data blocks reside in the data stores of multiple ones of the plurality of client computing devices;

in response to instructions to create a secondary copy in secondary storage of at least a subset of data blocks stored in the data store associated with the first client computing device,

query the global mapping to identify at least a first group of data blocks in the subset of data blocks that are stored in the data store associated with a second client computing device of the plurality of client computing devices;

wherein the first group of data blocks is retrieved from the data store associated with the second client computing device; and

wherein at least some of the remaining data blocks in the subset of data blocks are retrieved from the data store associated with the first client computing device.

13. The system of claim 12, wherein the data blocks retrieved from the data stores of the first and second client computing devices are communicated to the secondary storage subsystem which creates the secondary copy using the communicated data blocks.

14. The system of claim 12, wherein the signature repository agent executes on a computing device that is distinct from each of the plurality of client computing devices.

15. The system of claim 14, wherein the global mapping is maintained in one or more storage devices that are separate from the data stores of the plurality of client computing devices.

16. The system of claim 14, wherein the global mapping includes signatures which correspond to the data blocks that are stored in the data store associated with each of the plurality of client computing devices.

17. The system of claim 14, wherein the one or more storage devices on which the global mapping is stored additionally store, for each data block referenced in the global mapping:

a client computing device identifier indicating which client computing device of the plurality of client computing devices stores the referenced data block in its associated data store; and

a storage location for the referenced data block within the client computing device that stores the referenced data block.

18. The system of claim 12, wherein the signature repository agent is further configured to access priority information associated with the first client computing device and the second client computing device,

wherein the first group of data blocks is retrieved from the data store associated with the second client computing device only if the priority information indicates that it is preferable to source the data from the data store associated with the second data store rather than from the data store associated with the first client computing device.

19. The system of claim 18, wherein the priority information comprises one or more of user-defined priority, processing load information, and an amount of available resources.

20. The system of claim 12, wherein the global mapping includes only metadata and does not include copies of the data blocks referenced in the global mapping.

21. The system of claim 12, wherein the secondary storage comprises deduplicated data.

22. The system of claim 12, wherein the data stores of the plurality of client computing devices comprise deduplicated data.

23. A computer-readable, non-transitory storage medium having one or more computer-executable modules for generating a secondary copy data set for a client computing device by collaboratively sourcing data to be used in the secondary copy data set from at least one other client computing device, the one or more computer-executable modules comprising:

a first module in communication with a plurality of client computing devices and configured to;

maintain a global mapping indicating which data blocks are stored in data stores associated with a plurality of client computing devices, wherein the data blocks are generated by one or more software applications running on the plurality of client computing devices, wherein separate copies of at least some of the data blocks reside in the data stores of multiple ones of the plurality of client computing devices;

in response to instructions to create a secondary copy in secondary storage of at least a subset of data blocks stored in the data store associated with the first client computing device,

query the global mapping to identify at least a first group of data blocks in the subset of data blocks that are stored in the data store associated with the second client computing device;

wherein the first group of data blocks is retrieved from the data store associated with the second client computing device; and

wherein at least some of the remaining data blocks in the subset of data blocks are retrieved from the data store associated with the first client computing device.

24. The storage medium of claim 23, wherein the data blocks retrieved from the data stores of the first and second client computing devices are communicated to the secondary storage subsystem which creates the secondary copy using the communicated data blocks.