INTELLIGENT DATA SOURCING IN A NETWORKED STORAGE SYSTEM

• US 20130339390A1
• Filed: 06/12/2013
• Published: 12/19/2013
• Est. Priority Date: 06/13/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Muller, Marcus S., Ngo, David
• Granted Patent: US 9,218,376 B2
• Time in Patent Office: Days
• US Class Current: 707/782
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/1847 specifically adapted to static storage, e.g. adapted to flash memory or SSD
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 : Synchronous replication
• CPC Class Codes: G06F 16/90 : Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/95 : Retrieval from the web
• CPC Class Codes: G06F 2201/80 : Database-specific techniques

Abstract

A storage system according to certain embodiments includes a repository of client-side data block signature information representative of a set of data blocks stored in a primary storage subsystem. In some cases, the system sources data blocks for secondary copy and restore operations from the primary storage subsystem instead of the secondary storage subsystem. Where multiple primary storage components (e.g., multiple client computing devices) contain copies of a data blocks involved in a secondary copy or restore operation, the system can decide which client to source the data block from based on sourcing criteria.

Claims

1. A method of sourcing data from storage associated with a pool of computing devices during a data storage operation associated with one of the computing devices in the pool, the method comprising:

obtaining signatures corresponding to data units that form a data set associated with a data storage operation, the data set corresponding to a version of primary data of a first computing device in a pool of a plurality of computing devices, each respective computing device in the pool storing primary data generated by one or more software applications executing on the respective computing device, the primary data stored in at least one storage device associated with the respective computing device;

populating, by one or more processors, a shared signature repository that includes;

signatures corresponding to at least each unique data unit stored in the storage devices of the computing devices in the pool; and

for each signature included in the signature repository, an indication as to one or more of the computing devices whose at least one storage device includes a copy of the data unit corresponding to the signature;

comparing the obtained signatures with the signature repository to identify one or more of the computing devices in the pool whose respective at least one storage devices include copies of data units in the data set;

consulting, by one or more processors, a priority policy; and

based on the priority policy, and for at least some data units in the backup set, deciding to access copies of the at least some data units from one or more computing devices in the pool other than the first computing device.

2. The method of claim 1, wherein the priority policy includes an indication as to a relative priority of one or more of the computing devices with respect to one or more others of the computing devices.

3. The method of claim 1, wherein the data storage operation comprises a backup copy operation in which a copy of the data set is stored in secondary storage that is separate from the pool of computing devices and from each at least one storage device of the respective computing devices.

4. The method of claim 1, wherein the data storage operation comprises a restore operation in which the data set is restored to the at least one storage device associated with the first client computing device.

5. The method of claim 4, wherein at least some of the data units in the data set are sourced from secondary storage which is separate from the pool of computing devices and is separate from each at least one storage device of the respective computing devices.

6. The method of claim 1, wherein said consulting is performed by a data sourcing module executing on one or more processors of a computing device that is separate from the pool, and wherein the signature repository is separate from each of the at least one storage devices.

7. The method of claim 1, wherein at least 10 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

8. The method of claim 1, wherein at least 25 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

9. The method of claim 1, wherein at least 50 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

10. The method of claim 1, wherein all of the data units are sourced from the at least one storage device of one or more computing devices in the pool other than the first computing device.

11. A storage system for sourcing data from storage associated with a pool of computing devices during a data storage operation associated with one of the computing devices in the pool, the storage system comprising:

a global signature repository including;

signatures corresponding to at least each unique data unit stored in at least one storage device associated with each of a plurality of computing devices in a pool; and

for each signature included in the signature repository, an indication as to one or more of the plurality of computing devices whose at least one storage device includes a copy of the data unit corresponding to the signature; and

a repository agent executing in one or more processors and configured to;

obtain signatures corresponding to data units that form a data set associated with a data storage operation, the data set corresponding to a version of primary data of a first computing device in the pool, each respective computing device in the pool storing primary data generated by one or more software applications executing on the respective computing device and stored in at least one storage device associated with the respective computing device;

compare the obtained signatures with the signature repository to identify one or more of the computing devices in the pool whose respective at least one storage devices include copies of data units in the data set;

consult a priority policy; and

based on the priority policy and for at least some data units in the backup set, decide to access copies of the at least some data units from one or more computing devices in the pool other than the first computing device.

12. The system of claim 11, wherein the priority policy includes an indication as to a relative priority of one or more of the computing devices with respect to one or more others of the computing devices.

13. The system of claim 11, wherein the data storage operation comprises a backup copy operation in which a copy of the data set is stored in secondary storage that is separate from the plurality of computing devices and from each at least one storage device of the respective computing devices.

14. The system of claim 11, wherein the data storage operation comprises a restore operation in which the data set is restored to the at least one storage device associated with the first client computing device.

15. The system of claim 14, wherein at least some of the data units in the data set are sourced from secondary storage which is separate from the plurality of computing devices and is separate from each at least one storage device of the respective computing devices.

16. The system of claim 11, wherein the repository agent executes on one or more processors of a computing device that is separate from the pool and wherein the signature repository is separate from the each of the at least one storage devices.

17. The system of claim 11, wherein at least 10 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

18. The system of claim 11, wherein at least 25 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

19. The system of claim 11, wherein at least 50 percent of the data units are sourced from the at least one storage device of computing devices in the pool other than the first computing device.

20. The system of claim 11, wherein all of the data units are sourced from the at least one storage device of one or more computing devices in the pool other than the first computing device.