DATA TRANSFER TECHNIQUES WITHIN DATA STORAGE DEVICES, SUCH AS NETWORK ATTACHED STORAGE PERFORMING DATA MIGRATION

• US 20100070466A1
• Filed: 09/14/2009
• Published: 03/18/2010
• Est. Priority Date: 09/15/2008
• Status: Abandoned Application

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Lad, Kamleshkumar K., Gokhale, Parag, Attarde, Deepak R., Prahlad, Anand, Kottomtharayil, Rajiv, Vijayan, Manoj K.
• Application Number:
  US12/558,640
• Publication Number:
  US 20100070466A1
• Time in Patent Office: Days
• US Class Current: 707/609
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A stand-alone, network accessible data storage device, such as a filer or NAS device, is capable of transferring data objects based on portions of the data objects. The device transfers portions of files, folders, and other data objects from a data store within the device to external secondary storage based on certain criteria, such as time-based criteria, age-based criteria, and so on. A portion may be one or more blocks of a data object, or one or more chunks of a data object, or other segments that combine to form or store a data object. For example, the device identifies one or more blocks of a data object that satisfy a certain criteria, and migrates the identified blocks to external storage, thereby freeing up storage space within the device. The device may determine that a certain number of blocks of a file have not been modified or called by a file system in a certain time period, and migrate these blocks to secondary storage.

Claims

1. A network attached storage (NAS) device, wherein the network attached storage device is configured to be connected to a networked computing system, the networked computing system including one or more secondary data storage devices and one or more client computers connected via a network, the network attached storage device comprising:

a housing containing one or more components, the components including;

a data reception component, wherein the data reception component is configured to receive for storage multiple data files from the one or more client computers via the network;

an operating system, wherein the operating system is configured to provide a computing environment for the network attached storage device;

at least one processor, wherein the at least one processor is programmed to perform one or more data storage functions for the network attached storage device;

a non-volatile data store, wherein the non-volatile data store is configured to store the data files received from the data reception component;

a file system, wherein the file system is configured to manage, for the network attached storage device, the writing of data files to, and the reading of data files from, the non-volatile data store;

one or more media agents, wherein the one or more media agents are configured to receive instructions from the at least one processor and to transfer data stored in the non-volatile data store to the one or more secondary data storage devices, wherein the one or more secondary data storage devices are external to the network attached storage device but are accessible by the one or more media agents via the network; and

a data migration component, wherein the data migration component is configured to identify portions of at least some of the data files within the non-volatile data store and to migrate the identified data file portions from the network attached storage device,

wherein the identified data file portions are for storage by the one or more media agents to at least one of the one or more secondary storage devices,

wherein the data migration component is further configured to identify portions of a selected data file within the non-volatile data store based at least in part on a data storage criteria,

wherein the data storage criteria is associated with the writing of data to, and the reading of data from, portions of the selected data file by the file system,

wherein the identified portions of the selected data file are less than all of the selected data file, and

wherein the data migration component maintains a data structure that

tracks a logical location of the identified data file portions stored in the one or more secondary storage devices, and

maps the identified data file portions to the selected data file.

2. The network attached storage device of claim 1, further comprising:

a data interception component, wherein the data interception component is configured to intercept data transferred from the data reception component to the non-volatile data store and to update an index associating information identifying the transferred data with information identifying a time of transfer to the non-volatile data store.

3. The network attached storage device of claim 1, further comprising:

a data interception component, wherein the data interception component is configured to intercept access requests from the file system to the non-volatile data store and to update an index associating information identifying the access requests with information identifying a time of the access requests.

4. The network attached storage device of claim 1, wherein the data files consist of multiple blocks and wherein the data migration component identifies at least one block for migration from the network attached storage device, but not all of the multiple blocks, wherein the one block has an oldest access time as compared to other of the multiple blocks.

5. The network attached storage device of claim 1, wherein the data migration component is configured to identify data blocks in the non-volatile data store that have not been accessed by the file system within a predetermined time period or that have not changed since a predetermined time.

6. The network attached storage device of claim 1 wherein the non-volatile data store further comprising a reserved read-back cache, and wherein the network attached storage device further comprises a callback component configured to read back the identified data file portions stored in the one or more secondary storage devices and to write the identified data file portions to the read-back cache.

7. A computer-implemented method for tracking at least a first portion of at least one data object within a network attached storage (NAS) device coupled to a network, wherein the NAS device includes a NAS file system and a non-volatile data store, the method comprising:

accessing calls to or from the NAS file system for reading of data from or writing of data to the non-volatile data store of the NAS device,

wherein the at least one data object consists of multiple data blocks,

wherein the non-volatile data store of the NAS device stores the multiple data blocks of the at least one data object;

wherein the NAS file system of the NAS device controls the reading of data from or the writing of data to the multiple data blocks of the at least one data object, and

wherein the accessing includes identifying individual blocks or groups of blocks within the multiple data blocks of the at least one data object that the NAS file system of the NAS device reads data from or writes data to;

based on the accessing, identifying a portion of the multiple data blocks of the at least one data object that satisfies a data storage criteria,

wherein the data storage criteria is a time-based criteria; and

,

based on the identifying, and independently of the NAS file system of the NAS device, updating a data structure, wherein the data structure—

tracks the portion of the multiple data blocks, and

provides an indication of the at least one data object to which the portion of the multiple data blocks belongs.

8. The method of claim 7, further comprising:

transferring data stored in the portion of the multiple data blocks to a separate storage device, wherein the separate storage device is not contained by or within the NAS device but communicates with the NAS device over the network, wherein the network is a private network;

updating the data structure to include information associating the portion of the multiple data blocks with the separate storage device, wherein the data structure is an index is stored in the non-volatile data store of the NAS device; and

removing information from an allocation table associated with the NAS file system of the NAS device, and

wherein the at least one data object is a file, and wherein the portion of the multiple data blocks is less than all of the multiple data blocks for the file.

9. The method of claim 7, further comprising:

transferring data stored in the portion of the multiple data blocks to a separate storage device, wherein the separate storage device is not contained by or within the NAS device but communicates with the NAS device over the network; and

,

updating the data structure to track a location of the portion of the multiple data blocks as being located in the separate storage device.

10. The method of claim 7, wherein the data storage criteria comprises a time period in which to retain data in a cache of the NAS device, or a time period in which a recent access of the portion must occur of the multiple data blocks.

11. A stand-alone data storage device, coupled to one or more external computing devices over a network, wherein at least one external storage device is also connected to the data storage device via the network, the data storage device comprising:

at least one processor;

a communication component coupled to the at least one processor and associated with a network address for the data storage device,

wherein the communication component receives data transfer commands from the one or more external computing devices on the network,

wherein the one or more external computing devices direct the data transfer commands to the data storage device via the network address for the data storage device, and

wherein the data transfer commands direct operation of the data storage device;

a non-volatile, internal data store, coupled to the at least one processor, wherein the internal data store stores data objects, wherein at least some of the data objects are comprised of multiple data blocks;

a data storage component that comprises program code, which when executed by the processor, performs data storage tasks with respect to the internal data store;

a file system that comprises program code, which when executed by the processor, stores and organizes data objects stored in the internal data store;

a call intercept layer, in communication with the file system, that comprises program code, which when executed by the processor, recognizes calls to or from the file system for reading of data from or writing of data to individual data blocks or groups of data blocks stored within the internal data store;

a data block identification component, in communication with the call intercept layer, that comprises program code, which when executed by the processor, identifies data blocks of the data object that satisfy a criteria, wherein the criteria is associated with the recognized calls to or from the file system for the reading of data from or the writing of data to the individual data blocks or groups of data blocks; and

an index component that comprises program code, which when executed by the processor, updates an index to include information associating the identified data blocks with information identifying the data object.

12. The data storage device of claim 11 wherein at least one data object is a file having n number of data blocks, and wherein the data storage device further comprises:

a media agent, in communication with the data block identification component, that comprises program code, which when executed by the processor, copies or transfers, via the network and to the external storage device, data for no more than n-1 identified data blocks; and

wherein the index component includes a bitmap or a data location table mapping the no more than n-1 identified data blocks to a logical location on the network.

13. The data storage device of claim 11 wherein at least one data object is a file having n number of data blocks, and wherein the data storage device further comprises:

a media agent, in communication with the data block identification component, that comprises program code, which when executed by the processor, copies or transfers, via the network and to the external storage device, data for no more than n-1 identified data blocks; and

wherein the index component updates the index to include information associating the transferred data with information identifying tape offsets for the secondary storage device that contains the transferred data.

14. The data storage device of claim 11, wherein the criteria defines a time period related to when the file system last read data from or wrote data to individual data blocks or groups of data blocks.

15. The data storage device of claim 11, wherein the criteria defines a time period in which changes were made to data contained by blocks of the data object.

16. A method performed by a network attached storage device for storing a portion of a data object in a secondary storage device associated with the network attached storage device, the method comprising:

identifying, within a cache of the network attached storage device, a data object having at least a first portion, wherein the first portion was last accessed by a file system of the network attached storage device before a predetermined time,

wherein a second portion of the data object was last accessed by the file system after the predetermined time;

transferring the first portion of the data object out of the network attached storage and to a secondary storage device communicatively coupled with the network attached storage device,

wherein the secondary storage device is external to the network attached storage device; and

maintaining the second portion of the data object in the cache of the network attached storage device.

17. A network attached storage device, comprising:

a cache, wherein the cache stores one or more data objects;

a media agent, wherein the media agent is configured to transfer portions of the one or more data objects from the cache to associated secondary storage devices, wherein the secondary storage devices are located external to the network attached storage device and configured to provide long term storage of data; and

a data identification component, wherein the data identification component is configured to identify to the media agent the portions of the data objects to be transferred to the secondary storage devices based on one or more storage criteria.

18. The network attached storage device of claim 17, further comprising:

an intermediate component, wherein the intermediate component tracks in an index all accesses of the one or more data objects by a file system within the network attached storage device, and wherein the data identification component identifies the portions of the data objects based on information within the index.

19. The network attached storage device of claim 17, wherein the identified portions are proper subsets of data blocks of the data objects.

20. The network attached storage device of claim 17, wherein the identified portions include chunks of the data objects created from a rule-based process of dividing the data object into two or more portions.

21. A system for storing a portion of a data object in a secondary storage device associated with a network attached storage (NAS) device, the system comprising:

at least one processor;

a data store;

callback layer and block identifying means for identifying, from calls to or from a file system within the NAS device, a first portion a data object within the data store of the NAS device, wherein the first portion was last accessed by the file system of the network attached storage device before a predetermined time, wherein a second portion of the data object was last accessed by the file system after the predetermined time;

media agent means for transferring the first portion of the data object to a secondary storage device associated with the network attached storage device; and

index means for tracking the second portion of the data object in the data store of the network attached storage device.

22. The system of claim 21, further comprising a read-back cache, formed from the data store, and wherein the callback layer and block identifying means is further configured for reading back the first portion of the data object stored to the secondary storage device and for writing the first portion to the read-back cache.