Data transfer techniques within data storage devices, such as network attached storage performing data migration

• US 10,547,678 B2
• Filed: 12/09/2015
• Issued: 01/28/2020
• Est. Priority Date: 09/15/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Prahlad, Anand, Kottomtharayil, Rajiv, Vijayan, Manoj Kumar, Gokhale, Parag, Attarde, Deepak Raghunath, Lad, Kamleshkumar K.
• Primary Examiner(s): Gofman, Alex
• Assistant Examiner(s): Black, Linh
• Application Number:
  US14/963,954
• Publication Number:
  US 20160100007A1
• Time in Patent Office: 1,511 Days
• Field of Search: 707624, 707647, 707653, 707662, 707681, 707685, 707694, 707741, 707745, 707822, 707827
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

A stand-alone, network accessible data storage device, such as a filer or NAS device, is capable of transferring data objects based on portions of the data objects. The device transfers portions of files, folders, and other data objects from a data store within the device to external secondary storage based on certain criteria, such as time-based criteria, age-based criteria, and so on. A portion may be one or more blocks of a data object, or one or more chunks of a data object, or other segments that combine to form or store a data object. For example, the device identifies one or more blocks of a data object that satisfy a certain criteria, and migrates the identified blocks to external storage, thereby freeing up storage space within the device. The device may determine that a certain number of blocks of a file have not been modified or called by a file system in a certain time period, and migrate these blocks to secondary storage.

Claims

1. A network attached storage (NAS) device, wherein the network attached storage device is configured to be connected to a networked computing system, and wherein the networked computing system includes one or more non-volatile secondary data storage devices and one or more client computers connected via a network, the network attached storage device comprising:

a housing containing one or more components, the components including;

a data reception component, wherein the data reception component is configured to receive for storage multiple data files from the one or more client computers via the network;

an operating system, wherein the operating system is configured to provide a computing environment for the network attached storage device;

at least one processor, wherein the at least one processor is programmed to perform one or more data storage functions for the network attached storage device;

a non-volatile data store, wherein the non-volatile data store is configured to store the data files received from the data reception component;

a data interception component, wherein the data interception component is configured to intercept data transferred from the data reception component to the non-volatile data store and to update an index associating information identifying the transferred data with information identifying a time of transfer to the non-volatile data store;

a file system, wherein the file system is configured to manage, for the network attached storage device, the writing of data files to, and the reading of data files from, the non-volatile data store;

one or more media agents, wherein the one or more media agents are configured to receive instructions from the at least one processor and to transfer data stored in the non-volatile data store of the network attached storage device to the one or more non-volatile secondary data storage devices,

wherein the one or more non-volatile secondary data storage devices are external to the network attached storage device but are accessible by the one or more media agents of the network attached storage device via the network; and

a data migration component, wherein the data migration component is configured to identify portions of at least some of the data files within the non-volatile data store, and to migrate the identified data file portions from the network attached storage device to the one or more non-volatile secondary data storage devices, to thereby free up storage space in the non-volatile data store for the storage of one or more other data files,

wherein each of the data files is an individual file,

wherein the identified data file portions are less than all of the selected data file,

wherein the identified data file portions are for storage by the one or more media agents to at least one of the one or more non-volatile secondary storage devices,

wherein the data migration component is further configured to identify portions of a selected data file within the non-volatile data store based at least in part on a data storage criterion,

wherein the data storage criterion is associated with writing data to, and reading data from, portions of the selected data file,

wherein the data storage criterion includes data file portions that have not been modified within a predetermined period of time, and,

wherein the data migration component maintains a data structure that;

tracks a logical location of the identified data file portions stored in the one or more secondary storage devices, and

maps the identified data file portions to the selected data file.

2. The network attached storage device of claim 1, further comprising:

a data interception component, wherein the data interception component is configured to intercept access requests from the file system to the non-volatile data store and to update an index associating information identifying the access requests with information identifying a time of the access requests.

3. The network attached storage device of claim 1, wherein the data files consist of multiple blocks and wherein the data migration component identifies at least one block for migration from the network attached storage device, but not all of the multiple blocks, wherein the one block has an oldest access time as compared to other of the multiple blocks.

4. The network attached storage device of claim 1, wherein the data migration component is configured to identify data blocks in the non-volatile data store that have not been accessed by the file system within a predetermined time period.

5. The network attached storage device of claim 1 wherein the non-volatile data store further comprising a reserved read-back cache, and wherein the network attached storage device further comprises a callback component configured to read back the identified data file portions stored in the one or more secondary storage devices and to write the identified data file portions to the read-back cache, wherein the callback component is configured to read back the identified data file portions in response to a receipt of a request from the one or more client computers to access the identified data file portions.

6. A computer-implemented method for tracking at least a first portion of at least one data object within a network attached storage (NAS) device coupled to a network, wherein the NAS device includes a NAS file system and a non-volatile data store, the method comprising:

accessing calls to or from the NAS file system for reading of data from or writing of data to the non-volatile data store of the NAS device,

wherein the at least one data object consists of multiple data blocks,

wherein each data object is a single data object,

wherein the non-volatile data store of the NAS device stores the multiple data blocks of the data object;

wherein the NAS file system controls the reading of data from or the writing of data to the multiple data blocks of the data object, and wherein the accessing includes identifying individual blocks or groups of blocks within the multiple data blocks of the data object that the NAS file system reads data from or writes data to, wherein the identified individual blocks or groups of blocks are less that the entire data object;

based on the accessing, identifying a portion of the multiple data blocks of the data object that satisfies a data storage criterion, wherein the data storage criterion filters for the portion of the multiple data blocks that has not been modified since creation or that has not been modified within a predetermined period of time;

causing data stored in the portion of the multiple data blocks to be transferred to a separate non-volatile storage device, so as to free up available data storage on the NAS device for storage of other data objects, wherein the separate storage device is not contained by or within the NAS device but communicates with the NAS device over the network, wherein the network is a private network;

based on the identifying, and independently of the NAS file system of the NAS device,

updating a data structure, wherein the data structure,

tracks the portion of the multiple data blocks, and

provides an indication of the data object to which the portion of the multiple data blocks belongs;

updating the data structure to include information associating the portion of the multiple data blocks with the separate storage device,

wherein the data structure is an index is stored in the non-volatile data store of the NAS device; and

removing information from an allocation table associated with the NAS file system of the NAS device,

wherein the data object is a file, and wherein the portion of the multiple data blocks is less than all of the multiple data blocks for the file.

7. The method of claim 6,

wherein the separate storage device is not contained by or within the NAS device but communicates with the NAS device over the network, the method further comprising:

updating the data structure to track a location of the portion of the multiple data blocks as being located in the separate storage device.

8. The method of claim 6, wherein the data storage criterion comprises a time period in which to retain data in a cache of the NAS device, or a time period in which a recent access of the portion must occur of the multiple data blocks.

9. A stand-alone data storage device, coupled to one or more external computing devices over a network, wherein at least one non-volatile external storage device is also connected to the data storage device via the network, the data storage device comprising:

at least one processor;

a communication component coupled to the at least one processor and associated with a network address for the data storage device,

wherein the communication component receives data transfer commands from the one or more external computing devices on the network,

wherein the one or more external computing devices direct the data transfer commands to the data storage device via the network address for the data storage device, and

wherein the data transfer commands direct operation of the data storage device;

a non-volatile, internal data store, coupled to the at least one processor, wherein the internal data store stores data objects, wherein the data objects are comprised of multiple data blocks;

a data storage component that comprises program code, which when executed by the processor, performs data storage tasks with respect to the internal data store;

a file system that comprises program code, which when executed by the processor, stores and organizes data objects stored in the internal data store;

a call intercept layer, in communication with the file system, that comprises program code, which when executed by the processor, recognizes calls to or from the file system for reading of data from or writing of data to individual data blocks or groups of data blocks stored within the internal data store;

a data block identification component, in communication with the call intercept layer, that comprises program code, which when executed by the processor, identifies data blocks of the data object that satisfy a criterion,

wherein the criterion is associated with the recognized calls to or from the file system for the reading of data from or the writing of data to the individual data blocks or groups of data blocks,

wherein the criterion defines data blocks of the data object that have not been modified since creation or data blocks of the data object that have not been modified within a predetermined period of time,

wherein the identified data blocks of the data object are less than the whole data object;

an index component that comprises program code, which when executed by the processor, updates an index to include information associating the identified data blocks with information identifying the data object; and

a media agent, in communication with the data block identification component, that comprises program code, which when executed by the processor, copies or transfers, via the network and to the external storage device, data for no more than n−

1 identified data blocks to free up storage space on the data storage device for one or more other data objects, wherein at least one data object includes n number of data blocks.

10. The data storage device of claim 9 wherein at least one data object is a file having n number of data blocks,

wherein the index component includes a bitmap or a data location table mapping the no more than n−

1 identified data blocks to a logical location on the network.

11. The data storage device of claim 9 wherein at least one data object is a file having n number of data blocks, and

wherein the index component updates the index to include information associating the transferred data with information identifying tape offsets for the secondary storage device that contains the transferred data.

12. The data storage device of claim 9, wherein the criterion defines a time period related to when the file system last read data from or wrote data to individual data blocks or groups of data blocks.

13. The data storage device of claim 9, wherein the criterion defines a time period in which changes were made to data contained by blocks of the data object.

14. A network attached storage device, comprising:

a non-transitory cache, wherein the cache stores one or more data objects;

a media agent, wherein the media agent is configured to transfer portions of the one or more data objects from the cache to associated non-volatile secondary storage devices, wherein the transferring frees up storage space in the non-transitory cache for one or more other data objects to be stored at a location previously occupied by the portions of the one or more data objects,

wherein the portions of the one or more data objects are less than any one of the one or more data objects, and

wherein the secondary storage devices are located external to the network attached storage device and configured to provide long term storage of data; and

a data identification component, wherein the data identification component is configured to identify to the media agent the portions of the data objects to be transferred to the secondary storage devices based on one or more storage criteria, and

wherein the one or more storage criteria are met by the portions of the data objects that have not been modified since their creation, or the portions of the data objects that have not been modified within a predetermined period of time; and

an intermediate component, wherein the intermediate component tracks in an index all accesses of the one or more data objects by a file system within the network attached storage device, and

wherein the data identification component identifies the portions of the data objects based on information within the index.

15. The network attached storage device of claim 14, wherein the identified portions are proper subsets of data blocks of the data objects.

16. The network attached storage device of claim 14, wherein the identified portions include chunks of the data objects created from a rule-based process of dividing the data object into two or more portions.