Transferring or migrating portions of data objects, such as block-level data migration or chunk-based data migration

• US 10,572,445 B2
• Filed: 12/18/2013
• Issued: 02/25/2020
• Est. Priority Date: 09/12/2008
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Lad, Kamleshkumar K.
• Primary Examiner(s): Hutton, Nan
• Application Number:
  US14/132,458
• Publication Number:
  US 20140108470A1
• Time in Patent Office: 2,260 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A system and method for migrating data objects based on portions of the data objects is described. The system may transfer portions of files, folders, and other data objects from primary storage to secondary storage based on certain criteria, such as time-based criteria, age-based criteria, and so on. An increment may be one or more blocks of a data object, or one or more chunks of a data object, or other segments that combine to form or store a data object. For example, the system identifies one or more blocks of a data object that satisfy a certain criteria, and migrates the identified blocks. The system may determine that a certain number of blocks of a file have not been modified or called by a file system in a certain time period, and migrate these blocks to secondary storage.

Claims

1. At least one non-transitory, computer-readable storage medium comprising instructions, which when executed by at least one data processor, enable restoration from secondary storage of a portion of a file in a data storage system, comprising:

receiving, via a user interface associated with a file system, a request from a user to modify a portion of a file in the file system,

wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;

identifying one or more data blocks within the second storage device that contain data associated with the portion of the file requested to be modified;

retrieving from the second storage device the one or more data blocks without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;

presenting to the user data contained by the one or more data blocks retrieved from the second storage device, via the user interface associated with the file system;

upon receiving input from the user to modify the portion of the file, transferring data associated with the received input for storage by one of the first storage device and the second storage device; and

maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the received input to modify the portion of the file.

2. The at least one non-transitory, computer-readable storage medium of claim 1, wherein the data structure comprises one or more of:

a bitmap, a table, and a log.

3. The at least one non-transitory, computer-readable storage medium of claim 1, wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.

4. The at least one non-transitory, computer-readable storage medium of claim 1, wherein the identifying one or more data blocks identifies one or more chunks within the second storage device.

5. The at least one non-transitory, computer-readable storage medium of claim 1, further comprising:

dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of;

a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and

updating an index associated with the file to include information associating the identified one or more data blocks with the second storage device.

6. A method in a data storage system for restoring a portion of a file, the method comprising:

receiving, via a user interface associated with a file system that comprises a file, a request from a user to modify only a portion of the file in the file system,

wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;

determining one or more data blocks stored within the second storage device that contain data associated with the portion of the file requested to be modified;

retrieving from the second storage device the one or more data blocks that contain data associated with the portion of the file requested to be modified without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;

in response to the request, presenting to the user via the user interface associated with the file system one or more data blocks retrieved from the second storage device;

upon receiving input from the user, via the user interface, to modify the portion of the file, transferring data blocks that were changed by the received input for storage by one of the first storage device and the second storage device; and

maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the received input to modify the portion of the file.

7. The method of claim 6, wherein the data structure at the intermediate component comprises a bitmap.

8. The method of claim 6, wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.

9. The method of claim 6, wherein the identifying one or more data blocks identifies one or more chunks within the second storage device.

10. The method of claim 6, further comprising:

dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of;

a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and

updating an index associated with the file to include information associating the identified one or more data blocks with the storage device.

11. A system for restoring a portion of a file, the system comprising:

at least one processor;

means, at a file system, for receiving a request from a user to modify only a portion of a file in the file system,

wherein the file is partially stored in secondary storage on a second storage device that is distinct from a first storage device storing the file system;

means for identifying one or more data blocks stored within the second storage device that contain data associated with the portion of the file requested to be modified;

means for retrieving from the second storage device the one or more data blocks without retrieving data blocks therefrom that are not associated with the portion of the file requested to be modified;

means for presenting in a user interface the portion of the file requested to be modified, including the data blocks retrieved from the second storage device;

means for transferring data blocks that were changed by input received from the user to modify the portion of the file, for storage by one or more of the first storage device and the second storage device; and

means for maintaining a data structure in a memory of an intermediate component that resides between the file system and the second storage device, wherein the data structure reflects information about changes to the file, including information about data blocks that were changed by the input to modify the portion of the file.

12. The system of claim 11, wherein the identified one or more data blocks are a proper subset of a set of data blocks that contain file data.

13. The system of claim 11, wherein the means for identifying one or more data blocks identifies one or more chunks within the second storage device.

14. The system of claim 11, wherein the data structure is configured as a bitmap.

15. The system of claim 11, further comprising:

means for dividing data already stored in the second storage device into subsets of data based on logical divisions including one or more of;

a date of creation, a date of deletion, a date of reception, and a size of data to be divided; and

means for updating an index associated with the file to include information associating the identified one or more data blocks with the second storage device.

16. The system of claim 11, wherein the file is a slide presentation and wherein the data contained by the identified one or more data blocks includes data associated with a single slide of the slide presentation.

17. The system of claim 11, wherein the file is a .pst file for an email mailbox associated with the user and wherein the identified one or more data blocks comprise one or more emails within the .pst file.