Data recovery operations, such as recovery from modified network data management protocol data

• US 9,244,779 B2
• Filed: 09/23/2011
• Issued: 01/26/2016
• Est. Priority Date: 09/30/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Littlefield, Duncan A., Nallathambi, Vimal K., Chanchlani, Girish
• Primary Examiner(s): Alsip, Michael
• Application Number:
  US13/241,625
• Publication Number:
  US 20120084523A1
• Time in Patent Office: 1,586 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The systems and methods herein permit storage systems to correctly perform data recovery, such as direct access recovery, of Network Data Management Protocol (“NDMP”) backup data that was modified prior to being stored in secondary storage media, such as tape. For example, as described in greater detail herein, the systems and methods may permit NDMP backup data to be encrypted, compressed, deduplicated, and/or otherwise modified prior to storage. The systems and methods herein also permit a user to perform a precautionary snapshot of the current state of data (e.g., primary data) prior to reverting data to a previous state using point-in-time data.

Claims

1. A method of handling a Network Data Management Protocol (“

NDMP”

) backup or copy operation, the method comprising;

receiving an original NDMP backup data stream from a client;

modifying the received original NDMP backup data stream by performing the following modification procedures upon the original NDMP backup data stream;

deduplication, encryption, and compression,

wherein the modification procedures create a modified version of the NDMP backup data stream;

storing the modified version of the NDMP backup data stream in stored chunks,

wherein each chunk is addressable by a physical offset and a physical length,

wherein each chunk represents a modified version of a contiguous subset of the original NDMP backup data stream, and,

wherein the client associates a logical offset and a logical length with the contiguous subset of the original NDMP backup data stream that is represented by each chunk; and

,

for each stored chunk, associating the stored chunk with;

the physical offset and physical length that permits addressing of the chunk,

the logical offset and logical length that the client associates with the contiguous subset of the original NDMP backup data stream that is represented by the chunk, and

receiving a request for a certain subset of the original NDMP backup data stream, wherein the request includes a logical offset and a logical length;

using the received logical offset and logical length to;

identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and

determine a physical offset and a physical length that permits addressing of the identified chunk;

reading at least a portion of the identified chunk using the determined physical offset and physical length;

converting the portion of the identified chunk by reversing the modification procedures that were previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and

,

sending data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.

2. The method of claim 1, further comprising:

reading another portion of the identified chunk;

converting the other portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the other portion of the identified chunk;

buffering data from the converted version of the other portion of the identified chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and

,

responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

3. The method of claim 1, further comprising:

reading at least a portion of another chunk that represents a modified version of a contiguous subset of the original NDMP backup data stream that is adjacent to at least a portion of the requested subset of the original NDMP backup data stream;

converting the portion of the other chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the portion of the other chunk;

buffering data from the converted version of the portion of the other chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and

,

responding to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

4. The method of claim 1, wherein modifying the received original NDMP backup data stream further comprises adding metadata to the data stream.

5. The method of claim 1, wherein a storage policy specifies a configuration of the backup or copy operation.

6. The method of claim 1, wherein associating a stored chunk with a physical offset, physical length, logical offset, and logical length comprises associating a chunk identifier with the physical offset, physical length, logical offset, and logical length in a chunk mapping table.

7. The method of claim 1,

wherein modifying the received original NDMP backup data stream comprises performing deduplication;

and

wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.

8. The method of claim 1,

wherein modifying the received original NDMP backup data stream comprises performing deduplication;

and

wherein at least one chunk corresponds to a logical subdivision of a deduplication data structure that is created by the performance of deduplication.

9. The method of claim 1, wherein a storage policy specifies an approximate size of the stored chunks.

10. The method of claim 1, wherein the size of stored chunks is configurable to tune performance of restore operations.

11. The method of claim 1, further comprising:

determining that one or more stored chunks are being moved to a new physical location during a storage operation; and

,

for each of the one or more moved chunks, associating the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.

24. The method of claim 1, wherein storing the modified version of the NDMP backup data stream includes associating a magnetic tape location and a magnetic tape length with the physical offset and the physical length.

12. A system for handling a Network Data Management Protocol (“

NDMP”

) backup or copy operation, the system comprising;

a processor; and

,

a secondary storage computing device, coupled to the processor, and configured to;

receive an original NDMP backup data stream from a client;

modify the received original NDMP backup data stream by performing at least two the following modification procedures upon the original NDMP backup data stream;

deduplication, encryption, and compression,

wherein the at least two modification procedures create a modified version of the NDMP backup data stream;

store the modified version of the NDMP backup data stream in stored chunks,

wherein each chunk is addressable by a physical offset and a physical length,

wherein each chunk represents a modified version of a subset of the original NDMP backup data stream, and,

wherein the client associates a logical offset and a logical length with the subset of the original NDMP backup data stream that is represented by each chunk; and

,

for each stored chunk, associate the stored chunk with;

the physical offset and physical length that permits addressing of the chunk, and,

the logical offset and logical length that the client associates with the subset of the original NDMP backup data stream that is represented by the chunk;

receive a request for a certain subset of the original NDMP backup data stream,

wherein the request includes a logical offset and a logical length;

use the received logical offset and logical length to;

identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and

determine a physical offset and a physical length that permits addressing of the identified chunk;

read at least a portion of the identified chunk using the determined physical offset and physical length;

convert the portion of the identified chunk by reversing each of the modification procedures that were previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and

,

sending data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.

13. The system of claim 12, wherein the secondary storage computing device is further configured to

read another portion of the identified chunk;

convert the other portion of the identified chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the other portion of the identified chunk;

buffer data from the converted version of the other portion of the identified chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and

,

respond to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

14. The system of claim 12, wherein the secondary storage computing device is further configured to:

read at least a portion of another chunk that represents a modified version of a subset of the original NDMP backup data stream that is adjacent to at least a portion of the requested subset of the original NDMP backup data stream;

convert the portion of the other chunk by reversing each of the modification procedures that was previously performed upon the original NDMP backup data stream to create a converted version of the portion of the other chunk;

buffer data from the converted version of the portion of the other chunk, wherein the buffered data does not form a portion of the requested subset of the original NDMP backup data stream; and

,

respond to another request for a different subset of the original NDMP backup data stream with at least some of the buffered data.

15. The system of claim 12,

wherein the secondary storage computing device modifies the received original NDMP backup data stream by performing deduplication;

and

wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.

16. The system of claim 12, wherein a data storage policy specifies an approximate size of the stored chunks.

17. The system of claim 12, wherein the secondary storage computing device is further configured to:

determine that one or more stored chunks are being moved to a new physical location during a storage operation; and

,

for each of the one or more moved chunks, associate the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.

18. At least one non-transitory computer-readable medium storing instructions, which when executed by at least one data processor, performs a Network Data Management Protocol (“

NDMP”

) backup or copy operation, comprising;

receiving an original NDMP backup data stream from a client;

modifying the received original NDMP backup data stream by performing at least two of the following modification procedures upon the original NDMP backup data stream;

deduplication, encryption, and compression,

wherein the at least two modification procedures create a modified version of the NDMP backup data stream;

storing the modified version of the NDMP backup data stream in stored chunks,

wherein each chunk is addressable by a physical offset and a physical length,

wherein each chunk represents a modified version of a contiguous subset of the original NDMP backup data stream, and,

wherein the client associates a logical offset and a logical length with the contiguous subset of the original NDMP backup data stream that is represented by each chunk; and

,

for each stored chunk, associating the stored chunk with;

the physical offset and physical length that permits addressing of the chunk, and,

the logical offset and logical length that the client associates with the contiguous subset of the original NDMP backup data stream that is represented by the chunk;

using the received logical length to;

identify a stored chunk that comprises a modified version of at least a portion of the requested subset of the original NDMP backup data stream, and

determine a physical offset and a physical length that permits addressing of the identified chunk;

read at least a portion of the identified chunk using the determined physical offset and physical length;

converting the portion of the identified chunk by reversing the modification procedures that were previously performed upon the original NDMP backup data stream in order to create a converted version of the portion of the identified chunk; and

,

sending data from the converted version of the portion of the identified chunk that is at least a portion of the requested subset of the original NDMP backup data stream.

19. The non-transitory computer-readable medium of claim 18, wherein associating a stored chunk with a physical offset, physical length, logical offset, and logical length comprises associating a chunk identifier with the physical offset, physical length, logical offset, and logical length in a chunk mapping table.

20. The non-transitory computer-readable medium of claim 18,

wherein modifying the received original NDMP backup data stream comprises performing deduplication;

and

wherein at least one chunk corresponds to a deduplication data structure that is created by the performance of deduplication.

21. The non-transitory computer-readable medium of claim 18, further comprising:

determining that one or more stored chunks are being moved to a new physical location during a storage operation; and

,

for each of the one or more moved chunks, associating the chunk and its associated logical offset and logical length with a new physical offset and new physical length that permits addressing of the chunk at its new physical location.

22. The non-transitory computer-readable medium of claim 18, wherein modifying the received original NDMP backup data stream further comprises adding metadata to the data stream, and wherein a storage policy specifies a configuration of the backup or copy operation.

23. The non-transitory computer-readable medium of claim 18, further comprising:

receiving a request from a client to revert data to an earlier state at a previous time using previously obtained point-in-time data that is stored in a chuck as a modified version of NDMP data;

receiving instructions that the client has requested that a snapshot of the current state of the data for client be taken; and

sending the user the reverted data.