SYSTEMS AND METHODS FOR PERFORMING DATA REPLICATION

• US 20110295805A1
• Filed: 05/27/2011
• Published: 12/01/2011
• Est. Priority Date: 05/28/2010
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Erofeev, Andrei
• Granted Patent: US 8,489,656 B2
• Time in Patent Office: Days
• US Class Current: 707/634
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

Preparing source data to be replicated in a continuous data replication environment. Certain systems and methods populate a file name database with entries having a unique file identifier descriptor (FID), short name and a FID of the parent directory of each directory or file on a source storage device. Such information is advantageously gathered during scanning of a live file system without requiring a snapshot of the source storage device. The database can be further used to generate absolute file names associated with data operations to be replayed on a destination storage device. Based on the obtained FIDs, certain embodiments can further combine write operations to be replayed on the destination storage device and/or avoid replicating temporary files to the destination system.

Claims

1. A method for identifying data to be copied in a data replication system, the method comprising:

obtaining with a scanning module executing on a computing device a first file identifier descriptor (FID) of a first directory on a live source file system, the first FID being one of a plurality of unique identifiers corresponding to a plurality of directories and files on the source file system;

adding the first FID to a queue;

storing a current journal sequence number from a file system filter driver identifying a first time;

following said storing, accessing a current directory of the plurality of directories on the source file system that corresponds to a next FID stored in the queue;

obtaining additional FIDs for each immediate child directory and immediate child file in the current directory;

if no changes have been made to the current directory since the first time,

populating a file name database with the additional FIDs of each immediate child directory and immediate child file in the current directory,

adding the additional FIDs of each immediate child directory of the current directory to the queue, and

removing the next FID from the queue; and

if changes have been made to the first directory since the first time, repeating said storing, said accessing and said obtaining the additional FIDs.

2. The method of claim 1, wherein said obtaining the first FID is performed without performing a snapshot on the source file system.

3. The method of claim 1, wherein the first directory is the current directory.

4. The method of claim 1, additionally comprising repeating said storing, said accessing and said obtaining the additional FIDs for each FID stored in the queue.

5. The method of claim 1, wherein said populating the file name database comprises for each immediate child directory and immediate child file in the current directory and storing in the file name database:

the additional FID for the immediate child directory or immediate child file;

a corresponding short name for the immediate child directory or immediate child file; and

the next FID as a parent directory of the immediate child directory or immediate child file.

6. The method of claim 1, wherein said changes comprise namespace changes to the current directory.

7. The method of claim 1, wherein the first directory comprises a root directory of the live source file system.

8. The method of claim 1, additionally comprising monitoring at least one data management operation directed to first data stored in the source file system.

9. The method of claim 8, additionally comprising replaying the at least one data management operation on replication data stored on a destination file system.

10. The method of claim 9, additionally comprising:

constructing, from information populated in the file name database, an absolute file name that corresponds to the location of the first data on the source file system; and

transmitting the absolute file name to the destination system to direct said replaying of the at least one data management operation.

11. A system for preparing data for replication from a source computing device in a network, the system comprising:

a queue configured to store a plurality of file identifier descriptors (FIDs) each comprising a unique identifier that corresponds to one of a plurality of directories and files on a source file system;

a scanning module executing on a computing device and configured to scan the source file system while in a live state and to populate the queue with the plurality of FIDs;

a database comprising file name data that associates each of the plurality of FIDs with a short name and a parent FID, wherein the scanning module is further configured to populate the database with the file name data based on said scan of the source file system in the live state; and

at least one database thread configured to receive a data entry identifying a data management operation associated with at least one of the plurality of directories and files on the source file system and to construct from the FID associated with the at least one directory or file an absolute file name for transmission to a destination system along with a copy of the data management operation for replying on the destination system.

12. The system of claim 11, wherein the scanning module is further configured to:

access a current directory of the plurality of directories on the source file system that corresponds to a next FID in the queue; and

obtain additional FIDs for each immediate child directory and immediate child file in the current directory.

13. The system of claim 12, wherein the scanning module is further configured to:

populate the file name database with the additional FIDs of each immediate child directory and immediate child file in the current directory, and add the additional FIDs of each immediate child directory of the current directory to the queue.

14. The system of claim 11, further comprising a filter driver situated between the source file system and at least one application configured to request the data management operation.

15. The system of claim 14, wherein the filter driver is further configured to assign journal sequence numbers to each journal entry associated with a requested change to the source file system.

16. The system of claim 15, wherein the scanning module is further configured to receive a current journal sequence number from the filter driver prior to accessing the current directory.

17. The system of claim 16, wherein the scanning module is configured to repeat said accessing and obtaining when changes are detected to the current directory following a time of the current journal sequence number but prior to said obtaining.