Ensuring data persistence and consistency in enterprise storage backup systems

• US 8,121,977 B2
• Filed: 11/06/2009
• Issued: 02/21/2012
• Est. Priority Date: 08/30/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Iwmage Systems Incorporated
• Inventors: Atluri, Rajeev, Kumar, Srin, Kumar, Satish, Inala, Suman, Zavertnik, Greg, Gottemukkula, Yeganjaiah, Paygude, Kishor
• Primary Examiner(s): EHICHIOYA, IRETE FRED
• Application Number:
  US12/613,539
• Publication Number:
  US 20100280999A1
• Time in Patent Office: 837 Days
• Field of Search: 707/1, 707/100, 707/200, 707/609, 707/687, 707/705, 707/802, 709/217, 709/219, 709/224, 711/161, 711/162
• US Class Current: 707/609
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064

Abstract

Ensuring data persistence and consistency in enterprise storage backup systems method and apparatus are disclosed. In one embodiment, a method includes creating a data log structure (e.g., a log file) on a storage device (e.g., a Serial ATA drive, a SCSI drive, a SAS drive, a storage are network, etc.) coupled to an application server to store a backup data generated by a filter module in a continuous backup environment and buffering the backup data generated by the filter module through a memory module that transfers the backup data to the data log structure on the storage device based on an occurrence of an event. The data log structure may be created by configuring a portion of the storage device as a dedicated resource available to the memory module to copy the backup data and the data log structure may be a file on the storage device coupled to the application server having the file system.

Claims

1. A method, comprising:

implementing a filter module in an operating system environment associated with an application server to track changes on a storage device coupled to the application server;

generating, in a continuous backup environment, a backup data through the filter module associated with the application server by copying a data write associated with the application server;

utilizing a data log structure on the storage device coupled to the application server to store the backup data generated by the filter module;

buffering the backup data generated by the filter module through a memory module associated with the application server that transfers the backup data to the data log structure on the storage device based on an occurrence of an event;

differentiating the backup data generated by the filter module from the data write associated with the application server using a detection algorithm of the filter module when the memory module transfers the backup data to the data log structure; and

allowing the backup data to permeate through the filter module to prevent a recursion loop that arises when the filter module makes a redundant copy of the backup data during a transition of the backup data from the memory module to the data log structure on the storage device.

2. The method of claim 1, further comprising:

utilizing a paged memory of the memory module to buffer the backup data when a storage capacity of the memory module crosses a memory threshold in the continuous backup environment;

distinguishing a write associated with the paged memory from at least one of the backup data and the data write of the application server at the filter module using the detection algorithm when the paged memory accesses at least one of a page file and a swap volume on the storage device to buffer the backup data; and

allowing the write associated with the paged memory to pass through the filter module to prevent at least one of a system crash and the recursion loop that occurs when the filter module makes the redundant copy of the write during the transition of the write associated to the paged memory to the at least one of the page file and the swap volume on the storage device.

3. The method of claim 2, further comprising:

utilizing the data log structure on the storage device coupled to the application server to buffer the backup data when a capacity of the paged memory reaches a threshold value;

cycling the backup data through a plurality of data log structures on the storage device when a particular data log structure reaches a threshold capacity; and

inserting a tracking data in the memory module to mark the shift from the particular data log structure to another data log structure of the plurality of data log structures to keep track of the backup data in the continuous backup environment when the particular data log structure reaches the threshold capacity.

4. The method of claim 1,

wherein the storage device is at least one of a protected storage device and a non-protected storage device having a file system associated therewith, and

wherein the event is at least one of a planned shutdown event, an unplanned shutdown event and a user requested event.

5. The method of claim 4, wherein further comprising creating the data log structure by configuring a portion of the storage device as a dedicated resource available to the memory module to copy the backup data, wherein the data log structure is a file on the storage device coupled to the application server having the file system.

6. The method of claim 1, further comprising depositing, through the filter module, a collection of data writes of the application server that remains to be recorded in a meta-data registry of the memory module to the data log structure on the storage device coupled to the application server to prevent a data loss during the unplanned shutdown event.

7. The method of claim 1, wherein the filter module is applied at least one of a logical volume level and a physical volume level in an operating system storage stack of the application server.

8. The method of claim 1, further comprising:

generating a consistency request through a consistency module associated with the filter module in the continuous backup environment;

communicating the consistency request to a backup application programming interface of the application server to prepare the application server for a dynamic backup operation;

creating a marker data to connote a consistency point of the application server; and

introducing the marker data through the filter module into the memory module in the continuous backup environment such that the backup data can be restored to the consistency point using the marker data.

9. The method of claim 8, wherein the consistency request is triggered by a consistency event, and wherein the consistency event is at least one of a scheduled event, an unscheduled event and a user initiated event.

10. The method of claim 8, further comprising:

quiescing multiple storage devices coupled to the application server based on the consistency event; and

applying the marker data in a synchronous manner across the multiple storage devices to denote a particular consistency point.

11. The method of claim 8, further comprising applying the marker data across multiple application servers through a network based on the consistency event.

12. The method of claim 1, further comprising:

analyzing the data write of the application server through an algorithm of the filter module to determine a descriptor information associated with the data write of the application server; and

automatically introducing the marker data to indicate an event data based on the descriptor information.

13. The method of claim 12, wherein the descriptor information is at least one of an application file information, an application status information and an application data change information.

14. The method of claim 1, further comprising:

extracting a meta-data information associated with the data write of the application server to ascertain a location of the data write when the data write of the application server is received by the filter module;

comparing the meta-data information of a particular data write to the meta-data information of a plurality of data writes to detect an overlapping pattern of the particular data write with other data write in the storage device; and

performing a memory optimization when the storage capacity of the memory module crosses the memory threshold by retaining the particular data write when the particular data write exhibits the overlapping pattern.

15. The method of claim 14, further comprising:

enhancing the memory optimization by maintaining only a journal of the meta-data information associated with the data write on the memory module; and

applying the journal of the meta-data information to the storage device to locate a data corresponding to the data write of the application server.

16. The method of claim 15, wherein the meta-data information associated with the data write is at least one of a volume information, an offset information, a length information and a data size information.

17. The method of claim 14, further comprising:

retaining, through the memory module, the particular data write of the application server when the particular data write displays the overlapping pattern with the other data write in the storage device coupled to the application server; and

preserving, through the memory module, a registry of the meta data information associated with the other data write that can be utilized to selectively copy and create the backup data from the other data write in the continuous backup environment.

18. The method of claim 1, in a form of a machine-readable medium embodying a set of instructions that, when executed by a machine, causes the machine to perform the method of claim 1.

19. A network comprising:

an application server having a filter module implemented in an operating system environment associated therewith to track changes on a storage device coupled thereto, the filter module being coupled to a memory module to facilitate a continuous data backup of the application server and to prevent a data loss triggered by an event during the continuous data backup by facilitating transfer of a backup data buffered in the memory module to a data log structure resident on the storage device, the backup data being generated by the filter module by copying a data write associated with the application server, the backup data being differentiated from the data write of the application server using a detection algorithm of the filter module when the memory module transfers the data backup to the data log structure, and the backup data being allowed to permeate through the filter module to prevent a recursion loop that arises when the filter module makes a redundant copy of the backup data during a transition of the backup data from the memory module to the data log structure on the storage device; and

a backup server connected to the application server via the network to store the backup data on a backup storage device coupled to the backup server on a continuous basis.

20. The network of claim 19, further comprising a consistency module of the application server to place a flag in the memory module to indicate a data consistency point of an application during a dynamic backup of the application server in a continuous backup environment.

21. The network of claim 19, wherein the event is at least one of a planned shutdown event, an unplanned shut down event and a user requested event, and wherein the filter module coupled to the memory module dumps a backlog of data write of the application server to the data log structure resident on the storage device to prevent the data loss triggered by the unplanned shut down event.

22. An application server in a backup environment comprising:

a memory buffer having a paged memory capable of utilizing at least one of a page file and a swap volume on the application server to enhance a storage capacity of the memory buffer when the storage capacity crosses a threshold value;

a disk buffer on the application server to backup the memory buffer when the memory buffer faces an imminent data loss associated with an event; and

a filter module to extract a descriptor information from a data write of the application server to position a pointer data in the memory buffer based on an analysis of the descriptor information such that the pointer data corresponds to an event data of the application server, to extract a meta data information associated with the data write of the application server to ascertain a location of the data write when the data write of the application server is received thereat, to compare the meta-data information of a particular data write to the meta-data information of a plurality of data writes to detect an overlapping pattern of the particular data write with another data write in a storage device associated with the application server, and to enable performing a memory optimization when the storage capacity of the memory module crosses the memory threshold by retaining the particular data write when the particular data write exhibits the overlapping pattern.

23. The application server of claim 22, further comprising a consistency module to facilitate interaction between the filter module and a particular application of the application server to coordinate placement of the pointer data in a synchronous fashion across multiple storage devices of the application server to denote a common consistency marker across the multiple storage devices for the particular application.

24. The application server of claim 22, wherein a detection algorithm of the filter module differentiates the data write of the application server from at least one of a write of the paged memory and a backup data of the memory buffer, and wherein the filter module selectively copies only the data write of the application server to avoid a recursion phenomena caused due to a redundant copy generated from either of the write of the paged memory and the backup data of the memory buffer.