Method and system for real-time event journaling to provide enterprise data services

• US 20050251540A1
• Filed: 05/10/2004
• Published: 11/10/2005
• Est. Priority Date: 05/10/2004
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: BakBone Software, Inc. (Quest Software, Inc.)
• Inventors: Sim-Tang, Siew Yong
• Granted Patent: US 7,565,661 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1446 Point-in-time backing up or restoration of persistent data
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A data management system or “DMS” provides a wide range of data services to data sources associated with a set of application host servers. The data management system typically comprises one or more regions, with each region having one or more clusters. A given cluster has one or more nodes that share storage. To facilitate the data service, a host driver embedded in an application server connects an application and its data to a cluster. The host driver provides a method and apparatus for capturing real-time data modifications and application state notifications and, in response, generating data transactions in the form of an event journal that is provided to the data management system. The driver functions to translate traditional file/database/block I/O into a continuous, application-aware, output data stream. Using the streams generated in this manner, the DMS offers a wide range of data services that include, by way of example only: data protection (and recovery), and disaster recovery (data distribution and data replication).

Claims

1. Apparatus for use in providing a given data service, comprising:

a filter, located between an application and its primary data storage;

a module, connectable to the filter, that identifies to the filter a set of one or more events expected to occur as the application interfaces to its primary data storage, and a given action to take upon occurrence of each event;

a queue connectable to the filter to receive and store data generated as a function of the given action; and

a processor connectable to the queue to receive and process the data into a continuous, application-aware, data stream, wherein the data stream is useful in providing the given data service.

2. The apparatus as described in claim 1 further including an application module, connectable to the application, and that identifies an application level event, and a given action to take upon occurrence of the application level event.

3. The apparatus as described in claim 1 wherein the processor is further operative to generate a given event and to store the given event in the queue for subsequent processing.

4. The apparatus as described in claim 3 wherein the given event is an internal event that must be processed into an outbound event before being included in the data stream.

5. The apparatus as described in claim 1 wherein the events include at least one file system event.

6. The apparatus as described in claim 1 wherein the events include at least one block device I/O event.

7. The apparatus as described in claim 1 wherein the given action retrieves given data from the primary storage and forwards the given data to the queue.

8. The apparatus as described in claim 1 wherein the continuous, application-aware, data stream comprises one or more of the following data types:

file or block data, byte-level deltas, transaction data, a file checkpoint, a transaction checkpoint, a database checkpoint, an application checkpoint, and metadata.

20. The method as described in claim 3 wherein the given event is a failure event that must be processed into an outbound event-before being included in the data stream.

9. Apparatus for use in providing a given data service, comprising:

a filter, located between an application and its primary data storage;

an input/output module, connectable to the filter, that identifies to the filter a set of one or more events expected to occur as the application interfaces to its primary data storage, and a given action to take upon occurrence of each event;

a database module, connectable to a database associated with an application that stores data in the database;

a queue connectable to the filter and the database module to receive and store data generated by the database module as a function of the given action; and

a processor connectable to the queue to receive and process the data into a continuous, application-aware, data stream, wherein the data stream is useful in providing the given data service.

10. The apparatus as described in claim 9 wherein the processor is further operative to generate a given event and to store the given event in the queue for subsequent processing.

11. The apparatus as described in claim 10 wherein the given event is an internal event that must be processed into an outbound event before being included in the data stream.

12. The apparatus as described in claim 9 wherein the events include at least one database event.

13. The apparatus as described in claim 9 wherein the continuous, application-aware, data stream comprises one or more of the following data types:

file or block data, byte-level deltas, transaction data, a file checkpoint, a transaction checkpoint, a database checkpoint, an application checkpoint, and metadata.

14. A method of monitoring an application connectable to primary data storage to facilitate provision of a given data service, comprising:

identifying a set of one or more events expected to occur as the application interfaces to its primary data storage, and a given action to take upon occurrence of each event;

as the application interfaces to its primary data storage, receiving and storing data generated as a function of a given action; and

processing the data into a continuous, application-aware, data stream.

15. A system for providing a given data service to a data source, comprising:

a host having a processor; and

software executable on the processor to generate a continuous, application-aware, data stream;

wherein the data stream is useful in providing the given data service.

16. A method of providing a data service, comprising:

for a given application connectable to a data store, monitoring events as the application interfaces to its data store and, in response thereto, generating a continuous, application-aware, data stream; and

using the continuous application-aware data stream to provide a given data service.

17. The method as described in claim 16 wherein the given data service is data protection.

18. The method as described in claim 16 wherein the given data service is data replication.

19. The method as described in claim 16 wherein the given data service is data distribution.