Recovering a file system to any point-in-time in the past with guaranteed structure, content consistency and integrity

• US 8,972,347 B1
• Filed: 12/14/2012
• Issued: 03/03/2015
• Est. Priority Date: 03/30/2007
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Dell Software, Inc. (Dell Technologies Inc.)
• Inventors: Sim-Tang, Siew Yong
• Primary Examiner(s): VO, TRUONG V
• Application Number:
  US13/714,829
• Time in Patent Office: 809 Days
• Field of Search: 707/634
• US Class Current: 707/634
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files

Abstract

A data management system (DMS) includes a continuous real-time object store that captures all real-time activities, with associated object metadata information. The DMS is capable of reintroducing any point-in-time view of data ranging from a granular object to an entire file system. A set of algorithms (for creation of a file or directory, modification of a file or directory, deletion of a file or directory, and relocation/renaming of a file or directory) are used to generate and maintain a file system history in the DMS and to ensure that a latest version of a directory always refers to a latest version of its children until the directory changed. Any point-in-time recovery is implemented using the file system history in one of various ways to provide strong individual file integrity, exact point-in-time crash consistency, and/or recovery of last version of all files in the file system.

Claims

1. A machine-implemented data management method, comprising:

capturing, in real time, application data transactions of a data source;

deciding whether to accumulate the captured application data transactions;

receiving, from the data source, an application-aware data stream formed from the captured application data transactions;

wherein the received application-aware data stream represents a continuous real-time history of the data source;

storing in a data store the continuous real-time history of the data source, or a component thereof, as an object-oriented logical representation;

dynamically restructuring the object-oriented logical representation in response to one or more events occurring in the continuous real-time history;

wherein, for a relocation event, the dynamically restructuring comprises relocating based on object instance where object versions connect across at least a portion of a lifetime of an object wherever the object is relocated; and

using the object-oriented logical representation to provide an any point-in-time recovery of the data source.

2. The method of claim 1, wherein the received application-aware data stream comprises:

events;

metadata; and

data changes.

3. The method of claim 1, wherein the any point-in-time recovery of the data source has a recovery consistency characteristic.

4. The method of claim 1 wherein the object-oriented logical representation comprises at least a set of version metadata pages, and a set of one or more links that associate the version metadata pages of the set of version metadata pages.

5. The method of claim 4 wherein the object-oriented logical representation further comprises the entire binary content of a file associated with the data source.

6. The method of claim 4 wherein the object-oriented logical representation provides an n-dimensional logical representation of the data source at a given point in time.

7. The method of claim 1 wherein the data source is physically remote from the data store.

8. The method of claim 7 wherein the continuous real-time history of the data source provides one of:

a data protection service and a data distribution service.

9. The method of claim 1 wherein the any point-in-time recovery of the data source includes recovery of one of:

a file, a sub-directory, a file system volume, and a set of files that belong to a database.

10. The method of claim 9 wherein the set of files that belong to the database span one or more volumes.

11. The method of claim 1 further including extracting an audit trail from the data source identifying, for a given object, a change and an identifier associated with a user who initiated the change.

12. The method of claim 1 wherein capturing application data transactions occurs on the data source.

13. The method of claim 1 wherein capturing application data transactions occurs on a network data-source data path.

14. The method of claim 1 wherein exact point-in-time crash consistency constructs an image of the data source at a recovery point by obtaining a latest version and applying changes after the latest version and up to the recovery point.

15. The method of claim 1 wherein last version of all files in the data source recovers all files up to a latest version regardless of whether a state of a given file is consistent or suspect.

16. The method of claim 1, wherein the recovery consistency characteristic is selected from the group consisting of:

strong individual file integrity, exact point-in-time crash consistency, and last version of all files.

17. The method of claim 1, wherein, for the relocation event, the dynamically restructuring comprises relocating based on object path where object versions connect when an object associated with a directory remains in the directory when relocated.

18. The method of claim 1, wherein, for the relocation event, the dynamically restructuring comprises relocating based on a combination of object path and object instance.

19. A system comprising at least one processor, wherein the at least one processor is operable to implement a method comprising:

capturing, in real time, application data transactions of a data source;

deciding whether to accumulate the captured application data transactions;

receiving, from the data source, an application-aware data stream formed from the captured application data transactions;

wherein the received application-aware data stream represents a continuous real-time history of the data source;

storing in a data store the continuous real-time history of the data source, or a component thereof, as an object-oriented logical representation;

dynamically restructuring the object-oriented logical representation in response to one or more events occurring in the continuous real-time history;

wherein, for a relocation event, the dynamically restructuring comprises relocating based on object instance where object versions connect across at least a portion of a lifetime of an object wherever the object is relocated; and

using the object-oriented logical representation to provide an any point-in-time recovery of the data source.

20. A computer-program product comprising a non-transitory computer-usable medium having computer-readable program code embodied therein, the computer-readable program code adapted to be executed to implement a method comprising:

capturing, in real time, application data transactions of a data source;

deciding whether to accumulate the captured application data transactions;

receiving, from the data source, an application-aware data stream formed from the captured application data transactions;

wherein the received application-aware data stream represents a continuous real-time history of the data source;

storing in a data store the continuous real-time history of the data source, or a component thereof, as an object-oriented logical representation;

dynamically restructuring the object-oriented logical representation in response to one or more events occurring in the continuous real-time history;

wherein, for a relocation event, the dynamically restructuring comprises relocating based on object instance where object versions connect across at least a portion of a lifetime of an object wherever the object is relocated; and

using the object-oriented logical representation to provide an any point-in-time recovery of the data source.