PARALLELIZED BACKUP AND RESTORE PROCESS AND SYSTEM

• US 20110082832A1
• Filed: 10/05/2009
• Published: 04/07/2011
• Est. Priority Date: 10/05/2009
• Status: Abandoned Application

• Current Assignee: Teradata US, Inc. (Teradata Corporation)
• Original Assignee: Teradata US, Inc. (Teradata Corporation)
• Inventors: VADALI, RAMKUMAR, Chun, Brent
• Application Number:
  US12/573,164
• Publication Number:
  US 20110082832A1
• Time in Patent Office: Days
• US Class Current: 707/615
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/835 Timestamp

Abstract

A system and methods for parallelized backup and restore process and system are disclosed. In one embodiment, a method includes providing a massively parallelized analytic database, serializing a schedule of a transaction history of the massively parallelized analytic database, and creating a transactionally consistent copy of the massively parallelized analytic database. The method may include restoring one or more of an original system and a configurationally equivalent system to a transaction consistent state as of a time the transactionally consistent copy was created. The transactionally consistent copy may be stored on a separate system than the original system. Accessibility to the transactionally consistent copy may be retained on the separate system even when the original system is inaccessible by the storing of the transactionally consistent copy on the separate system.

Claims

1. A method, comprising:

providing a massively parallelized analytic database;

serializing a schedule of a transaction history of the massively parallelized analytic database; and

creating a transactionally consistent copy of the massively parallelized analytic database.

2. The method of claim 1 further comprising:

restoring at least one of an original system and a configurationally equivalent system to a transaction consistent state as of a time the transactionally consistent copy was created.

3. The method of claim 2 further comprising:

storing the transactionally consistent copy on a separate system than the original system; and

retaining accessibility to the transactionally consistent copy on the separate system even when the original system is inaccessible by the storing of the transactionally consistent copy on the separate system.

4. The method of claim 2 further comprising parallelizing the creation of the transactionally consistent copy and the restoration of the transactionally consistent state to ensure that performance of backup and restore processes of the massively parallelized analytic database are scalable.

5. The method of claim 2 further comprising applying an auto-sensing mechanism to determine a most-efficient method to allocate resources when performing at least one of a compression operation and a decompression operation.

6. The method of claim 2 further comprising streamlining the restoration of the transactionally consistent state through a direct application of a latest incremental backup.

7. The method of claim 2 wherein the restoration is efficient because a copy of files is a primary means of restoring data rather than a transaction log entry roll forward.

8. The method of claim 2 wherein the creation of the transactionally consistent copy and the restoration of the transactionally consistent state are fail-safe in that they are pausable and resumable during backup and restoration processes.

9. The method of claim 2 further comprising:

using a time stamp to determine which files have not changed, wherein the time stamp is pegged to a past point in time such that the time stamp is controllable and emulates a version number; and

using hard links to point the files that have not changed since last backup so that the restore process begins with restoring the latest backup,

wherein the time stamp is applied on a destination server to only successfully transferred files, and

wherein the time stamp is applied on a source server only when a file has been recently changed.

10. The method of claim 8 wherein the backup and restoration processes are external to a PostGres instance in that an interaction between the backup and restore processes and PostGres instances are through a published PostGres backup/recovery interface.

11. The method of claim 1 further comprising employing a file level copy during the creation of the transactionally consistent copy.

12. The method of claim 10 further comprising wherein a file system monitoring method is used during the creation of the transactionally consistent copy to determine whether files have changed so that a minimum set of files are copied during the backup process.

22. The system of claim 12 wherein during a restore process of the production cluster, a massively parallel analytic database that is configurationally equivalent to an original system is made available to restore at least one of a full backup and an incremental backup of the original system.

24. The system of claim 22 wherein files in a backup file set are copied in parallel to appropriate virtual worker nodes during the restore process, and wherein an auto-sensing mechanism is used to perform a file decompression using a most efficient resource allocation method.

26. The system of claim 24 wherein the massively parallel analytic database is brought up and PostGres instances on the virtual worker nodes of the production cluster goes through transaction recovery until the massively parallel analytic database is fully restored.

13. A system comprising:

a production cluster to process queries and a Data Manipulation Language (DML); and

a backup cluster with a number of physical worker nodes on a different rack than the production cluster to backup and restore multiple production clusters thereby creating a centralized system to manage backups from all production systems.

14. The system of claim 13 wherein the backup cluster comprises at least five physical worker nodes, and wherein at least one of the physical worker nodes is a manager node.

15. The system of claim 13 wherein the production cluster comprises eighty virtual worker nodes which are each a PostGres instance, and wherein at least one of the virtual worker nodes is a queen node.

16. The system of claim 14 wherein a backup process in the production cluster begins with a control phase in which the manager node and the queen node communicate with each other to assign the virtual worker nodes to the physical worker nodes in a round-robbin manner, and wherein assigned ones of the virtual worker nodes subsequently communicate with their assigned physical worker nodes directly during at least one of a file transfer and a log transfer.

17. The system of claim 15 wherein the backup process determines which files are to be copied through a comparison of time stamps of a file system, and wherein file changes are monitored to streamline incremental backups by registering with the file system.

18. The system of claim 16 wherein the file transfer and the log transfer are copied in parallel from the production cluster to the backup cluster, and a compression auto-sensing technique is used to make best use of network and processor resource trade-off.

19. The system of claim 17 wherein a quiescent mode is entered by the production cluster after a best-effort attempt to copy all changed files, and wherein transaction commits are blocked in the quiescent mode.

20. The system of claim 18 wherein when PostGres instances are placed into a hot backup mode, a set of files that changed between when the backup process began and a time immediately after the quiescent mode is determined and copied in parallel.

21. The system of claim 19 wherein transaction logs are copied and the production cluster is taken out of the quiescent mode.

23. The system of claim 21 wherein a manager node of the backup cluster and a queen node of the production cluster to communicate with each other to establish a correspondence between virtual worker nodes of the production cluster and physical worker nodes of the backup cluster.

25. The system of claim 23 wherein logs in a backup log set are copied in parallel to appropriate virtual nodes of the production cluster and to the queen node of the production cluster.

27. A machine-readable medium providing instructions, which when read by a processor, cause the machine to perform operations, comprising:

providing a massively parallelized analytic database;

serializing a schedule of a transaction history of the massively parallelized analytic database;

creating a transactionally consistent copy of the massively parallelized analytic database; and

restoring at least one of an original system and a configurationally equivalent system to a transaction consistent state as of a time the transactionally consistent copy was created.

28. The machine-readable medium of claim 27 further comprising:

storing the transactionally consistent copy on a separate system than the original system; and

retaining accessibility to the transactionally consistent copy on the separate system even when the original system is inaccessible by the storing of the transactionally consistent copy on the separate system.

29. The machine-readable medium of claim 27 further comprising:

parallelizing the creation of the transactionally consistent copy and the restoration of the transactionally consistent state to ensure that performance of backup and restore processes of the massively parallelized analytic database are scalable.

30. The machine-readable medium of claim 27 further comprising:

applying an auto-sensing mechanism to determine a most-efficient method to allocate resources when performing at least one of a compression operation and a decompression operation.

31. The machine-readable medium of claim 27 further comprising:

streamlining the restoration of the transactionally consistent state through a direct application of a latest incremental backup.