Manifest-based snapshots in distributed computing environments

• US 9,690,671 B2
• Filed: 10/29/2014
• Issued: 06/27/2017
• Est. Priority Date: 11/01/2013
• Status: Active Grant

• Current Assignee: Cloudera Incorporated
• Original Assignee: Cloudera Incorporated
• Inventors: Hsieh, Jonathan Ming-Cyn, Bertozzi, Matteo
• Primary Examiner(s): HO, BINH VAN
• Application Number:
  US14/527,563
• Publication Number:
  US 20150127608A1
• Time in Patent Office: 972 Days
• Field of Search: 707623
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Scalable architectures, systems, and services are provided herein for creating manifest-based snapshots in distributed computing environments. In some embodiments, responsive to receiving a request to create a snapshot of a data object, a master node identifies multiple slave nodes on which a data object is stored in the cloud-computing platform and creates a snapshot manifest representing the snapshot of the data object. The snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system. The snapshot can be created without disrupting I/O operations, e.g., in an online mode by various region servers as directed by the master node. Additionally, a log roll approach to creating the snapshot is also disclosed in which log files are marked. The replaying of log entries can reduce the probability of causal consistency in the snapshot.

Claims

1. A computer implemented method of creating a manifest-based snapshot of a data object in a distributed cloud-computing platform, the method comprising:

responsive to receiving a request to create a snapshot of the data object,

identifying, by a master node of a distributed database system, multiple slave nodes of the distributed database system on which a data object is stored in the cloud-computing platform,

wherein the distributed database system includes the master node and the multiple slave nodes,

wherein each slave node implements a region server and includes a data node associated with the region server;

creating a snapshot manifest representing the snapshot of the data object,

wherein the snapshot manifest comprises a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system,

wherein creating the snapshot manifest further comprises directing each region server to create a portion of the snapshot manifest corresponding to the partition of the data on the data node with which the region server is associated,

wherein directing each region server to create the portion of the snapshot manifest corresponding to the partition of the data on the data node with which the region server is associated comprises;

generating, by the master node, a request to each region server for the respective portions of the snapshot manifest corresponding to the partition of the data on the data node with which the region servers are associate; and

sending, by the master node, the requests to the corresponding region servers.

2. The computer implemented method of claim 1, further comprising:

receiving the request to create a snapshot of the data object.

3. The computer implemented method of claim 1, further comprising:

verifying, by the master node of a distributed database system, that the snapshot manifest is successfully or unsuccessfully created.

4. The computer implemented method of claim 1, wherein the data object is stored in partitions on multiple of the data nodes.

5. The computer implemented method of claim 4, wherein directing each region server to create the portion of the snapshot manifest corresponding to the partition of the data on the data node further comprises requesting that the region server flush memory prior to creation of the portion of the snapshot manifest.

6. The computer implemented method of claim 4, wherein directing each region server to create the portion of the snapshot manifest corresponding to the partition of the data on the data node further comprises directing each region server to drop a marker in a log the indicating a position of the log file associated with the snapshot manifest.

7. The computer implemented method of claim 4, wherein each portion of the snapshot manifest comprises a single file including a listing of multiple file names in the portion of the snapshot manifest and reference information for locating the multiple files on corresponding data nodes.

8. The computer implemented method of claim 4, wherein each portion of the snapshot manifest represents a single computing region.

9. The computer implemented method of claim 1, further comprising:

receiving the respective portions of the snapshot manifest.

10. The computer implemented method of claim 9, wherein creating the snapshot manifest further comprises:

combining the respective portions of the snapshot manifest to form the snapshot manifest.

11. The computer implemented method of claim 1, wherein the master node comprises an HBASE master node and the slave nodes comprise HBASE slave nodes.

12. The computer implemented method of claim 1, wherein the data object comprises a Hadoop table.

13. The computer implemented method of claim 1, wherein the distributed database system continues to accept reads and writes from clients during creation of the snapshot.

14. A distributed database system having a master node comprising:

one or more processors;

a memory unit having instructions stored thereon which, when executed by the one or more processors cause the master node to create a manifest-based snapshot of a data object in a distributed cloud-computing platform by;

identifying multiple slave nodes of the distributed database system on which a data object is stored in the distributed cloud-computing platform, wherein the distributed database system includes the master node and the multiple slave nodes, wherein each slave node implements a region server and includes a data node associated with the region server;

creating a snapshot manifest representing the snapshot of the data object, wherein the snapshot manifest includes a file including a listing of multiple file names in the snapshot manifest and reference information for locating the multiple files in the distributed database system,

wherein creating the snapshot manifest further comprises directing each region server to create a portion of the snapshot manifest corresponding to the partition of the data on the data node with which the region server is associated,

wherein directing each region server to create the portion of the snapshot manifest corresponding to the partition of the data on the data node with which the region server is associated comprises;

generating, by the master node, a request to each region server for the respective portions of the snapshot manifest corresponding to the partition of the data on the data node with which the region servers are associated; and

sending, by the master node, the request to the corresponding region servers.

15. The distributed database system of claim 14, further comprising a slave node configured to perform a method comprising;

receiving, by the slave node of the distributed database system, a request to create the portion of the manifest-based snapshot from the master node of the distributed database system;

responsive to the request, creating, by the slave node the portion of the manifest-based snapshot corresponding to the partition of the data on the data node with which the region server is associated.

16. The distributed database system of claim 15, the method the slave node configured to perform further comprising:

prior to creation of the portion of the snapshot manifest, flushing memory of the region server.

17. The distributed database system of claim 15, the method the slave node configured to perform further comprising:

sending the portion of the snapshot manifest corresponding to the partition of the data on the data node with which the region server is associated to the master node.

18. The distributed database system of claim 15, wherein creating the portion of the manifest-based snapshot corresponding to the partition of the data on the data node with which the region server is associated further comprises dropping a marker in a log file indicating a position of the log file associated with the snapshot manifest.

19. The distributed database system of claim 15, wherein the portion of the snapshot manifest comprises a single file including a listing of multiple file names in the portion of the snapshot manifest and reference information for locating the multiple files on the data node.

20. The distributed database system of claim 15, wherein the portion of the snapshot manifest represents a single region.

21. The distributed database system of claim 15, wherein the master node comprises an HBASE master node and the slave nodes comprise HBASE slave nodes.

22. The distributed database system of claim 15, wherein the data object comprises a Hadoop table.