Database replication

• US 10,007,715 B1
• Filed: 06/03/2016
• Issued: 06/26/2018
• Est. Priority Date: 10/05/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Ashcraft, Kenneth M., Kasera, Vishal, Leon, Jean-Michel, Agarwal, Amit
• Primary Examiner(s): Le, Debbie
• Application Number:
  US15/172,582
• Time in Patent Office: 753 Days
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 49/9005 using dynamic buffer space allocation
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A database server receives a request from a client application for performing a data transaction on persistent data storage. The request is sent to a set of replication servers. An acknowledgement for the request is received from each replication server, including a start sequence number and an end sequence number for data that is stored in local cache of the replication server, and a latest committed sequence number for data that was written to the persistent data storage by the replication server. A maximum value of latest committed sequence numbers received from the set of replication servers is determined. For each replication server, it is examined whether there is a gap between the start sequence number for data stored in local cache and the maximum value of the latest committed sequence numbers. Based on the examining, it is determined whether there is an occurrence of loss of data.

Claims

1. A method comprising:

providing, to a user associated with a client application, a set of different levels of quality guarantee associated with a database server, wherein the database server is configured for performing reads and writes on persistent data storage for the client application;

receiving, from the user, selection of a particular level of quality guarantee from the set of different levels of quality guarantee;

responsive to receiving the selection of the particular level of quality guarantee, determining values of one or more attributes associated with the database server, the one or more attributes including different values for different levels of quality guarantee; and

based on determining the values of the one or more attributes, configuring a set of replications servers for performing read and write operations on the persistent data storage based on the read and write requests from the client application;

receiving, at a database server, a write request;

in response to receiving the write request, selecting, by the database server, a set of multiple replication servers based on the determined value of the one or more attributes;

sending the data from the database server to the selected set of multiple replication servers for writing to the persistent data storage;

determining that less than all of the replication servers successfully wrote the data to the persistent data storage; and

in response to determining that less than all of the replication servers successfully wrote the data to the persistent data storage, selecting, by the database server and based on the determined value of the one or more attributes, one or more additional replication servers as replacement for the replication servers that did not successfully write the data to persistent storage; and

sending the data from the database server to the additional replication servers.

2. The method of claim 1, wherein a replication server is configured for performing read and write operations directly on the persistent data storage.

3. The method of claim 1, wherein the different levels of quality guarantee are based on different price structures.

4. The method of claim 1, wherein the one or more attributes include data durability provided to data associated with the client application, latency experienced by the client application in using the database server for performing read and write requests on the persistent data storage and peak throughput of the replication servers in writing data to the persistent data storage.

5. The method of claim 4, wherein configuring a set of replications servers comprises:

selecting a number and geographic distribution of replication servers for performing read and write requests on persistent data storage for the client application, the number and the geographic distribution based on the value of data durability to be provided to data associated with the client application.

6. The method of claim 5, wherein the number of the replication servers selected is higher for higher values of data durability than lower values of data durability.

7. The method of claim 5, wherein the geographic distribution of the replication servers selected is greater for higher values of data durability than lower values of data durability.

8. The method of claim 7, wherein the geographic distribution of the replication servers selected is lower for higher values of the peak throughput.

9. The method of claim 4, wherein configuring a set of replications servers comprises:

configuring a size of local cache in each replication server, the local cache in a replication server configured for temporary storage of data for the client application before being written to the persistent data storage.

10. The method of claim 9, wherein the size of the local cache in a replication server is configured to be smaller for higher values of data durability than lower values of data durability.

11. The method of claim 10, wherein the size of the local cache in a replication server is configured to be larger for lower values of latency experienced by the client application than higher values of latency.

12. The method of claim 10, wherein the size of the local cache in a replication server is configured to be larger for higher values of the peak throughput.

13. The method of claim 10, wherein configuring a set of replications servers comprises:

selecting a number and geographic distribution of persistent data storage for performing read and write requests for the client application, the number and the geographic distribution based on at least one of the value of data durability to be provided to data associated with the client application and the latency experienced by the client application.

14. The method of claim 13, wherein the number of the persistent data storage selected is higher for higher values of data durability than lower values of data durability.

15. The method of claim 13, wherein the number of the persistent data storage selected is lower for lower values of latency experienced by the client application than higher values of latency.

16. The method of claim 13, wherein the geographic distribution of the persistent data storage selected is greater for higher values of data durability than lower values of data durability.

17. The method of claim 13, wherein the geographic distribution of the persistent data storage selected is lower for higher values of the peak throughput.

18. The method of claim 1, further comprising:

receiving, at the database server from the user of the client application at a time when the database server is processing read and write requests for the client application based on the particular level of quality guarantee selected previously, selection of a new level of quality guarantee from the set of different levels of quality guarantee, the new level of quality guarantee being different from the particular level of quality guarantee selected previously;

responsive to receiving the selection of the new level of quality guarantee, determining new values of the one or more attributes associated with the database server; and

based on the new values of the one or more attributes, configuring a new set of replications servers for performing read and write operations on the persistent data storage based on the read and write requests from the client application.

19. The method of claim 1, wherein selecting the set of multiple replication servers in response to receiving the write request comprises:

determining, by the database server, that a mastership is acquired by the database server for servicing the write request; and

in response to determining that a mastership is acquired, selecting, by the database server, the set of multiple replication servers.

20. The method of claim 19, wherein the mastership indicates a lease on persistent data storage by an instance of the database server that is configured for servicing the write request from the client application.

21. The method of claim 1, wherein selecting the set of multiple replication servers comprises:

selecting the set of multiple replication servers from an available pool of replication servers based on a random selection mechanism.

22. The method of claim 1, wherein selecting the set of multiple replication servers comprises:

determining an attribute chosen by a user associated with the client application, the attribute selected from the group consisting of a number of replication servers, location of replication servers, size of cache in each replication server used for the client application, location of the persistent data storage and size of the persistent data storage; and

selecting the set of multiple replication servers based on the determined attribute.

23. The method of claim 1, wherein selecting the set of multiple replication servers comprises:

performing the selection of the set of multiple replication servers independently for each write request.

24. The method of claim 1, further comprising:

receiving, at the database server, confirmation from the one or more additional replication servers; and

in response to receiving confirmation from the one or more additional replication servers, sending from the database server to the client application, information indicating success of the write request.

25. A computer program product embodied in a non-transitory machine-readable medium and including instructions executable by one or more computers, the instructions configured to cause the one or more computers to perform operations comprising:

providing, to a user associated with a client application, a set of different levels of quality guarantee associated with a database server, wherein the database server is configured for performing reads and writes on persistent data storage for the client application;

receiving, from the user, selection of a particular level of quality guarantee from the set of different levels of quality guarantee;

responsive to receiving the selection of the particular level of quality guarantee, determining values of one or more attributes associated with the database server, the one or more attributes including different values for different levels of quality guarantee; and

based on determining the values of the one or more attributes, configuring a set of replications servers for performing read and write operations on the persistent data storage based on the read and write requests from the client application;

receiving, at a database server, a write request;

in response to receiving the write request, selecting, by the database server, a set of multiple replication servers based on the determined value of the one or more attributes;

sending the data from the database server to the selected set of multiple replication servers for writing to the persistent data storage;

determining that less than all of the replication servers successfully wrote the data to the persistent data storage; and

in response to determining that less than all of the replication servers successfully wrote the data to the persistent data storage, selecting, by the database server and based on the determined value of the one or more attributes, one or more additional replication servers as replacement for the replication servers that did not successfully write the data to persistent storage; and

sending the data from the database server to the additional replication servers.