Database replication

• US 8,838,539 B1
• Filed: 10/05/2012
• Issued: 09/16/2014
• Est. Priority Date: 10/05/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kasera, Vishal, Leon, Jean-Michel, Agarwal, Amit, Ashcraft, Kenneth M
• Primary Examiner(s): LE, DEBBIE M
• Application Number:
  US13/646,015
• Time in Patent Office: 711 Days
• Field of Search: 707/636, 707/637
• US Class Current: 707/637
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 49/9005 using dynamic buffer space allocation
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A database server receives a request from a client application for performing a data transaction on persistent data storage. The request is sent to a set of replication servers. An acknowledgement for the request is received from each replication server, including a start sequence number and an end sequence number for data that is stored in local cache of the replication server, and a latest committed sequence number for data that was written to the persistent data storage by the replication server. A maximum value of latest committed sequence numbers received from the set of replication servers is determined. For each replication server, it is examined whether there is a gap between the start sequence number for data stored in local cache and the maximum value of the latest committed sequence numbers. Based on the examining, it is determined whether there is an occurrence of loss of data.

Claims

1. A method for detecting data loss in a replicated database, comprising:

receiving, at a database server, a request from a client application for performing a data transaction on persistent data storage;

sending the request to a set of replication servers;

receiving, from each replication server, an acknowledgement for the request, including a start sequence number and an end sequence number for data associated with the client application that is stored in local cache of the replication server, and a latest committed sequence number for data associated with the client application that was written to the persistent data storage by the replication server;

determining a maximum value of latest committed sequence numbers received from the set of replication servers;

examining, for each replication server, whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application.

2. The method of claim 1, wherein examining whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers comprises:

identifying that there is a gap between the start sequence number for data stored in local cache of each replication server and the determined maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application comprises determining that there is an occurrence of loss of data associated with the client application based on identifying that there is a gap for each replication server.

3. The method of claim 1, wherein examining whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers comprises:

identifying at least one replication server for which there is no gap between the start sequence number for data stored in the respective local cache and the determined maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application comprises determining that there is no occurrence of loss of data associated with the client application based on identifying the at least one replication server.

4. The method of claim 1, wherein a replication server is configured for performing read and write operations directly on the persistent data storage.

5. The method of claim 1, wherein sending the request to a set of replication servers comprises:

determining a set of replication servers that are configured for performing data transactions associated with the client application; and

sending the request to the determined set of replication servers.

6. A computer program product, suitably embodied in a non-transitory machine-readable medium and including instructions executable by a processor for detecting data loss in a replicated database, the instructions configured to cause the processor to perform operations comprising:

receiving, at a database server, a request from a client application for performing a data transaction on persistent data storage;

sending the request to a set of replication servers;

receiving, from each replication server, an acknowledgement for the request, including a start sequence number and an end sequence number for data associated with the client application that is stored in local cache of the replication server, and a latest committed sequence number for data associated with the client application that was written to the persistent data storage by the replication server;

determining a maximum value of latest committed sequence numbers received from the set of replication servers;

examining, for each replication server, whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application.

7. The computer program product of claim 6, wherein the instructions that are configured to cause the processor to perform operations comprising examining whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers includes instructions that are configured to cause the processor to perform operations comprising:

identifying that there is a gap between the start sequence number for data stored in local cache of each replication server and the determined maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application comprises determining that there is an occurrence of loss of data associated with the client application based on identifying that there is a gap for each replication server.

8. The computer program product of claim 6, wherein the instructions that are configured to cause the processor to perform operations comprising examining whether there is a gap between the start sequence number for data stored in local cache of the replication server and the maximum value of the latest committed sequence numbers includes instructions that are configured to cause the processor to perform operations comprising:

identifying at least one replication server for which there is no gap between the start sequence number for data stored in the respective local cache and the determined maximum value of the latest committed sequence numbers; and

based on the examining, determining whether there is an occurrence of loss of data associated with the client application comprises determining that there is no occurrence of loss of data associated with the client application based on identifying the at least one replication server.

9. The computer program product of claim 6, wherein a replication server is configured for performing read and write operations directly on the persistent data storage.

10. The computer program product of claim 6, wherein the instructions that are configured to cause the processor to perform operations comprising sending the request to a set of replication servers includes instructions that are configured to cause the processor to perform operations comprising:

determining a set of replication servers that are configured for performing data transactions associated with the client application; and

sending the request to the determined set of replication servers.