Data processing service

• US 8,918,363 B2
• Filed: 11/14/2012
• Issued: 12/23/2014
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Naidu, Siddartha, Agarwal, Amit, Sheldon, Michael, Kadatch, Andrew, Tigani, Jordan, Condit, Jeremy P.
• Primary Examiner(s): LE, HUNG D
• Application Number:
  US13/676,997
• Publication Number:
  US 20130124466A1
• Time in Patent Office: 769 Days
• Field of Search: None
• US Class Current: 707/610
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 7/00 Methods or arrangements for processing data by operating upon the order or content of the data handled logic circuits H03K19/00
• CPC Class Codes: G06F 8/453 Data distribution

Abstract

In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products. A computing system receives a first request to insert one or more first data values into a database table. The computing system identifies that first data stored by the database table is stored in a first logical partition of a logical collection of data. The logical collection of data is designated for replication among multiple data centers such that a copy of the logical collection of data is designated to be stored by each of the multiple data centers. The logical collection of data is logically partitioned into multiple logical partitions which together comprise the logical collection of data. The computing system sends, by the computing system and to the first data center, the first request to insert the one or more first data values into the database table.

Claims

1. A computer-implemented method, comprising:

receiving, by a computing system and from a remote computing device, a first request to insert one or more first data values into a first database table;

identifying, by the computing system, that first data stored by the first database table is stored in a first logical partition of a logical collection of data, wherein;

(i) the logical collection of data is designated for replication among multiple data centers such that a copy of the logical collection of data is designated to be stored by each of the multiple data centers,

(ii) the logical collection of data is logically partitioned into multiple logical partitions which together comprise the logical collection of data, and

(iii) the first logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that a first data center of the multiple data centers is designated as one data center, of the multiple data centers, that initially writes to the first logical partition of data;

sending, by the computing system and to the first data center, the first request to insert the one or more first data values into the first database table;

inserting, at the first data center, the one or more first data values into the first database table;

replicating the one or more first data values from the first data center to each of the multiple data centers other than the first data center, including a second data center of the multiple data centers;

receiving, by the computing system and from the remote device, a second request to insert one or more second data values into a second database table;

identifying, by the computing system, that second data stored by the second database table is stored in a second logical partition of the logical collection of data, wherein the second logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that the second data center of the multiple data centers is designated as one data center, of the multiple data centers, that initially writes to the second logical partition of data;

sending, by the computing system and to the second data center, the second request to insert the one or more second data values into the second database table;

inserting, at the second data center, the one or more second data values into the second database table; and

replicating the one or more second data values from the second data center to each of the multiple data centers other than the second data center, including the first data center.

2. The computer-implemented method of claim 1, wherein:

the first data center comprises over fifty computer processors; and

the second data center comprises over fifty computer processors.

3. The computer-implemented method of claim 1, further comprising:

receiving, by the computing system and from the remote device, a third request to insert one or more third data values into a third database table;

identifying, by the computing system, that third data stored by the third database table is stored in a third logical partition of the logical collection of data, wherein the third logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that a third data center of the multiple data centers is designated as a data center that initially writes to the third logical partition of data; and

sending, by the computing system and to the third data center, the third request to insert the one or more third data values into the third record of the third database table.

4. The computer-implemented method of claim 1, wherein:

the one or more first data values are replicated after the first data center has inserted the one or more first data values into the first database table; and

the one or more second data values are replicated after the second data center has inserted the one or more second data values into the second database table.

5. The computer-implemented method of claim 1, wherein:

the first request is received by the computing system before the replicating of the one or more first data values; and

the second request is received by the computing system before the replicating of the one or more second data values.

6. The computer-implemented method of claim 1, further comprising:

receiving, at the first data center, the one or more second data values as having been replicated from the second data center;

inserting, at the first data center, the one or more second data values into the second database table;

receiving, at the second data center, the one or more first data values as having been replicated from the first data center; and

inserting, at the second data center, the one or more first data values into the first database table.

7. The computer-implemented method of claim 6, wherein:

inserting, at first data center, the one or more first data values into the first database table includes inserting the one or more first data values into a copy of the first database table stored at first data center;

inserting, at second data center, the one or more second data values into the second database table includes inserting the one or more second data values into a copy of the second database table stored at second data center;

inserting, at first data center, the one or more second data values into the second database table includes inserting the one or more second data values into a copy of the second database table stored at first data center; and

inserting, at second data center, the one or more first data values into the first database table includes inserting the one or more first data values into a copy of the first database table stored at second data center.

8. The computer-implemented method of claim 1, wherein:

the first data stored by the first database table is columnar data arranged for querying by a columnar database querying system; and

the second data stored by the second database table is columnar data arranged for querying by the columnar database querying system.

9. The computer-implemented method of claim 1, further comprising:

receiving, by the computing system and from the remote computing device, a first query to select data from the first database table; and

sending, by the computing system, the first query to the second data center for selection of data that is responsive to the first query from a copy of the first database table stored at second data center.

10. The computer-implemented method of claim 1, wherein:

the first data center is an only of the multiple data centers that is designated as initially writing to the first logical partition of data; and

the second data center is an only of the multiple data centers that is designated as initially writing to the second logical partition of data.

11. The computer-implemented method of claim 1, wherein:

all data stored by the first database table is stored within the first logical partition; and

all data stored by the second database table is stored within the second logical partition.

12. The computer-implemented method of claim 1, wherein one of the multiple data centers comprises the computing system.

13. The computer-implemented method of claim 1, wherein the first data center is designated as an only data center of the multiple data centers to write to the first logical partition of data concurrently as the second data center is designated as an only data center of the multiple data centers to write to the second logical partition of data.

14. A computer-implemented system, comprising:

one or more computer processors; and

one or more computer-readable devices storing instructions that, when executed by the one or more computer processors, cause performance of operations that include;

receiving, by a computing system and from a remote computing device, a first request to insert one or more first data values into a first database table;

identifying, by the computing system, that first data stored by the first database table is stored in a first logical partition of a logical collection of data, wherein;

(i) the logical collection of data is designated for replication among multiple data centers such that a copy of the logical collection of data is designated to be stored by each of the multiple data centers,

(ii) the logical collection of data is logically partitioned into multiple logical partitions which together comprise the logical collection of data, and

(iii) the first logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that a first data center of the multiple data centers is designated as one data center, of the multiple data centers, that initially writes to the first logical partition of data;

sending, by the computing system and to the first data center, the first request to insert the one or more first data values into the first database table;

inserting, at the first data center, the one or more first data values into the first database table;

replicating the one or more first data values from the first data center to each of the multiple data centers other than the first data center, including a second data center of the multiple data centers;

receiving, by the computing system and from the remote device, a second request to insert one or more second data values into a second database table;

identifying, by the computing system, that second data stored by the second database table is stored in a second logical partition of the logical collection of data, wherein the second logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that the second data center of the multiple data centers is designated as one data center, of the multiple data centers, that initially writes to the second logical partition of data;

sending, by the computing system and to the second data center, the second request to insert the one or more second data values into the second database table;

inserting, at the second data center, the one or more second data values into the second database table; and

replicating the one or more second data values from the second data center to each of the multiple data centers other than the second data center, including the first data center.

15. A computer-implemented method, comprising:

receiving, by a computing system and from a remote computing device, a first request to insert one or more first data values into a first database table;

identifying, by the computing system, that first data stored by the first database table is stored in a first logical partition of a logical collection of data, wherein;

(i) the logical collection of data is designated for replication among multiple data centers such that a copy of the logical collection of data is designated to be stored by each of the multiple data centers,

(ii) each of the multiple data centers comprises over a fifty computer processors;

(iii) the logical collection of data is logically partitioned into multiple logical partitions which together comprise the logical collection of data, and

(iv) the first logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that a first data center of the multiple data centers is designated as an only one data center, of the multiple data centers, that initially writes to the first logical partition of data;

sending, by the computing system and to the first data center, the first request to insert the one or more first data values into the first database table;

inserting, by the first data center, the one or more first data values into a copy of the first database table stored by the first data center;

replicating, after the insertion of the one or more first data values into the copy of the first database table stored by the first data center, the one or more first data values from the first data center to each of the multiple data centers other than the first data center, including the second data center;

receiving, by a second data center of the multiple data centers, the one or more first data values as having been replicated from the first data center;

inserting, by the second data center, the one or more first data values into a copy of the first database table stored by the second data center;

receiving, by the computing system and from the remote device, a second request to insert one or more second data values into a second database table;

identifying, by the computing system, that second data stored by the second database table is stored in a second logical partition of the logical collection of data, wherein the second logical partition is one of the multiple logical partitions of data;

identifying, by the computing system, that the second data center is designated as an only one data center, of the multiple data centers, that initially writes to the second logical partition of data, wherein the second data center is designated as an only one data center, of the multiple data centers, that initially writes to the second logical partition of data concurrent with the first data center being designated as the only one data center, of the multiple data centers, that initially writes to the first logical partition of data;

sending, by the computing system and to the second data center, the second request to insert the one or more second data values into the second database table;

inserting, by the second data center, the one or more second data values into a copy of the second database table stored by the second data center;

replicating, after the insertion of the one or more second data values into the copy of the second database table stored by the second data center, the one or more second data values from the second data center to each of the multiple data centers other than the second data center, including the first data center;

receiving, by the first data center, the one or more second data values as having been replicated from the second data center; and

inserting, by the first data center, the one or more second data values into a copy of the second database table stored by the first data center.