Table and index communications channels

• US 10,216,768 B1
• Filed: 02/17/2014
• Issued: 02/26/2019
• Est. Priority Date: 02/17/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Huang, Xianglong, Lu, Yijun, Xiao, Wei, Zheng, Jiandan
• Primary Examiner(s): Meng, Jau Shya
• Application Number:
  US14/182,258
• Time in Patent Office: 1,835 Days
• Field of Search: None
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning

Abstract

One or more table partitions may communicate with an index partition that may be a master of a replication group. A communications channel may exist between table partitions and the index partition. Upon splitting the index partition, communications between the table partitions and the index partition may be suspended. Upon completion of the split, communications may be reestablished between the table partitions and a partition, of the replication group of index partitions, designated to be a master following the split. Messages accumulated by the table partitions during the split may be sent to the index partition upon reestablishing communications.

Claims

1. A system comprising:

at least one computing node comprising at least one processor and at least one memory, wherein the at least one memory comprises instructions that, when executed by the at least one processor, cause the system to at least;

store one or more messages in a first buffer, the one or more messages indicative of one or more updates to a plurality of items stored on a first table partition, the first buffer associated with a first communications channel operable to transmit the one or more messages from the first table partition to a first index partition comprising a first index of the plurality of items;

suspend transmission of the one or more messages over the first communications channel in response to receiving information indicative of a split of the first index;

form a second index on a second index partition, the second index corresponding to a subset of the plurality of items;

form a second communications channel between the first table partition and the second index partition, the second index partition identified as a master of a replication group comprising the first and second index partitions; and

transmit, over the second communications channel, a message of the one or more messages in the first buffer to the second index partition.

2. The system of claim 1, wherein the at least one memory comprises further instructions that, upon execution by the at least one processor, cause the system to at least:

transmit the message of the one or more messages to the second index partition based at least in part on information indicative of a last message processed by the first index partition.

3. The system of claim 1, wherein the at least one memory comprises further instructions that, upon execution by the at least one processor, cause the system to at least:

transmit an additional message to the second index partition over the second communications channel, the additional message indicative of an update to the first table that occurred after suspension of communications between the first table partition and the first index partition.

4. The system of claim 1, wherein the at least one memory comprises further instructions that, upon execution by the at least one processor, cause the system to at least:

form the second communications channel in response to receiving, at the first table partition, a message indicative of forming the second index.

5. The system of claim 1, wherein the at least one memory comprises further instructions that, upon execution by the at least one processor, cause the system to at least:

store an additional message in a second buffer associated with the second communications channel, the additional message indicative of an update to the subset of the plurality of items.

6. A computer-implemented method comprising:

storing, by one or more processors of one or more computers, one or more messages in a first buffer, the one or more messages indicative of one or more updates to a plurality of items stored on a first table partition, the first buffer associated with a first communications channel between a first table partition and a first index partition;

suspending, by the one or more processors, transmission of the one or more messages over the first communications channel; and

forming, by the one or more processors, a second communications channel between the table partition and a second index partition formed from the first index partition, the second index partition selected from a plurality of index partitions to receive messages from the first table partition, the second index partition comprising a second index of a subset of the plurality of items.

7. The computer-implemented method of claim 6, wherein a replication group comprises the plurality of index partitions.

8. The computer-implemented method of claim 6, further comprising:

transmitting a message of the one or more messages to the second index via the second communications channel.

9. The computer-implemented method of claim 6, further comprising:

transmitting a message of the one or more messages to the second index partition based at least in part on information indicative of a last message processed by the first index partition.

10. The computer-implemented method of claim 6, further comprising:

transmitting an additional message to the second index partition over the second communications channel, the additional message indicative of an update stored in the first buffer after suspending communications between the first table partition and the first index partition.

11. The computer-implemented method of claim 6, further comprising:

forming the second communications channel in response to a message indicative of forming the second index.

12. The computer-implemented method of claim 6, further comprising:

forming the second communications channel based at least in part on information indicative of the second index partition being selected from a plurality of index partitions to receive messages from the first table partition, the plurality of index partitions comprising the second index partition.

13. The computer-implemented method of claim 6, further comprising:

storing an additional message in a second buffer associated with the second communications channel, the additional message indicative of an update to the subset of the plurality of items.

14. A non-transitory computer-readable storage medium having stored thereon instructions that, upon execution by one or more computing devices, cause the one or more computing devices at least to:

store one or more messages in a first buffer, the one or more messages indicative of one or more updates to a plurality of items maintained on a first table partition, the buffer associated with a first communications channel between a first table partition and a first index partition;

suspend transmission of the one or more messages over the first communications channel; and

form a second communications channel between the table partition and a second index partition formed from the first index partition, the second index partition selected from a plurality of index partitions to receive messages from the first table partition.

15. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

transmit a message of the one or more messages to the second index partition in response to receiving information indicative of a last message processed by the first index partition.

16. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

transmit an additional message to the second index partition over the second communications channel, the additional message indicative of an update stored in the first buffer after suspending communications between the first table partition and the first index partition.

17. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

form a second buffer associated with the second communications channel, the second buffer comprising the one or more messages.

18. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

receive, at the table partition, information indicative of the second index partition being selected from a plurality of index partitions to receive messages from the first table partition, the plurality of index partitions comprising the second index partition.

19. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

form the second communications channel by at least identifying the first table partition as a master table partition.

20. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

form the second communications channel by performing steps selected from a group consisting of communicating directly between the first table partition and the first index partition, communicating between the first table partition and a plurality of index partitions, communicating between a plurality of table partitions and the first index partition, retrieving metadata from a repository, and communicating via a connection broker.