UPDATING DISTRIBUTED SHARDS WITHOUT COMPROMISING ON CONSISTENCY

• US 20160203168A1
• Filed: 03/11/2015
• Published: 07/14/2016
• Est. Priority Date: 01/09/2015
• Status: Active Grant

• Current Assignee: Ariba Incorporated (SAP SE)
• Original Assignee: Ariba Incorporated (SAP SE)
• Inventors: Shahane, Dinesh, Gangadharappa, Kiran
• Granted Patent: US 10,303,796 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

In an example embodiment, a first data change and a second data change to data stored in a distributed database are received. The first data change and the second data change are batched in a communication to an index manager. The distributed database is reindexed based on the first data change and the second data change by creating a revised first shard index for a first shard in the distributed database and a revised second shard index for a second shard in the distributed database. The first shard and the second shard are instructed to update respective shard indexes. Confirmation that the first shard index has been updated is received. Then confirmation that the second shard index has been updated is received. In response to both of the confirmations, both the first shard and the second shard are instructed to commit their respective updates.

Claims

1. A method comprising:

receiving a first data change to data stored in a distributed database;

receiving a second data change to the data stored in the distributed database;

batching the first data change and the second data change in a communication to an index manager;

reindexing the distributed database based on the first data change and the second data change by creating a revised first shard index for a first shard in the distributed database and a revised second shard index for a second shard in the distributed database;

instructing the first shard to update a first shard index based on the revised first shard index;

instructing the second shard to update a second shard index based on the revised second shard index;

receiving confirmation that the first shard index has been updated;

receiving confirmation that the second shard index has been updated; and

in response to both the receiving the confirmation that the first shard index has been updated and the confirmation that the second shard index has been updated, instructing the first shard to commit the revised first shard index and the second shard to commit the revised second shard index.

2. The method of claim 1, wherein the first data change is made to primary data.

3. The method of claim 1, wherein the first data change is made to auxiliary data related to primary data.

4. The method of claim 1, wherein the first data change is a change to a factor influencing rebalancing of data among the first and second shards.

5. The method of claim 1, wherein the instructing the first shard to commit the revised first shard index and the second shard to commit the revised second shard index is performed in an order identical to an order in which the first data change and the second data change were received.

6. The method of claim 1, wherein the reindexing is performed by an index builder in communication with a coordinator.

7. The method of claim 6, wherein the coordinator maintains shard information.

8. A system comprising:

a distributed database;

a search core; and

an indexing core executable on one or more processors, the indexing core comprising;

a plurality of different index adapters, each index adapter corresponding to a different document type of a different client application;

an index builder;

an index manager;

a queue manager configured to;

receive a first data change to data stored in the distributed database;

receive a second data change to the data stored in the distributed database; and

batch the first data change and the second data change in a communication to the index manager;

the index manager configured to;

cause the index builder to reindex the distributed database based on the first data change and the second data change by creating a revised first shard index for a first shard in the distributed database and a revised second shard index for a second shard in the distributed database;

instruct the first shard to update a first shard index based on the revised first shard index;

instruct the second shard to update a second shard index based on the revised second shard index;

receive confirmation that the first shard index has been updated;

receive confirmation that the second shard index has been updated; and

in response to both the receiving the confirmation that the first shard index has been updated and the confirmation that the second shard index has been updated, instruct the first shard to commit the revised first shard index and the second shard to commit the revised second shard index.

9. The system of claim 8, wherein the first data change is made to primary data.

10. The system of claim 8, wherein the first data change is made to auxiliary data related to primary data.

11. The system of claim 8, wherein the first data change is a change to a factor influencing rebalancing of data among the first and second shards.

12. The system of claim 8, wherein the instructing the first shard to commit the revised first shard index and the second shard to commit the revised second shard index is performed in an order identical to an order in which the first data change and the second data change were received.

13. The system of claim 8, further comprising a coordinator configured to maintain shard information.

14. A non-transitory machine-readable storage medium comprising instructions, which when implemented by one or more machines, cause the one or more machines to perform operations comprising:

receiving a first data change to data stored in a distributed database;

receiving a second data change to the data stored in the distributed database;

batching the first data change and the second data change in a communication to an index manager;

reindexing the distributed database based on the first data change and the second data change by creating a revised first shard index for a first shard in the distributed database and a revised second shard index for a second shard in the distributed database;

instructing the first shard to update a first shard index based on the revised first shard index;

instructing the second shard to update a second shard index based on the revised second shard index;

receiving confirmation that the first shard index has been updated;

receiving confirmation that the second shard index has been updated; and

in response to both the receiving the confirmation that the first shard index has been updated and the confirmation that the second shard index has been updated, instructing the first shard to commit the revised first shard index and the second shard to commit the revised second shard index.

15. The non-transitory machine-readable storage medium of claim 14, wherein the first data change is made to primary data.

16. The non-transitory machine-readable storage medium of claim 14, wherein the first data change is made to auxiliary data related to primary data.

17. The non-transitory machine-readable storage medium of claim 14, wherein the first data change is a change to a factor influencing rebalancing of data among the first and second shards.

18. The non-transitory machine-readable storage medium of claim 14, wherein the instructing the first shard to commit the revised first shard index and the second shard to commit the revised second shard index is performed in an order identical to an order in which the first data change and the second data change were received.

19. The non-transitory machine-readable storage medium of claim 14, wherein the reindexing is performed by an index builder in communication with a coordinator.

20. The non-transitory machine-readable storage medium of claim 19, wherein the coordinator maintains shard information.