Local key-value database synchronization

• US 9,703,814 B1
• Filed: 10/15/2013
• Issued: 07/11/2017
• Est. Priority Date: 10/15/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Shams, Khawaja Salman, Pandey, Prashant, Zaki, Omer Ahmed, Yanacek, David Craig, Olson, Johanna S, de Kock, Hendrik Jacobus, Wu, Johnny Ying, Sivasubramanian, Swaminathan
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US14/054,760
• Time in Patent Office: 1,365 Days
• Field of Search: 707638
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

A remote distributed data store may be configured to process data updates received through invocation of a common API with reference to a common schema. A local data store may also be configured to process updates through the common API with reference to the common schema. Updates to the local data store may be mapped from the local data store schema to the common schema, and applied to the distributed data store. Updates to the distributed data store may be mapped from the common schema to the local data store schema. User identity may be verified to limit data synchronization to authorized users.

Claims

1. A system comprising:

one or more computing nodes operating a distributed data store, wherein the one or more computing nodes at least;

receive a request to update a first item of a first collection of items stored in the distributed data store, wherein the first collection of items corresponds to a first user and is stored according to a first schema;

receive, from a computing device storing a second collection of items, information indicative of an update to a second item of the second collection of items, wherein the update to the second item is based on a second schema;

map the update to the second item based on the second schema to an update to the second item based on the first schema; and

synchronize, in accordance with throughput restrictions corresponding to the first user and based at least in part on mapping the update to the second item based on the second schema to the update to the second item based on the first schema, the first item of the first collection of items with the second item of the second collection of items.

2. The system of claim 1, wherein the one or more computing nodes at least:

send information indicative of the request to update the first item based at least in part on reestablishing network connectivity between the one or more computing nodes and the computing device.

3. The system of claim 1, wherein the request to update the first item is received from a web service providing a hosted application to a plurality of users.

4. The system of claim 1, wherein the first schema comprises a table name and key description.

5. The system of claim 1, wherein the first one or more computing nodes at least:

delay sending the information indicative of the request to update the first item, based at least in part on an amount of throughput consumed by reading or writing data stored in the distributed data store.

6. A method comprising:

receiving, on one or more computing nodes, a request to update a first one or more items of a first collection of items stored on the one or more computing nodes, the first collection of items conforming to a first schema, the first collection of items corresponding to a first user;

receiving, from a computing device storing a second collection of items, information indicative of a request to update a second one or more items in the second collection of items, the second collection of items conforming to a second schema;

mapping the update to the second one or more items based on the second schema to an update to the second one or more items based on the first schema; and

synchronizing, in accordance with throughput restrictions corresponding to the first user and based on the mapped update to the second one or more items based on the first schema, the first one or more items of the first collection of items with the second one or more items of the second collection of items.

7. The method of claim 6, further comprising:

determining a time to send the information indicative of the request to update the second one or more items, the time based at least in part on a throughput usage pattern for accessing the first collection of items stored on the one or more computing nodes.

8. The method of claim 6, wherein the second schema corresponds to a relational data model.

9. The method of claim 6, wherein mapping from the second schema to the first schema comprises mapping from one or more relational database tables to a key-value collection.

10. The method of claim 6, further comprising:

sending information indicative of all items in the second collection of items to at least one of the one or more computing nodes.

11. The method of claim 6, wherein mapping from the first schema to the second schema comprises mapping from a first set of one or more key-value collections to a second set of one or more key value collections.

12. The method of claim 6, wherein the request to update the first one or more items is received from a web service providing a hosted application to a plurality of users.

13. The method of claim 6, further comprising:

sending information indicative of the request to update of the first one or more items to the computing device, based at least in part on an amount of time since receiving the request to update the first one or more items.

14. A non-transitory computer-readable storage medium having stored thereon instructions that, upon execution by one or more computing devices, cause the one or more computing devices at least to:

receive, at the one or more computing devices, a request to update a first one or more items of a first collection of items stored on the one or more computing devices, the first collection of items conforming to a first schema, the first collection of items corresponding to a first user;

receive, from a computer system storing a second collection of items, information indicative of a request to update a second one or more items in the second collection of items, the second collection of items conforming to a second schema;

map the update to the second one or more items based on the second schema to an update to the second one or more items based on the first schema; and

synchronize, in accordance with throughput restrictions corresponding to the first user and based on the mapped update to the second one or more items based on the first schema, the first one or more items of the first collection of items with the second one or more items of the second collection of items.

15. The non-transitory computer-readable medium of claim 14, wherein the request to update the second one or more items is based at least in part on an invocation based at least in part on the first schema, wherein the request to update the first one or more items is based at least in part on an invocation based at least in part on the first schema.

16. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

determine to include the information indicative of the request to update the second one or more items, based at least in part on a filtering criteria.

17. The non-transitory computer-readable medium of claim 14, wherein the second schema corresponds to at least one of a document-oriented database schema, an object-oriented database schema, a JSON store schema, or an in-memory data structure schema.

18. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

send information indicative of the request to update of the first one or more items to the computer system, based at least in part on an amount of time since receiving the request to update the first one or more items.

19. The non-transitory computer-readable medium of claim 14, comprising further instructions that, upon execution by the one or more computing devices, cause the one or more computing devices to at least:

store an association between a verified identity of a user, group, or role and an item in the first collection of items.