File-backed in-memory structured storage for service synchronization

• US 8,635,186 B2
• Filed: 02/25/2013
• Issued: 01/21/2014
• Est. Priority Date: 09/10/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Greene, Aaron K., Jayaraman, Maheshwar, Novik, Lev, Clark, Michael R.
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US13/776,205
• Publication Number:
  US 20130173545A1
• Time in Patent Office: 330 Days
• Field of Search: 707/635, 707/690, 707/634
• US Class Current: 707/635
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Providing synchronization to a local system. Embodiments may include downloading a batch of changes in a heterogeneous batch. The batch of changes is correlated to an anchor affiliated with a synchronization service. The anchor is a reference point indicating time or relative order. The batch of changes and the anchor are serialized to a non-volatile storage medium as a heterogeneous set. After serializing the batch of changes to a non-volatile storage medium as a heterogeneous set, entities in the batch of changes are parsed out into entities in in-memory representations. Similarly, embodiments may receive user input modifying a plurality of data entities, store on a non-volatile storage medium a serialized heterogeneous representation of the modified data entities, and upload the serialized heterogeneous representation to a synchronization service.

Claims

1. A hardware storage device having stored computer-executable instructions which, are executable by at least one hardware processor of a computing system having a non-volatile file system and memory for implementing the following:

downloading a batch of changes in a heterogeneous batch, wherein the batch of changes is correlated to an anchor affiliated with a synchronization service,

wherein the anchor, which is received from the synchronization service, is a reference point indicating time or relative order;

serializing the batch of changes and the anchor to a non-volatile storage medium as a heterogeneous set, such that the batch of changes, which includes a plurality of changes to a plurality of different entity types is stored as a single file in the non-volatile file system;

after serializing the batch of changes to a non-volatile storage medium as a heterogeneous set, parsing out entities in the batch of changes into a plurality of different homogeneous collections of entities by entity type, such that different entity types are grouped together into different collections;

storing the different entity types as separate collections as in-memory representations, such that the different entities in the batch of changes that are received from the synchronization service are maintained as in-memory representations that are separated by entity type as well as within a file in the file system that is associated with the anchor and that includes a grouping of heterogeneous entity changes;

detecting modification to a plurality of data entities and storing on the non-volatile storage medium a serialized heterogeneous representation of the modified data entities in a local change file that excludes the anchor;

uploading the serialized heterogeneous representation included in the local change file to the synchronization service; and

receiving an upload response from the synchronization service, wherein the upload response comprises a corresponding new anchor.

2. The storage device of claim 1, wherein the batch of changes in the heterogeneous batch is a delta from a previous in-memory representation.

3. The storage device of claim 2, wherein the parsing out entities in the batch of changes into entities in in-memory representations comprises combining changes in the batch of changes with an existing in-memory representation.

4. The storage device of claim 1, wherein the parsing out entities in the batch of changes into entities in in-memory representations comprises creating a new in-memory representation.

5. The storage device of claim 1, wherein the serializing the batch of changes to a non-volatile storage medium as a heterogeneous set comprises storing the batch of changes to a file system as a single download file that contains all of the entities sent in the batch of changes.

6. The storage device of claim 5, wherein the single download file comprises a file name or extension that indicates that the download file is a download file that contains entities sent in a batch of changes.

7. The storage device of claim 5, wherein the single download file comprises a file name or extension that includes a tick count representing a relative order position that the download file was written to the non-volatile storage medium.

8. The storage device of claim 1, wherein the acts are performed to provide a level of consistency without the use of atomic transactions.

9. The storage device of claim 1, wherein the computer-executable instructions, when executed by the at least one processor, are further configured to implement the following:

based on information in the upload response, storing the at least one current version of an entity that is also included in the upload response in an upload response file along with the new anchor returned with the upload response; and

storing any conflict or error returned in the upload response in a conflict file or error file, respectively, without the new anchor.

10. The storage device of claim 1, computer-executable instructions, when executed by the at least one processor, are further configured to implement the following:

retrieving a list of files;

for each file that is a conflict or error file, adding the file to a queue of files to be read when reading the rest of the files is complete;

for each file that is not a conflict or error file, reading entities and an anchor from the file, when the anchor is available; and

if the file is a file with an anchor affiliated with the synchronization service then clearing out existing in-memory data associated with the file; and

for each deferred conflict or error file added to the queue of files, reading the conflict or error into memory and mapping the conflict or error as a live entity reference to an in-memory version of the entity.

11. A computing system comprising:

at least one processor; and

memory having stored computer-executable instructions which, when executed by the at least one processor, implement a method of providing synchronization, the method comprising;

downloading a batch of changes in a heterogeneous batch, wherein the batch of changes is correlated to an anchor affiliated with a synchronization service,

wherein the anchor, which is received from the synchronization service, is a reference point indicating time or relative order;

serializing the batch of changes and the anchor to a non-volatile storage medium as a heterogeneous set, such that the batch of changes, which includes a plurality of changes to a plurality of different entity types is stored as a single file in the non-volatile file system;

after serializing the batch of changes to a non-volatile storage medium as a heterogeneous set, parsing out entities in the batch of changes into a plurality of different homogeneous collections of entities by entity type, such that different entity types are grouped together into different collections;

storing the different entity types as separate collections as in-memory representations, such that the different entities in the batch of changes that are received from the synchronization service are maintained as in-memory representations that are separated by entity type as well as within a file in the file system that is associated with the anchor and that includes a grouping of heterogeneous entity changes;

detecting modification to a plurality of data entities and storing on the non-volatile storage medium a serialized heterogeneous representation of the modified data entities in a local change file that excludes the anchor;

uploading the serialized heterogeneous representation included in the local change file to the synchronization service; and

receiving an upload response from the synchronization service, wherein the upload response comprises a corresponding new anchor.

12. The computing system of claim 11, wherein the batch of changes in the heterogeneous batch is a delta from a previous in-memory representation.

13. The computing system of claim 12, wherein the parsing out entities in the batch of changes into entities in in-memory representations comprises combining changes in the batch of changes with an existing in-memory representation.

14. The computing system of claim 11, wherein the parsing out entities in the batch of changes into entities in in-memory representations comprises creating a new in-memory representation.

15. The computing system of claim 11, wherein the file in the file system comprises a file name or extension that indicates that the download file is a download file that contains entities sent in a batch of changes, the file name or extension also including a tick count representing a relative order position that the file was written to the file system.

16. The computing system of claim 11, wherein the acts are performed to provide a level of consistency without the use of atomic transactions.

17. A computer-implemented method of providing synchronization to a local computing system, the method comprising:

downloading a batch of changes in a heterogeneous batch, wherein the batch of changes is correlated to an anchor affiliated with a synchronization service,

wherein the anchor, which is received from the synchronization service, is a reference point indicating time or relative order;

serializing the batch of changes and the anchor to a non-volatile storage medium as a heterogeneous set, such that the batch of changes, which includes a plurality of changes to a plurality of different entity types is stored as a single file in the non-volatile file system;

after serializing the batch of changes to a non-volatile storage medium as a heterogeneous set, parsing out entities in the batch of changes into a plurality of different homogeneous collections of entities by entity type, such that different entity types are grouped together into different collections;

storing the different entity types as separate collections as in-memory representations, such that the different entities in the batch of changes that are received from the synchronization service are maintained as in-memory representations that are separated by entity type as well as within a file in the file system that is associated with the anchor and that includes a grouping of heterogeneous entity changes;

detecting modification to a plurality of data entities and storing on the non-volatile storage medium a serialized heterogeneous representation of the modified data entities in a local change file that excludes the anchor;

uploading the serialized heterogeneous representation included in the local change file to the synchronization service; and

receiving an upload response from the synchronization service, wherein the upload response comprises a corresponding new anchor.

18. The method of claim 17, wherein the batch of changes in the heterogeneous batch is a delta from a previous in-memory representation and wherein parsing out entities in the batch of changes into entities in in-memory representations comprises combining changes in the batch of changes with an existing in-memory representation.

19. The method of claim 17, wherein the file contains all of the entities sent in the batch of changes, and wherein the file further comprises:

a file name or extension that indicates that the download file is a download file that contains entities sent in a batch of changes; and

a tick count representing a relative order position that the download file was written to the non-volatile storage medium.

20. The method of claim 17, wherein the upload response further includes at least one current version of an entity that corresponds to the batch of changes.