Seamless data migration across databases

• US 10,831,779 B2
• Filed: 06/27/2017
• Issued: 11/10/2020
• Est. Priority Date: 03/04/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kritchko, Vladimir, He, Xin, Grebenik, Vladimir V., Yan, Kangrong
• Primary Examiner(s): Lee, Wilson
• Application Number:
  US15/634,552
• Publication Number:
  US 20170357645A1
• Time in Patent Office: 1,232 Days
• Field of Search: None
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Techniques are disclosed for enabling the migration of data with minimized impact on consumers of the data. A data migration agent updates pointers to active data locations and coordinates a migration from a first data resource to a second data resource so that seamless migration may be carried out. A data access layer of a distributed computing environment can take advantage of the pointers and metadata written by the data migration agent to serve requests to resources of the distributed computing environment so that read availability is available regardless of migration status.

Claims

1. A method of facilitating data requests from consumers to resources in a distributed computing environment, comprising:

receiving a request for interacting with a database from a data consumer; and

communicating the request to a data storage resource storing the database, the communicating carried out regardless of migration state of the database, wherein communicating the request comprises;

confirming whether one or more partitions of the database are in an active state, wherein confirming that the one or more partitions of the database are in the active state comprises;

communicating with a data locator to retrieve an indication of an active one or more partitions of the database to serve the request, wherein the indication of the active one or more partitions comprises a directed partition;

reading metadata associated with the one or more partitions;

determining whether or not the data locator and the directed partition indicate a same active partition;

in response to determining that the data locator and the directed partition indicate the same active partition;

returning the active partition; and

in response to determining that the data locator and the directed partition do not indicate the same active partition;

querying metadata on a corresponding partition on a remaining database for the active partition;

selecting the active partition directed by a majority of the data locator, the directed partition and the corresponding partition; and

returning the active partition; and

in response to the active state of the one or more partitions being not confirmed, communicating the request to the database by performing the request on one of a source resource from or a target resource to which at least one of the one or more partitions is being migrated.

2. The method of claim 1, further comprising:

in response to the active state of the one or more partitions being confirmed, communicating the request to the database by performing the request on the one or more partitions.

3. The method of claim 1, wherein the request for interacting with the database is a read request, wherein communicating the request to the data storage resource storing the database further comprises:

mapping the request to the one or more active partitions; and

retrieving data from the one or more active partitions.

5. The method of claim 1, wherein the request for interacting with the database from the data consumer comprises a request for a cached data object, a value from a system-controlled identity field being appended to a user-controlled attribute in the target resource;

wherein communicating the request to the data storage resource storing the database further comprises;

requesting, from the data locator, a location of an active partition for the cached data object; and

identifying and matching an identical data object on either the source resource by using the value from the system-controlled identity field or the target resource by using the user-controlled attribute.

4. A method of facilitating data requests from consumers to resources in a distributed computing environment, comprising:

receiving a request for interacting with a database from a data consumer;

obtaining a location of an active partition according to a pointer indicating an active data source, the pointer indicating whether a source partition or a target partition is the active data source;

if the pointer indicates that the source partition is the active data source, communicating with the source partition to read source partition metadata, wherein the source partition metadata indicates whether the source partition is in an active state or an inactive state;

if the source partition metadata of the source partition indicates that the source partition is in the active state, confirming that the source partition is the active partition and servicing the request from the data consumer using the source partition;

if the source partition metadata of the source partition indicates that the source partition is in the inactive state, communicating with the target partition to confirm that the target partition is the active partition and service the request from the data consumer using the target partition;

if the pointer indicates that the target partition is the active data source, communicating with the target partition to read target partition metadata, wherein the target partition metadata indicates whether the target partition is in the active state or the inactive state;

if the target partition metadata of the target partition indicates that the target partition is in the active state, confirming that the target partition is the active partition and servicing the request from the data consumer using the target partition; and

if the target partition metadata of the target partition indicates that the target partition is in the inactive state, communicating with the source partition to confirm that the source partition is the active partition and servicing the request from the data consumer using the source partition.

13. The method of claim4, wherein during migration of data from the source partition to the target partition, the source partition is read-access available,

wherein during the migration of the data from the source partition to the target partition, the source partition is also write-access available except during a time period when deltas are being transferred from the source partition to the target partition, wherein during the time period when deltas are being transferred from the source partition to the target partition, the source partition remains read-access available.

14. The method of claim 4, wherein the pointer indicates the source partition as the active partition during migration of a replica from the source partition to the target partition, wherein the pointer indicates the target partition as the active partition after a source delta is applied to the target partition.

15. The method of claim 14, wherein the source partition metadata changes from the active state to the inactive state and the target partition metadata changes from the inactive state to the active state after the source delta is applied to the target position.

6. A system for facilitating data requests from consumers to resources, the system comprising:

a server comprising a data access layer, the data access layer receiving a request for interacting with a database from a data consumer and communicating the request to a data storage resource storing the database, the communication carried out regardless of migration state of the database, wherein the communicating of the request by the data access layer to the data storage resource storing the database comprises;

communicating with a data locator service to obtain a location of an active partition according to a pointer indicating an active data source, the pointer indicating whether a source partition or a target partition is the active data source;

if the pointer indicates that the source partition is the active data source, communicating with the source partition to read source partition metadata, wherein the source partition metadata indicates whether the source partition is in an active state or an inactive state;

if the source partition metadata of the source partition indicates that the source partition is in the active state, confirming that the source partition is the active partition and servicing the request from the data consumer using the source partition;

if the source partition metadata of the source partition indicates that the source partition is in the inactive state, communicating with the target partition to confirm that the target partition is the active partition and service the request from the data consumer using the target partition;

if the pointer indicates that the target partition is the active data source, communicating with the target partition to read target partition metadata, wherein the target partition metadata indicates whether the target partition is in the active state or the inactive state;

if the target partition metadata of the target partition indicates that the target partition is in the active state, confirming that the target partition is the active partition and servicing the request from the data consumer using the target partition; and

if the target partition metadata of the target partition indicates that the target partition is in the inactive state, communicating with the source partition to confirm that the source partition is the active partition and servicing the request from the data consumer using the source partition.

7. The system of claim 6, wherein the server is a front end server of a distributed computing environment.

8. The system of claim 7, wherein the front end server is the data consumer.

9. The system of claim 6, wherein the server is a web server and includes a presentation layer, a business logic layer, and a data layer containing the data access layer.

10. The system of claim 6, wherein the data access layer includes the data locator service as an integrated data locator service.

11. The system of claim 6, wherein the data access layer further caches partition metadata such that the cache is first read before communicating with the source partition or the target partition to read or confirm the corresponding source partition metadata or target partition metadata.

12. The system of claim 11, wherein the partition metadata is cached for a variable length of time depending on what state of migration a current partition is in, the state of migration comprising a normal operation state and a migration state.