SEAMLESS DATA MIGRATION ACROSS DATABASES

• US 20170357645A1
• Filed: 06/27/2017
• Published: 12/14/2017
• Est. Priority Date: 03/04/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: KRITCHKO, VLADIMIR, HE, XIN, GREBENIK, VLADIMIR V., YAN, KANGRONG
• Granted Patent: US 10,831,779 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation

Abstract

Techniques are disclosed for enabling the migration of data with minimized impact on consumers of the data. A data migration agent updates pointers to active data locations and coordinates a migration from a first data resource to a second data resource so that seamless migration may be carried out. A data access layer of a distributed computing environment can take advantage of the pointers and metadata written by the data migration agent to serve requests to resources of the distributed computing environment so that read availability is available regardless of migration status.

Claims

1. A method of facilitating data requests from consumers to resources in a distributed computing environment, comprising:

receiving a request for interacting with a database from a data consumer; and

communicating the request to a data storage resource storing the database, the communicating carried out regardless of migration state of the database.

2. The method of claim 1, wherein communicating the request to the data storage resource storing the database comprises:

communicating with a data locator to retrieve an indication of an active one or more partitions of the database to serve the request;

reading metadata associated with the one or more partitions to confirm that the one or more partitions indicate an active state; and

in response to the active state of the one or more partitions being confirmed, communicating the request to the database by performing the request on the one or more partitions;

in response to the active state of the identified active resource being not confirmed, communicating the request to the database by performing the request on a target resource to which at least one of the one or more partitions is being migrated.

3. The method of claim 1, wherein communicating the request to the data storage resource storing the database comprises:

communicating with a data locator to retrieve an indication of an active one or more partitions of the database to serve the request;

reading metadata associated with the one or more partitions to confirm that the one or more partitions indicate an active state; and

in response to the active state of the one or more partitions being confirmed, communicating the request to the database by performing the request on the one or more partitions;

in response to the active state of the identified active resource being not confirmed, communicating the request to the database by performing the request on a source resource from which at least one of the one or more partitions is being migrated.

4. The method of claim 1, wherein the request for interacting with the database is a read request, wherein communicating the request to the data storage resource storing the database comprises:

locating one or more active partitions of the database;

mapping the request to the one or more active partitions; and

retrieving data from the one or more active partitions.

5. The method of claim 4, wherein locating the one or more active partitions of the database comprises:

querying a data locator for an active partition, the data locator providing a directed partition;

querying metadata on the directed partition to for the active partition;

determining whether or not the data locator and the directed partition indicate a same active partition; and

in response to determining that the data locator and the directed partition indicate the same active partition;

returning the active partition;

in response to determining that the data locator and the directed partition do not indicate the same active partition;

querying metadata on a corresponding partition on a remaining database for the active partition;

selecting the active partition directed by a majority of the data locator, directed partition and the corresponding partition; and

returning the active partition.

6. The method of claim 1, wherein communicating the request to the data storage resource storing the database comprises:

communicating with a data locator service to obtain a location of an active partition according to a pointer indicating an active data source, the pointer indicating whether a source partition or a target partition is the active data source;

if the pointer indicates that the source partition is the active data source, communicating with the source partition to read source partition metadata, wherein the source partition metadata indicates whether the source partition is in an active state or an inactive state;

if the source partition metadata of the source partition indicates that the source partition is in the active state, confirming that the source partition is the active partition and servicing the request from the data consumer using the source partition;

if the source partition metadata of the source partition indicates that the source partition is in the inactive state, communicating with the target partition to confirm that the target partition is the active partition and service the request from the data consumer using the target partition, wherein the target partition metadata indicates whether the target partition is in the active state or the inactive state;

if the pointer indicates that the target partition is the active data source, communicating with the target partition to read the target partition metadata;

if the target partition metadata of the target partition indicates that the target partition is in the active state, confirming that the target partition is the active partition and servicing the request from the data consumer using the target partition;

if the target partition metadata of the target partition indicates that the target partition is in the inactive state, communicating with the source partition to confirm that the source partition is the active partition and servicing the request from the data consumer using the source partition.

7. The method of claim 1, wherein the request for interacting with the database from the data consumer comprises a request for a cached data object, a value from a system-controlled identity field being appended to a user-controlled attribute in a target;

wherein communicating the request to the data storage resource storing the database comprises;

requesting, from a data locator, a location of an active partition for the cached data object; and

identifying and matching an identical data object on either a source by using the system-controlled identity value or the target by using the user-controlled identity value.

8. A system for facilitating data requests from consumers to resources, the system comprising:

a server comprising a data access layer, the data access layer receiving a request for interacting with a database from a data consumer and communicating the request to a data storage resource storing the database, the communication carried out regardless of migration state of the database.

9. The system of claim 8, wherein the server is a front end server of a distributed computing environment.

10. The system of claim 9, wherein the front end server is the data consumer.

11. The system of claim 8, wherein the server is a web server and includes a presentation layer, a business logic layer, and a data layer containing the data access layer.

12. The system of claim 8, further comprising a data locator service, wherein the data access layer communicates with the data locator service to retrieve an indication of an active one or more partitions of the database to serve the request.

13. The system of claim 12, wherein the data access layer includes the data locator service as an integrated data locator service.

14. The system of claim 8, wherein the communicating of the request by the data access layer to the data storage resource storing the database comprises:

communicating with a data locator service to obtain a location of an active partition according to a pointer indicating an active data source, the pointer indicating whether a source partition or a target partition is the active data source;

if the pointer indicates that the source partition is the active data source, communicating with the source partition to read source partition metadata, wherein the source partition metadata indicates whether the source partition is in an active state or an inactive state;

if the source partition metadata of the source partition indicates that the source partition is in the active state, confirming that the source partition is the active partition and servicing the request from the data consumer using the source partition;

if the source partition metadata of the source partition indicates that the source partition is in the inactive state, communicating with the target partition to confirm that the target partition is the active partition and service the request from the data consumer using the target partition, wherein the target partition metadata indicates whether the target partition is in the active state or the inactive state;

if the pointer indicates that the target partition is the active data source, communicating with the target partition to read the target partition metadata;

if the target partition metadata of the target partition indicates that the target partition is in the active state, confirming that the target partition is the active partition and servicing the request from the data consumer using the target partition;

if the target partition metadata of the target partition indicates that the target partition is in the inactive state, communicating with the source partition to confirm that the source partition is the active partition and servicing the request from the data consumer using the source partition.

15. One or more machine-readable storage media storing instructions for a data access layer thereon that when executed by a processing system of a server, direct the processing system to at least:

in response to receiving a request from a consumer to access data, communicate with a data locator service to obtain a location of an active partition according to a pointer indicating an active data source, the pointer indicating whether a source partition or a target partition is the active data source;

if the pointer indicates that the source partition is the active data source, communicate with the source partition to read source partition metadata, wherein the source partition metadata indicates whether the source partition is in an active state or an inactive state;

if the source partition metadata of the source partition indicates that the source partition is in the active state, confirm that the source partition is the active partition and service the request from the consumer using the source partition;

if the source partition metadata of the source partition indicates that the source partition is in the inactive state, communicate with the target partition to confirm that the target partition is the active partition and service the request from the consumer using the target partition, wherein the target partition metadata indicates whether the target partition is in the active state or the inactive state;

if the pointer indicates that the target partition is the active data source, communicate with the target partition to read the target partition metadata;

if the target partition metadata of the target partition indicates that the target partition is in the active state, confirm that the target partition is the active partition and service the request from the consumer using the target partition;

if the target partition metadata of the target partition indicates that the target partition is in the inactive state, communicate with the source partition to confirm that the source partition is the active partition and service the request from the consumer using the source partition.

16. The media of claim 15, wherein during migration of data from the source partition to the target partition, the source partition is read-access available,

wherein during the migration of the data from the source partition to the target partition, the source partition is also write-access available except during a time period when deltas are being transferred from the source partition to the target partition, wherein during the time period when deltas are being transferred from the source partition to the target partition, the source partition remains read-access available.

17. The media of claim 15, wherein the pointer indicates the source partition as the active partition during migration of a replica from the source partition to the target partition, wherein the pointer indicates the target partition as the active partition after a source delta is applied to the target partition.

18. The media of claim 17, wherein the source partition metadata changes from the active state to the inactive state and the target partition metadata changes from the inactive state to the active state after the source delta is applied to the target position.

19. The media of claim 15, wherein the instructions for the data access layer thereon further direct the processing system to

cache partition metadata such that the cache is first read before communicating with the source partition or the target partition to read or confirm the corresponding source partition metadata or target partition metadata.

20. The media of claim 19, wherein the partition metadata is cached for a variable length of time depending on what state of migration a current partition is in, the state of migration comprising a normal operation state and a migration state.