Efficient live-migration of remotely accessed data

• US 10,187,466 B2
• Filed: 02/22/2018
• Issued: 01/22/2019
• Est. Priority Date: 03/16/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Sanderson, Tyler
• Primary Examiner(s): Barot, Bharat
• Application Number:
  US15/902,844
• Publication Number:
  US 20180183869A1
• Time in Patent Office: 334 Days
• Field of Search: 709201-203, 709208, 709213-216
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for migrating data from a first storage system to a second storage system. That data is for a process (e.g., a virtual machine, application, or some other process) operating on a computer system that is separate from the first storage system and the second storage system. That data is stored according to data subsets that are each exclusive of each other. As each data set is migrated, access to the data set by the process is precluded, but access for the remaining data sets is not affected. Once the data migration is complete, access to the data set by the process is restored.

Claims

1. A method performed by a data processing system, the method comprising:

storing a mapping for a data processing process that processes data stored in a first physical machine, the data stored according to a plurality of data subsets that are each exclusive of each other;

for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;

enabling read and write access to the data for the data processing process according to the logical addressing;

in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets; and

in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.

2. The method of claim 1, wherein the data processing process is executed on a third physical machine separate from the first physical machine and the second physical machine.

3. The method of claim 2, wherein:

the determination that the first data subset stored on the first physical machine is to be migrated to the second physical machine is performed by the first physical machine; and

blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets comprises blocking, by the first physical machine, read and write access for the data processing process to the first data subset.

4. The method of claim 3, further comprising:

receiving, from the first physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and

wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to the data processing process receiving the notification provided by the first physical machine.

5. The method of claim 4, wherein the notification instructs the data processing process to not access the first data subset.

6. The method of claim 4, wherein the first data subset is sent from the first physical machine directly to the second physical machine during the migration.

7. The method of claim 3, further comprising:

receiving, from the second physical machine by the data processing process, a notification of the migration of the first data subset to the second physical machine; and

wherein updating the mapping by logically mapping the address range for the first data subset on the second physical machine to the respective logical address range for the data processing process is done in response to receiving the notification by the data processing process.

8. The method of claim 2, further comprising determining, by the data processing process, that the first data subset stored on the first physical machine is to be migrated to the second physical machine;

and

migrating, from the first physical machine to the second physical machine, the first data subset to store the data subset on the second physical machine comprises;

instructing, by the data processing process, the first physical machine to send the first data subset to the data processing process; and

sending, by the data processing process, the first data subset received from the first physical machine to the second physical machine;

the updating the mapping is done by the data processing process.

9. The method of claim 2, further comprising restoring read and write access to the first data subset for the data processing process after the migration of the first data subset from the first physical machine to the second physical machine.

10. The method of claim 1, wherein the data subsets are data chunks logically mapped to a physical address range.

11. The method of claim 1, the data subsets are data blocks.

12. A system, comprising:

a data processing device; and

a non-transitory computer-readable memory in data communication with the data processing device and storing instructions executable by the data processing device and that upon such execution cause the data processing device to perform operations comprising;

storing a mapping for a data processing process that processes data stored in a first physical machine, the data stored according to a plurality of data subsets that are each exclusive of each other;

for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;

enabling read and write access to the data by the data processing process according to the logical addressing;

in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets; and

in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.

13. The system of claim 12, wherein the data processing device is a physical machine separate from the first physical machine and the second physical machine.

14. The system of claim 12, wherein the data processing process is executed on the data processing device.

15. The system of claim 12, wherein the data processing process is executed on a third physical machine separate from the data processing device, the first physical machine, and the second physical machine.

16. A non-transitory computer-readable memory in data communication with the data processing device and storing instructions executable by the data processing device and that upon such execution cause the data processing device to perform operations comprising:

storing a mapping for a data processing process that processes data stored in a first physical machine, the data stored according to a plurality of data subsets that are each exclusive of each other;

for each data subset, logically mapping in the mapping an address range for the data subset on the first physical machine to a respective logical address range for the data processing process;

enabling read and write access to the data by the data processing process according to the logical addressing;

in response to a determination that a first data subset stored on the first physical machine is to be migrated to a second physical machine separate from the first physical machine, blocking read and write access for the data processing process to the first data subset while maintaining read and write access for the data processing process to the other data subsets; and

in response to a migration of the first data subset from the first physical machine to the second physical machine to store the data subset on the second physical machine, updating the mapping by logically mapping an address range for the first data subset on the second physical machine to the respective logical address range for the data processing process.