Virtual machine state replication using DMA write records

• US 10,198,377 B2
• Filed: 06/03/2016
• Issued: 02/05/2019
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wilson, Matthew Shawn, Liguori, Anthony Nicholas, Ganguly, Shuvabrata
• Primary Examiner(s): Martinez, David E
• Application Number:
  US15/173,445
• Publication Number:
  US 20160283421A1
• Time in Patent Office: 977 Days
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues

Abstract

A DMA-capable device of a virtualization host stores a DMA write record, indicating a portion of host memory that is targeted by a DMA write operation, in a write buffer accessible from a virtualization management component of the host. The virtualization management component uses the DMA write record to identify a portion of memory to be copied to a target location to save a representation of a state of a particular virtual machine instantiated at the host.

Claims

1. A DMA-capable hardware device configured to:

determine that a particular DMA write operation directed to a portion of memory device assigned to a virtual machine instantiated at a virtualization host is to be scheduled;

store, in a particular slot of a record buffer accessible from a virtualization management component of the virtualization host, a first DMA write record indicating the portion of memory to which the particular DMA write operation is directed; and

,

subsequent to a determination that the virtualization management component has examined the particular slot, store a different DMA write record indicative of a different DMA write operation in the particular slot.

2. The DMA-capable hardware device as recited in claim 1, wherein the DMA-capable hardware device is attached to a communication bus of the virtualization host.

3. The DMA-capable hardware devices as recited in claim 1, wherein the DMA-capable hardware device is attached to a particular interconnect of the virtualization host, wherein the particular interconnect is compliant with a version of the PCI (Peripheral Component Interconnect) or PCIe (Peripheral Component Interconnect Express) standard.

4. The DMA-capable hardware device as recited in claim 1, wherein the DMA-capable hardware device is configured to:

offload a networking-related I/O processing operation from one or more processors of the virtualization host.

5. The DMA-capable hardware device as recited in claim 1, wherein the DMA-capable hardware device is configured to:

offload a storage-related I/O processing operation from one or more processors of the virtualization host.

6. The DMA-capable hardware device as recited in claim 1, wherein the DMA-capable hardware device comprises a graphical processing unit (GPU).

7. The DMA-capable hardware device as recited in claim 1, wherein the DMA-capable hardware device is compliant with a Single Root I/O Virtualization (SR-IOV) specification of the PCI Special Interest Group (PCI SIG).

8. A method, comprising:

performing, by a DMA-capable hardware device communicatively coupled to a memory;

determining that a particular DMA write operation directed to a portion of the memory assigned to a virtual machine instantiated at a virtualization host is to be performed by the DMA-capable hardware device;

generating a first DMA write record indicating the portion of memory to which the particular DMA write operation is directed; and

storing, in a particular slot of a record buffer accessible from a virtualization management component of the virtualization host, the first DMA write record.

9. The method as recited in claim 8, further comprising performing, by the DMA-capable hardware device:

subsequent to determining that the virtualization management component has examined the particular slot, storing a different DMA write record indicative of a different DMA write operation in the particular slot.

10. The method as recited in claim 9, wherein said determining that the virtualization management component has examined the particular slot comprises analyzing a read-completed flag updated by the virtualization management component.

11. The method as recited in claim 8, further comprising performing, by the DMA-capable hardware device:

receiving, prior to said storing, a command from the virtualization management component indicating that the first DMA write record is to be generated.

12. The method as recited in claim 11, wherein the virtualization management component comprises one or more of:

(a) a hypervisor or (b) an administrative instance of an operating system.

13. The method as recited in claim 8, wherein the record buffer is incorporated within a physical memory of the virtualization host.

14. The method as recited in claim 8, wherein the record buffer is incorporated within a local memory of the DMA-capable device.

15. The method as recited in claim 8, further comprising performing, by the DMA-capable hardware device:

receiving, after storing the first DMA write record in the particular slot, an indication that respective DMA write records associated one or more additional DMA write operations are not to be generated; and

initiating an additional DMA write operation without generating a corresponding DMA write record.

16. The method as recited in claim 8, wherein the record buffer comprises one or more slots including the particular slot, further comprising performing, by the DMA-capable hardware device:

determining, after storing the first DMA write record in the particular slot, that none of the slots of the one or more slots of the record buffer is available for storing another DMA write record corresponding to an additional DMA write operation; and

deferring the additional DMA write operation.

17. A non-transitory computer-accessible storage medium storing program instructions that when executed at a DMA-capable hardware:

determine that a particular DMA write operation directed to a portion of memory device communicatively coupled to the DMA-capable hardware device and assigned to a virtual machine instantiated at a virtualization host is to be performed by the DMA-capable hardware device;

generate a first DMA write record indicating the portion of memory to which the particular DMA write operation is directed; and

store, in a particular slot of a record buffer accessible from a virtualization management component of the virtualization host, the first DMA write record.

18. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the program instructions when executed at the DMA-capable hardware device:

subsequent to determining that the virtualization management component has examined the particular slot, store a different DMA write record indicative of a different DMA write operation in the particular slot.

19. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the program instructions when executed at a DMA-capable hardware device:

determine, based at least in part on a command generated by the virtualization management component, that the first DMA write record is to be generated.

20. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the program instructions when executed at a DMA-capable hardware device:

determine, after storing the first DMA write record in the particular slot, that the record buffer does not comprise a free slot for a DMA write record corresponding to an additional DMA write operation; and

defer the additional DMA write operation.