Facilitating the recovery of a virtual machine using a distributed filesystem

• US 9,613,064 B1
• Filed: 03/01/2013
• Issued: 04/04/2017
• Est. Priority Date: 05/03/2010
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Chou, Randy Yen-pang, Taylor, John Richard, Davis, Andrew P.
• Primary Examiner(s): Bates, Kevin
• Assistant Examiner(s): RAHMAN, SM AZIZUR
• Application Number:
  US13/782,729
• Time in Patent Office: 1,495 Days
• Field of Search: 709219
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases

Abstract

The disclosed embodiments disclose techniques that facilitate the recovery of a virtual machine using a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a host server executes program instructions for an application in a virtual machine (VM); data associated with this application and/or this virtual machine is stored in the distributed filesystem. Upon detecting a subsequent failure, the system can recover and resume the execution of the virtual machine and application using the previous application and virtual machine data that was stored in the distributed filesystem.

Claims

1. A computer-implemented method for facilitating the recovery of a virtual machine using a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in a cloud storage system, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage system, wherein each cloud controller is associated with a local storage pool; and

caching data for the distributed filesystem in the local storage pool of each cloud controller;

executing program instructions for an application in the virtual machine, wherein the virtual machine receives the program instructions for the application from the distributed filesystem via a cloud controller;

storing data associated with the application and the virtual machine in the distributed filesystem via the cloud controller, wherein metadata updates for the application and the virtual machine are distributed to the two or more cloud controllers managing the distributed filesystem using incremental metadata updates and data updates for the application and the virtual machine are synchronized to the cloud storage system via incremental data updates, wherein synchronizing data updates to the cloud storage system facilitates accessing the data of the application and virtual machine and restarting the application and virtual machine via any cloud controller that can access the distributed filesystem;

detecting a failure for the application and the virtual machine; and

recovering the application by seamlessly resuming the execution of the application using the data stored in the distributed filesystem.

2. The computer-implemented method of claim 1,

wherein the virtual machine executes on a host server;

wherein the cloud controller exports a filesystem mount point for the distributed filesystem that exposes the data stored in the distributed filesystem to the host server; and

wherein the application and the virtual machine transparently access the distributed filesystem via the cloud controller and the filesystem mount point.

3. The computer-implemented method of claim 2, wherein temporary files and swap space associated with the application and the virtual machine are written to the distributed filesystem, wherein the host server fails, wherein recovering the application further comprises:

re-starting the virtual machine on a second host server;

re-starting the application on the restarted virtual machine; and

accessing the temporary files and swap space from the cache of the cloud controller to ensure that application re-starts to the same previous executing state as on the failed host server;

wherein subsequent data associated with the application continues to be written to the distributed filesystem to ensure that the application can subsequently be re-started on other host servers as needed.

4. The computer-implemented method of claim 2, wherein temporary files and swap space associated with the application and the virtual machine are written to the distributed filesystem, wherein the host server and the cloud controller fail, wherein recovering the application further comprises:

determining a second cloud controller for the distributed filesystem that will support the application;

re-starting the virtual machine on a second host server that is located in geographic proximity to the second cloud controller;

re-starting the application on the restarted virtual machine; and

accessing the temporary files and swap space from the second cloud controller to ensure that application re-starts to the same previous executing state as on the failed host server, wherein the second cloud controller loads the temporary files and swap space from the cloud storage system;

wherein subsequent data associated with the application continues to be written to the distributed filesystem to ensure that the application can subsequently be re-started on other host servers as needed.

5. The computer-implemented method of claim 3, wherein the cloud controller comprises a dedicated computing device that collaborates with additional cloud controllers to manage and provide access to the distributed filesystem.

6. The computer-implemented method of claim 3,

wherein the cloud controller comprises a cloud controller application that executes in a second virtual machine on the host server;

wherein the cloud controller application comprises cloud controller functionality that collaborates with additional cloud controllers to manage and provide access to the distributed filesystem; and

wherein the host server allocates a portion of a local storage device to serve as the local storage pool for the cloud controller.

7. The computer-implemented method of claim 6, wherein the method further comprises:

receiving in the cloud controller via the filesystem mount point one or more new data blocks to be written to the distributed filesystem;

caching the new data blocks in the local storage pool;

sending an incremental data snapshot that includes the new data blocks to a cloud storage system; and

distributing an incremental metadata snapshot for the new data blocks to the other cloud controllers of the distributed filesystem.

8. The computer-implemented method of claim 7,

wherein one or more other applications executing in the host server can only access distributed filesystem data that is cached in the storage pool via the cloud controller and the exported filesystem mount point;

and

wherein the one or more applications can only access the distributed filesystem when the cloud controller application and the second virtual machine are executing on the host server.

9. The computer-implemented method of claim 8, wherein executing the program instructions for the application further comprises:

storing the program instructions for the application in the distributed filesystem;

loading the program instructions for the application from the distributed filesystem; and

storing temporary data, swap space, application data, and virtual machine data that are generated while executing the program instructions for the application in the distributed filesystem.

10. The computer-implemented method of claim 9, wherein the method further comprises using a locality policy to pin the program instructions for the application in the local storage pool of one or more cloud controllers to ensure that the application can be quickly re-started in case of failure.

11. The computer-implemented method of claim 9, wherein the method further comprises:

using a locality policy to identify a set of data that is likely to be used by the application; and

pre-loading the identified set of data into the local storage pool, thereby reducing an access latency for the application.

12. The computer-implemented method of claim 9, wherein the method further comprises:

wherein cloud controllers are configured to own and manage write locks for distinct portions of the distributed filesystem, wherein the virtual machine attempts to write a portion of the distributed filesystem that is owned by a remote cloud controller, wherein writing the portion of the distributed filesystem comprises contacting the remote cloud controller to request a write lock;

migrating a write lock for the portion of the distributed filesystem from the remote cloud controller to the cloud controller, thereby reducing a write access latency for the application; and

ensuring that the write lock cannot be requested by and moved to any other cloud controllers for a given time interval while the virtual machine is executing.

13. The computer-implemented method of claim 9, wherein a network connection between the cloud controller and the cloud storage system has limited bandwidth, wherein the method further comprises:

ensuring that all of the data associated with the virtual machine and the application remains cached on the cloud controller to facilitate rapidly re-starting the application without impacting other operations that use the network connection;

classifying a portion of the data associated with the application and the virtual machine as low priority; and

delaying the write-through of the low-priority data to the cloud storage system, thereby reducing network congestion for the network connection during a period of high network activity.

14. The computer-implemented method of claim 9, wherein multiple cloud controller instances executed in different virtual machines on the host server facilitate simultaneous access to multiple, distinct distributed filesystems.

15. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for facilitating the recovery of a virtual machine using a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in a cloud storage system, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage system, wherein each cloud controller is associated with a local storage pool; and

caching data for the distributed filesystem in the local storage pool of each cloud controller;

executing program instructions for an application in the virtual machine, wherein the virtual machine receives the program instructions for the application from the distributed filesystem via a cloud controller;

storing data associated with the application and the virtual machine in the distributed filesystem via the cloud controller, wherein metadata updates for the application and the virtual machine are distributed to the two or more cloud controllers managing the distributed filesystem using incremental metadata updates and data updates for the application and the virtual machine are synchronized to the cloud storage system via incremental data updates, wherein synchronizing data updates to the cloud storage system facilitates accessing the data of the application and virtual machine and restarting the application and virtual machine via any cloud controller that can access the distributed filesystem;

detecting a failure for the application and the virtual machine; and

recovering the application by seamlessly resuming the execution of the application using the data stored in the distributed filesystem.

16. The non-transitory computer-readable storage medium of claim 15, wherein the virtual machine executes on a host server;

wherein the cloud controller exports a filesystem mount point for the distributed filesystem that exposes the data stored in the distributed filesystem to the host server; and

wherein the application and the virtual machine transparently access the distributed filesystem via the cloud controller and the filesystem mount point.

17. The non-transitory computer-readable storage medium of claim 16, wherein the method further comprises:

detecting a failure in at least one of the application, the virtual machine, the cloud controller, and the host server; and

recovering the virtual machine by using data stored in the distributed filesystem to resume the execution of the application on at least one of the host server or a second host server.

18. The non-transitory computer-readable storage medium of claim 17, wherein the cloud controller comprises a dedicated computing device that collaborates with additional cloud controllers to manage and provide access to the distributed filesystem.

19. The non-transitory computer-readable storage medium of claim 17,

wherein the cloud controller comprises a cloud controller application that executes in a second virtual machine on the host server;

wherein the cloud controller application comprises cloud controller functionality that collaborates with additional cloud controllers to manage and provide access to the distributed filesystem; and

wherein the host server allocates a portion of a local storage device to serve as the local storage pool for the cloud controller.

20. A host server that facilitates the recovery of a virtual machine using a distributed filesystem, comprising:

a processor; and

a storage mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein the processor is configured to execute program instructions for an application in the virtual machine; and

wherein the processor is further configured to execute a cloud controller application in a second virtual machine, wherein the cloud controller application comprises cloud controller functionality that collaborates with additional cloud controllers to manage and provide access to the distributed filesystem, wherein the virtual machine receives the program instructions for the application from the distributed filesystem via the cloud controller application;

wherein the host server is configured to allocate a portion of the storage mechanism to serve as a local storage pool for the cloud controller application;

wherein the cloud controller application is configured to cache data for the distributed filesystem in the local storage pool; and

wherein the cloud controller application is further configured to store data associated with the application and the virtual machine in the distributed filesystem, wherein updates for the data for the application and the virtual machine are distributed to the two or more cloud controllers managing the distributed filesystem and a cloud storage system, wherein metadata updates for the application and the virtual machine are distributed to the two or more cloud controllers managing the distributed filesystem using incremental metadata updates and data updates for the application and the virtual machine are synchronized to the cloud storage system via incremental data updates, wherein synchronizing data updates to the cloud storage system facilitates accessing the data of the application and virtual machine and restarting the application and virtual machine via any cloud controller that can access the distributed filesystem, and wherein, upon detecting a failure for the application and the virtual machine, the cloud controller application is configured to recover the application by seamlessly resuming the execution of the application using the data stored in the distributed filesystem.