Automatically generating volume images and launching virtual computing instances

• US 9,430,212 B2
• Filed: 02/10/2014
• Issued: 08/30/2016
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: Netflix, Inc.
• Original Assignee: Netflix, Inc.
• Inventors: Tripoli, Michael, Moyles, Brian
• Primary Examiner(s): Dao, Thuy
• Assistant Examiner(s): Nguyen, Mongbao
• Application Number:
  US14/177,182
• Publication Number:
  US 20150227355A1
• Time in Patent Office: 932 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 8/63 Image based installation; Cloning; Build to order
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Abstract

In an embodiment, a method comprises using a first server computer, executing a process that is configured to generate virtual computer instances; receiving a request to generate a first custom virtual computer instance, wherein the request identifies one or more software packages to install on the first custom virtual computer instance; copying a first pre-configured volume image onto a volume of a data storage device, wherein the volume includes one or more base software packages in a file system; mounting the volume to the first server computer, in response to receiving the request; installing each of the one or more software packages on the volume; unmounting the volume; creating a snapshot of the volume; registering the snapshot as a registered custom volume image.

Claims

1. A method comprising:

using a first server computer, executing a process that is configured to generate virtual computer instances;

receiving a request to generate a first custom virtual computer instance, wherein the request identifies one or more software packages to install on the first custom virtual computer instance;

copying a first pre-configured volume image onto a volume of a data storage device, wherein the volume includes one or more base software packages in a file system;

mounting the volume to the first server computer, in response to receiving the request;

determining a particular operating system is installed on the volume;

selecting, from a plurality of plugins, a particular plugin associated with the particular operating system, wherein each plugin is associated with an operating system of a plurality of operating systems;

causing the particular plugin, in accordance with the particular operating system, to;

change a first root directory on the first server computer to a second root directory on the volume; and

install each of the one or more software packages on the volume relative to the second root directory on the volume;

unmounting the volume;

creating a snapshot of the volume;

registering the snapshot as a registered custom volume image;

wherein the process is performed using one or more computing devices.

2. The method of claim 1 comprising:

launching the first custom virtual computer instance from the volume;

communicating data indicating that the first custom virtual computer instance is instantiated;

receiving, at the process, a second request to generate a second custom virtual computer instance;

determining that the second request identifies the one or more software packages installed on the registered custom volume image;

in response to determining the one or more software packages are installed on the registered custom volume image;

copying the registered custom volume image to a second volume;

launching the second custom virtual computer instance from the second volume, without mounting the second volume onto the first server computer, without installing the one or more software packages, and without unmounting the second volume from the first server computer.

3. The method of claim 1 comprising:

receiving a second request to generate a second custom virtual computer instance and identifying the one or more software packages installed on the first custom virtual computer instance;

in response to receiving the second request;

copying the first pre-configured volume image to a second volume;

mounting the second volume that is a copy of the first pre-configured volume image to the first server computer, wherein the second volume includes the one or more base software packages;

installing each of the one or more software packages on the second volume;

unmounting the second volume;

launching the second custom virtual computer instance from the second volume;

communicating data indicating that the second custom virtual computer instance is instantiated.

4. The method of claim 1, wherein the request identifies at least a base software package and the one or more base software packages, and the method comprises:

maintaining a pool of pre-configured volume images comprising the first pre-configured volume image, wherein each pre-configured volume image of the pre-configured volume images includes a unique set of base software packages that are installed;

determining that the one or more base software packages are installed on the first pre-configured volume image;

selecting the first pre-configured volume image to copy to the volume based, at least in part, on determining that the one or more base software packages are installed on the first pre-configured volume image.

5. The method of claim 1, comprising:

receiving a commit request, at a source code version control server computer, to commit a set of source code to a version control system;

committing the set of source code to the version control system;

determining that the set of source code was committed successfully, and in response, compiling the set of source code and generating a compiled software package;

sending the request to the process to generate the first custom virtual computer instance;

wherein the request identifies the one or more software packages that should be installed on the first custom virtual computer instance and includes the compiled software package;

installing the compiled software package;

running a set of tests against the compiled software package on the first custom virtual computer instance;

generating a notification specifying which tests failed, if any.

6. The method of claim 1, wherein the process is a first long-running process of a plurality of long-running processes and each long-running process of the plurality of long-running processes is being executed on a different server computer in a particular geographic region and is capable of launching one or more custom virtual computer instances at the particular geographic region, the request identifies a geographic region, and the method comprises:

executing a master process at a master server computer;

receiving the request at the master process;

determining that the first server computer is located in the geographic region identified in the request;

sending the request to the first long-running process, executed by the first server computer, to instantiate the first custom virtual computer instance at the geographic region.

7. The method of claim 1 comprising communicating data indicating a progress state of the process while the process is copying the first pre-configured volume image onto the volume, mounting the volume, installing the one or more software packages, unmounting the volume, and/or launching the first custom virtual computer instance.

8. One or more non-transitory computer-readable media storing one or more sequences of instructions which, when executed by one or more computing devices, cause:

using a first server computer, executing a process that is configured to generate virtual computer instances;

receiving a request to generate a first custom virtual computer instance, wherein the request identifies one or more software packages to install on the first custom virtual computer instance;

copying a first pre-configured volume image onto a volume of a data storage device, wherein the volume includes one or more base software packages in a file system;

mounting the volume to the first server computer, in response to receiving the request;

determining a particular operating system is installed on the volume;

selecting, from a plurality of plugins, a particular plugin associated with the particular operating system, wherein each plugin is associated with an operating system of a plurality of operating systems;

causing the particular plugin, in accordance with the particular operating system, to;

change a first root directory on the first server computer to a second root directory on the volume; and

install each of the one or more software packages on the volume relative to the second root directory on the volume;

unmounting the volume;

creating a snapshot of the volume;

registering the snapshot as a registered custom volume image.

9. The one or more non-transitory computer-readable media of claim 8 comprising sequences of instructions which when executed cause:

launching the first custom virtual computer instance from the volume;

communicating data indicating that the first custom virtual computer instance is instantiated;

receiving, at the process, a second request to generate a second custom virtual computer instance;

determining that the second request identifies the one or more software packages installed on the registered custom volume image;

in response to determining the one or more software packages are installed on the registered custom volume image;

copying the registered custom volume image to a second volume;

launching the second custom virtual computer instance from the second volume, without mounting the second volume onto the first server computer, without installing the one or more software packages, and without unmounting the second volume from the first server computer.

10. The one or more non-transitory computer-readable media of claim 8 comprising sequences of instructions which when executed cause:

receiving a second request to generate a second custom virtual computer instance and identifying the one or more software packages installed on the first custom virtual computer instance;

in response to receiving the second request;

copying the first pre-configured volume image to a second volume;

mounting the second volume that is a copy of the first pre-configured volume image to the first server computer, wherein the second volume includes the one or more base software packages;

installing each of the one or more software packages on the second volume;

unmounting the second volume;

launching the second custom virtual computer instance from the second volume;

communicating data indicating that the second custom virtual computer instance is instantiated.

11. The one or more non-transitory computer-readable media of claim 8, wherein the request identifies at least a base software package and the one or more base software packages, and comprising sequences of instructions which when executed cause:

maintaining a pool of pre-configured volume images comprising the first pre-configured volume image, wherein each pre-configured volume image of the pre-configured volume images includes a unique set of base software packages that are installed;

determining that the one or more base software packages are installed on the first pre-configured volume image;

selecting the first pre-configured volume image to copy to the volume based, at least in part, on determining that the one or more base software packages are installed on the first pre-configured volume image.

12. The one or more non-transitory computer-readable media of claim 8, comprising sequences of instructions which when executed cause:

receiving a commit request, at a source code version control server computer, to commit a set of source code to a version control system;

committing the set of source code to the version control system;

determining that the set of source code was committed successfully, and in response, compiling the set of source code and generating a compiled software package;

sending the request to the process to generate the first custom virtual computer instance;

wherein the request identifies the one or more software packages that should be installed on the first custom virtual computer instance and includes the compiled software package;

installing the compiled software package;

running a set of tests against the compiled software package on the first custom virtual computer instance;

generating a notification specifying which tests failed, if any.

13. The one or more non-transitory computer-readable media of claim 8, wherein the process is a first long-running process of a plurality of long-running processes and each long-running process of the plurality of long-running processes is being executed on a different server computer in a particular geographic region and is capable of launching one or more custom virtual computer instances at the particular geographic region, the request identifies a geographic region, and comprising sequences of instructions which when executed cause:

executing a master process at a master server computer;

receiving the request at the master process;

determining that the first server computer is located in the geographic region identified in the request;

sending the request to the first long-running process, executed by the first server computer, to instantiate the first custom virtual computer instance at the geographic region.

14. The one or more non-transitory computer-readable media of claim 8 comprising sequences of instructions which when executed cause communicating data indicating a progress state of the process while the process is copying the first pre-configured volume image onto the volume, mounting the volume, installing the one or more software packages, unmounting the volume, and/or launching the first custom virtual computer instance.

15. A system comprising a first server computer with a memory, a processor, a process for generating virtual computer instances, wherein the process is executed by the processor and configured to:

receive a request to generate a first custom virtual computer instance, wherein the request identifies one or more software packages to install on the first custom virtual computer instance;

copy a first pre-configured volume image onto a volume of a data storage device, wherein the volume includes one or more base software packages in a file system;

mount the volume to the first server computer, in response to receiving the request;

determine a particular operating system is installed on the volume;

select, from a plurality of plugins, a particular plugin associated with the particular operating system, wherein each plugin is associated with an operating system of a plurality of operating systems;

cause the particular plugin, in accordance with the particular operating system, to;

change a first root directory on the first server computer to a second root directory on the volume; and

install each of the one or more software packages on the volume relative to the second root directory on the volume;

unmount the volume;

create a snapshot of the volume;

register the snapshot as a registered custom volume image.

16. The system of claim 15, wherein the process is further configured to:

launch the first custom virtual computer instance from the volume;

communicate data indicating that the first custom virtual computer instance is instantiated;

receive, at the process, a second request to generate a second custom virtual computer instance;

determine that the second request identifies the one or more software packages installed on the registered custom volume image, and in response;

copy the registered custom volume image to a second volume;

launch the second custom virtual computer instance from the second volume, without mounting the second volume onto the first server computer, without installing the one or more software packages, and without unmounting the second volume from the first server computer.

17. The system of claim 15, wherein the process is further configured to:

receive a second request to generate a second custom virtual computer instance and identifying the one or more software packages installed on the first custom virtual computer instance, and in response;

copy the first pre-configured volume image to a second volume;

mount the second volume that is a copy of the first pre-configured volume image to the first server computer, wherein the second volume includes the one or more base software packages;

install each of the one or more software packages on the second volume;

unmount the second volume;

launch the second custom virtual computer instance from the second volume;

communicate data indicating that the second custom virtual computer instance is instantiated.

18. The system of claim 15, wherein the request identifies at least a base software package and the one or more base software packages, and the process is further configured to:

maintain a pool of pre-configured volume images comprising the first pre-configured volume image, wherein each pre-configured volume image of the pre-configured volume images includes a unique set of base software packages that are installed;

determine that the one or more base software packages are installed on the first pre-configured volume image;

select the first pre-configured volume image to copy to the volume based, at least in part, on determining that the one or more base software packages are installed on the first pre-configured volume image.

19. The system of claim 15 comprising:

a second server computer with a second memory and a second processor;

a repository process executed by the second processor;

wherein the repository process configured to;

receive a commit request, at a source code version control server computer, to commit a set of source code to a version control system;

commit the set of source code to the version control system;

determine that the set of source code was committed successfully, and in response, compiling the set of source code and generating a compiled software package;

send the request to the process to generate the first custom virtual computer instance, wherein the request identifies the one or more software packages that should be installed on the first custom virtual computer instance and includes the compiled software package;

wherein the process is further configured to;

install the compiled software package;

run a set of tests against the compiled software package on the first custom virtual computer instance;

generate a notification specifying which tests failed, if any.

20. The system of claim 15, wherein the process is a first long-running process of a plurality of long-running processes and each long-running process of the plurality of long-running processes is being executed on a different server computer in a particular geographic region and is capable of launching one or more custom virtual computer instances at the particular geographic region, the request identifies a geographic region, and the system comprises:

a master server computer with a third memory and a third processor;

a master process executed by the third processor and configured to;

receive the request;

determine that the first server computer is located in the geographic region identified in the request;

send the request to the first long-running process, executed by the first server computer, to instantiate the first custom virtual computer instance at the geographic region.

21. The system of claim 15, wherein the process is further configured to communicate data indicating a progress state of the process while the process is copying the first pre-configured volume image onto the volume, mount the volume, install the one or more software packages, unmount the volume, and/or launch the first custom virtual computer instance.