Analyzing deployment pipelines used to update production computing services using a live pipeline template process

• US 9,787,779 B2
• Filed: 12/21/2015
• Issued: 10/10/2017
• Est. Priority Date: 12/21/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Frank, Martin Robert, Mosher, Ian Aird, Jodoin, Felix Walter Blue, Mansour, Mark Sidney James, Gu, Sixiang
• Primary Examiner(s): BROPHY, MATTHEW J
• Application Number:
  US14/977,192
• Publication Number:
  US 20170180487A1
• Time in Patent Office: 659 Days
• Field of Search: None
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3476 Data logging G06F11/14, G06F11/2205 take precedence
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3696 Methods or tools to render software testable
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: H04L 41/0813 characterised by the conditions triggering a change of settings
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

Techniques are presented for managing a deployment pipeline using an inheritable and extensible source code template—generally referred to as a live pipeline template (LPT). As described, live pipeline templates may be used to manage deployment pipelines which, in turn, are used to launch, maintain, and update the services and systems used to host and provide computing services.

Claims

1. A system, comprising:

a processor; and

a memory storing one or more applications, which when executed on the processor, perform an operation to determine a current configuration state of a deployment pipeline used to propagate updates to a production computing service, the operation comprising;

identifying a plurality of computing services included in the deployment pipeline, for each identified computing service;

determining a current configuration state of a respective one of the computing services included in the deployment pipeline,

generating a description of the current configuration state of the computing service, and

generating, from the generated descriptions, a first application definition, wherein the first application definition provides a fully-specified configuration of the current operational state of the deployment pipeline;

determining, from the first application definition, a service type associated with the production computing service deployed using the deployment pipeline; and

determining, based on at least the determined service type, a live pipeline template to associate with the deployment pipeline, wherein the live pipeline template comprises source code for one or more base pipeline templates.

2. The system of claim 1, wherein determining a current configuration state of the respective computing service included in the deployment pipeline comprises, invoking a pipeline analysis driver corresponding to the respective computing service and wherein the pipeline analysis driver inspects the respective computing service by making one or more API calls to determine the current configuration state of the respective computing service.

3. The system of claim 2, wherein the pipeline analysis drivers includes one or more of a pipeline driver, a deployment driver, a host driver, a network driver, a security driver, an identity and access management (IAM) driver, a performance monitor driver, and an alarm driver.

4. The system of claim 1, wherein the operation further comprises:

determining that the first application definition does not satisfy at least a first rule specifying a requirement for the current configuration state of the deployment pipeline.

5. The system of claim 4, wherein the operation further comprises:

determining a modification to apply to at least a first one of the computing services included in the deployment pipeline needed to satisfy the first rule.

6. The system of claim 5, wherein the operation further comprises:

invoking at least a first synthesis driver to reconfigure the first computing service to conform to the fully-specified configuration of the deployment pipeline provided in the application definition.

7. The system of claim 1, wherein the one or more base pipeline templates include source code of a first base pipeline template which specifies a set of configuration parameters for one or more deployment stages of the deployment pipeline.

8. The system of claim 7, wherein the one or more base pipeline templates include source code of a second base pipeline template which extends the first base pipeline and wherein the second base pipeline template specifies a plurality of service type parameters for the deployment pipeline.

9. The system of claim 7, wherein the operation further comprises:

receiving source code specifying instance specific parameters used to specialize the live pipeline template as a live pipeline template (LPT) instance.

10. The system of claim 9, wherein changes to the deployment pipeline are made by committing, to a version control system, changes to source code of the first base pipeline template, second base pipeline template, or instance specific parameters of the LPT instance.

11. A computer-implemented method for evaluating a deployment pipeline, the method comprising:

generating a first application definition by invoking a plurality of analysis drivers, wherein each of the plurality of analysis drivers inspects a configuration of a respective computing service included in a deployment pipeline, wherein the first application definition reflects a current configuration state of the deployment pipeline;

receiving source code corresponding to a live pipeline template;

generating, from the live pipeline template, a second application definition, wherein the second application definition describes a fully-specified configuration for a plurality of computing services used to provide a second deployment pipeline; and

determining a set of differences between the first application definition and the second application definition; and

generating a report describing the determined set of differences.

12. The method of claim 11, wherein the source code corresponding to the live pipeline includes one or more base pipeline templates.

13. The method of claim 11, wherein the one or more base pipeline templates include a first base pipeline template specifying a set of configuration parameters for one or more deployment stages of the deployment pipeline and a second pipeline template which extends the first base pipeline template with a plurality of instance specific parameters associated with the determined service type.

14. A computer-readable storage medium storing instructions, which, when executed on a processor, perform an operation to determine a current configuration state of a deployment pipeline used to propagate updates to a production computing service, the operation comprising:

identifying a plurality of computing services included in the deployment pipeline;

for each identified computing service;

determining a current configuration state of a respective one of the computing services included in the deployment pipeline,

generating a description of the current configuration state of the computing service, and

generating, from the generated descriptions, a first application definition, wherein the first application definition provides a fully-specified configuration of the current operational state of the deployment pipeline;

determining, from the first application definition, a service type associated with the production computing service deployed using the deployment pipeline; and

determining, based on at least the determined service type, a live pipeline template to associate with the deployment pipeline, wherein the live pipeline template comprises source code for one or more base pipeline templates.

15. The computer-readable storage medium of claim 14, wherein determining a current configuration state of the respective computing service included in the deployment pipeline comprises, invoking a pipeline analysis driver corresponding to the respective computing service and wherein the pipeline analysis driver inspects the respective computing service by making one or more API calls to determine the current configuration state of the respective computing service.

16. The computer-readable storage medium of claim 15, wherein the pipeline analysis drivers includes one or more of a pipeline driver, a deployment driver, a host driver, a network driver, a security driver, an identity and access management (IAM) driver, a performance monitor driver, and an alarm driver.

17. The computer-readable storage medium of claim 14, wherein the operation further comprises:

determining that the first application definition does not satisfy at least a first rule specifying a requirement for the current configuration state of the deployment pipeline.

18. The computer-readable storage medium of claim 17, wherein the operation further comprises:

determining a modification to apply to at least a first one of the computing services included in the deployment pipeline needed to satisfy the first rule.

19. The computer-readable storage medium of claim 18, wherein the operation further comprises:

invoking at least a first synthesis driver to reconfigure the first computing service to conform to the fully-specified configuration of the deployment pipeline provided in the application definition.

20. The computer-readable storage medium of claim 14, wherein the one or more base pipeline templates include source code of a first base pipeline template which specifies a set of configuration parameters for one or more deployment stages of the deployment pipeline.