Software container registry inspection

• US 10,032,032 B2
• Filed: 12/18/2015
• Issued: 07/24/2018
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Suarez, Anthony Joseph, Windsor, Scott Kerns, Hayrapetyan, Nare, Gerdesmeier, Daniel Robert, Prakash, Pooja Kalpana
• Primary Examiner(s): ZHAO, DON GORDON
• Application Number:
  US14/975,637
• Publication Number:
  US 20170177877A1
• Time in Patent Office: 949 Days
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/033 Test or assess software
• CPC Class Codes: G06F 8/63 Image based installation; Cloning; Build to order
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A request to a scan a software image for specified criteria is received, the software image comprising layers stored in a first data store. Metadata in a second data store, different from the first data store, is searched through to obtain information corresponding to the software image. A first set of the layers that matches the specified criteria is determined, based at least in part on the information. The first set of layers is marked as un-referenceable. Asynchronous to fulfillment of the request, a second set of layers of the layers to be deleted is determined, based at least in part on the metadata, the second set of layers including layers marked as un-referenceable, and the second set of layers is deleted.

Claims

1. A system, comprising:

one or more processors; and

memory including executable instructions that, as a result of execution by the one or more processors, cause the system to;

receive a request to perform a scan of a set of container images stored in at least one repository, a container image of the set of container images comprising encrypted image layers assigned to an account, the request including criteria for identifying insecure image layers associated with a security vulnerability; and

in response to receiving the request;

search a set of manifests stored in a database of a structured data store to obtain content-addressable identifiers for the encrypted image layers, the set of manifests comprising metadata about the set of container images;

determine, based at least in part on the content-addressable identifiers, an insecure image layer at least in part by causing the system to;

decrypt the encrypted image layers using a decryption key to form decrypted image layers, the decryption key being obtained via an entity associated with the account; and

determine the insecure image layer from one of the decrypted image layer that is associated with a match to the criteria; and

flag the insecure image layer as un-referenceable; and

as a result of an occurrence of a current time corresponding to a time scheduled for performing a deletion operation;

determine, by analyzing the set of manifests, a superset of image layers that includes one or more image layers that are flagged as un-referenceable; and

delete the superset of image layers.

2. The system of claim 1, wherein:

the criteria comprise a specified content-addressable identifier;

the request is a request to search the content-addressable identifiers for an occurrence of the specified content-addressable identifier; and

the executable instructions that cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer further cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer from a layer that has a content-addressable identifier that matches the specified content-addressable identifier.

3. The system of claim 1, wherein:

the criteria comprise a digital fingerprint;

the request is a request to search one or more files in the encrypted image layers for an occurrence of the digital fingerprint; and

the executable instructions that cause the system to determine, based at least in part on the content-addressable identifiers, the insecure image layer further cause the system to determine the insecure image layer by identifying, as the insecure image layer, a layer of the decrypted image layers in which at least one file has the occurrence of the digital fingerprint.

4. The system of claim 1, wherein flagging the insecure image layers as un-referenceable:

causes the insecure image layer flagged as un-referenceable to be inaccessible to customers of a computing resource service provider that hosts the system; and

prevents the system from launching a container image that includes the insecure image layer flagged as un-referenceable.

5. The system of claim 1, wherein the criteria includes a digital fingerprint associated with malicious software (malware) or sensitive data.

6. The system of claim 1, wherein the executable instructions further cause the system to, as a result of the performing the scan of the set of container images, update a table to include a timestamp that indicates a time at which the set of container images were scanned.

7. A computer-implemented method, comprising:

receiving a request to scan a software image for a match to specified criteria, the software image comprising image layers stored as encrypted image layers in a data object store assigned to an account;

searching metadata in a structured data store, different from the data object store, to obtain a set of identifiers for the image layers;

determining, based at least in part on the set of identifiers, that a first set of the image layers is associated with a match to the specified criteria at least in part by;

obtaining a decryption key from an entity associated with the account;

decrypting the encrypted image layers using the decryption key to form decrypted image layers; and

determining the first set of the image layers from one or more layers of the decrypted image layers that are associated with the match to the specified criteria;

marking the first set of the image layers as un-referenceable;

detecting an occurrence of an event that triggers deletion of un-referenceable image layers;

determining, by analyzing the metadata, a set of un-referenceable layers of the image layers, the set of un-referenceable layers including the first set of the image layers and a second set of image layers comprising image layers stored in the data object store that are associated with an untagged software image; and

deleting the set of un-referenceable layers.

8. The computer-implemented method of claim 7, wherein:

the data object store assigned to the account is maintained by a computing resource service provider that provides the software image to one or more customers of the computing resource service provider; and

as a result of determining that the first set of the image layers is associated with the match to the specified criteria, the method further comprises;

determining a set of the one or more customers having instances in which the software image has been launched; and

notifying the set of the one or more customers of a potential vulnerability found with the software image.

9. The computer-implemented method of claim 7, wherein determining the set of un-referenceable layers includes:

searching the metadata of the structured data store for one or more layers of the image layers that are unassociated with a tagged software image; and

identifying, for deletion, the one or more layers as the second set of image layers.

10. The computer-implemented method of claim 7, wherein the event is one of an instruction from an entity associated with the account received through an application programming interface or a clock event wherein a current time corresponds to a predetermined schedule for performing a deletion operation.

11. The computer-implemented method of claim 7, wherein the metadata includes a manifest for the software image that includes, for each image layer of the image layers, a content-addressable identifier that uniquely corresponds to the image layer and a checksum for verifying integrity of the image layer.

12. The computer-implemented method of claim 7, wherein:

the data object store includes a set of repositories associated with the account;

the method further comprises;

receiving, through an application programming interface, an indication from a customer of a computing resource service provider associated with the account, a selection of one or more repositories of the set of repositories for the scan; and

determining that the first set of the image layers is associated with the match to the specified criteria further includes determining the first set of the image layers from image layers of the image layers that are stored in the one or more repositories.

13. The computer-implemented method of claim 7, wherein:

the method further comprises receiving, through an application programming interface, an indication from a customer of one or more levels of vulnerabilities for the scan; and

determining that the first set of the image layers is associated with the match to the specified criteria further includes determining that the match to the specified criteria is associated with the one or more levels of vulnerabilities indicated by the customer.

14. A one or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, as a result of execution by one or more processors of a distributed computer system, cause the distributed computer system to at least:

receive a request to scan a software image for specified criteria, the software image comprising image layers stored in a first data store as an encrypted software image;

search through metadata in a second data store, different from the first data store, to obtain information corresponding to the software image;

determine, based at least in part on the information, a first set of the image layers that matches the specified criteria at least in part by causing the distributed computer system to;

decrypt the encrypted software image using a cryptographic key shared between an entity and the distributed computer system to form decrypted layers, the entity being associated with the encrypted software image through an account that is hosted by a computing resource service provider that hosts the distributed computer system; and

determine the first set of the image layers at least in part from layers of the decrypted layers that contain one or more files that match reference criteria;

mark the first set of the image layers as un-referenceable; and

asynchronous to fulfilment of the request;

determine, based at least in part on the metadata, a second set of the image layers to be deleted, the second set of the image layers including the first set of the image layers; and

delete the second set of the image layers.

15. The one or more non-transitory computer-readable storage media of claim 14, wherein the executable instructions further include executable instructions that cause the distributed computer system to:

determine, based at least in part on the metadata, a third set of layers from one or more layers of the image layers that are unlinked to an image that has a tag; and

delete the third set of layers.

16. The one or more non-transitory computer-readable storage media of claim 14, wherein an event that triggers the distributed computer system to determine the second set of the image layers to be deleted is one of:

receiving, from a device associated with a customer of a computing resource service provider hosting the distributed computer system, an application programming interface request to clean a repository of the customer, the repository located in the second data store,

receiving, from the device associated with the customer, an application programming interface request to delete a particular version of the software image from the repository, or

an occurrence of a current time that corresponds to a predefined schedule for performing garbage collection.

17. The one or more non-transitory computer-readable storage media of claim 14, wherein the executable instructions further include executable instructions that cause the distributed computer system to, in response to receiving a second request to launch the software image to run as a software container in an instance:

make a determination whether the image layers includes a layer that has been marked as un-referenceable; and

based at least in part on the determination, deny the second request.

18. The one or more non-transitory computer-readable storage media of claim 14, wherein:

the first data store is a data object store that stores the image layers as a set of data objects; and

the second data store is structured data storage that hosts a scalable, distributed database for storing metadata about images.

19. The one or more non-transitory computer-readable storage media of claim 18, wherein:

the specified criteria include at least one specified content-addressable identifier of a layer; and

the executable instructions that cause the distributed computer system to determine the first set of the image layers include executable instructions that cause the distributed computer system to;

search the metadata of the second data store for one or more layers of set of the image layers that have content-addressable identifiers that match the at least one specified content-addressable identifier; and

identify the one or more layers as the first set of the image layers.

20. The one or more non-transitory computer-readable storage media of claim 14, wherein:

the specified criteria include the reference criteria for identifying one or more files in a layer; and

the executable instructions that cause the distributed computer system to determine the first set of the image layers include executable instructions that cause the distributed computer system to;

obtain the software image from the first data store;

open the image layers of the software image to form opened layers; and

determine the first set of the image layers at least in part from layers of the opened layers that contain one or more files that match the reference criteria.