SOFTWARE CONTAINER REGISTRY INSPECTION

• US 20170177877A1
• Filed: 12/18/2015
• Published: 06/22/2017
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Suarez, Anthony Joseph, Windsor, Scott Kerns, Hayrapetyan, Nare, Gerdesmeier, Daniel Robert, Prakash, Pooja Kalpana
• Granted Patent: US 10,032,032 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 2009/45562 Creating, deleting, cloning virtual machine instances
• CPC Class Codes: G06F 21/562 Static detection
• CPC Class Codes: G06F 21/577 Assessing vulnerabilities and evaluating computer system security
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2221/033 Test or assess software
• CPC Class Codes: G06F 8/63 Image based installation; Cloning; Build to order
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A request to a scan a software image for specified criteria is received, the software image comprising layers stored in a first data store. Metadata in a second data store, different from the first data store, is searched through to obtain information corresponding to the software image. A first set of the layers that matches the specified criteria is determined, based at least in part on the information. The first set of layers is marked as un-referenceable. Asynchronous to fulfillment of the request, a second set of layers of the layers to be deleted is determined, based at least in part on the metadata, the second set of layers including layers marked as un-referenceable, and the second set of layers is deleted.

Claims

1. A system, comprising:

one or more processors; and

memory including instructions that, as a result of execution by the one or more processors, cause the system to;

receive a request to perform a scan of a set of container images stored in at least one repository, container images of the set of container images comprising image layers stored in the at least one repository, the request including criteria for identifying image layers associated with a security vulnerability; and

in response to receiving the request;

search a set of manifests stored in a database of a structured data store to obtain content-addressable identifiers for the image layers, the set of manifests comprising metadata about the set of container images;

determine, based at least in part on the content-addressable identifiers, image layers that match the criteria; and

flag the image layers that match the criteria as un-referenceable; and

as a result of an occurrence of a current time corresponding to a time scheduled for performing a deletion operation;

determine, by analyzing the set of manifests, one or more unreferenced image layers, the one or more unreferenced image layers being;

flagged as un-referenceable, or

unreferenced by a manifest of a tagged container image; and

delete the one or more unreferenced image layers.

2. The system of claim 1, wherein:

the criteria is a specified content-addressable identifier;

the request is a request to search the content-addressable identifiers for an occurrence of the specified content-addressable identifier; and

the instructions that cause the system to determine the image layers that match the criteria include instructions that cause the system to determine, based at least in part on the content-addressable identifiers, the image layers from layers that have a content-addressable identifier that matches the specified content-addressable identifier.

3. The system of claim 1, wherein:

the criteria is a digital fingerprint;

the request is a request to search one or more files in the image layers for an occurrence of the digital fingerprint; and

the instructions that cause the system to determine the image layers that match the criteria include instructions that cause the system to determine the image layers by identifying one or more layers of the image layers in which at least one file has the occurrence of the digital fingerprint, the one or more layers identified forming the image layers.

4. The system of claim 1, wherein flagging the image layers as un-referenceable:

causes the image layers flagged as un-referenceable to be inaccessible to customers of a computing resource service provider that hosts the system; and

prevents the system from launching a container image that includes a layer of the image layers flagged as un-referenceable.

5. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

receiving a request to scan a software image for a match to specified criteria, the software image comprising image layers stored in a data object store assigned to an account;

searching metadata in a structured data store, different from the data object store, to obtain a set of identifiers for the image layers;

determining, based at least in part on the set of identifiers, that a first set of the image layers is associated with a match to the specified criteria;

marking the first set of the image layers as un-referenceable;

detecting an occurrence of an event that triggers deletion of un-referenceable software image layers;

determining, by analyzing the metadata, a set of un-referenceable layers of the image layers, the set of un-referenceable layers including the first set of the image layers and a second set of image layers comprising image layers stored in the data object store that are associated with an untagged software image; and

deleting the set of un-referenceable layers.

6. The computer-implemented method of claim 5, wherein:

the image layers are stored in the data object store as encrypted image layers; and

determining that the first set of the image layers is associated with the match to the specified criteria include;

obtaining a decryption key from an entity associated with the account;

decrypting the encrypted image layers using the decryption key to form decrypted image layers; and

determining the first set of the image layers from one or more layers of the decrypted image layers that is associated with the match to the specified criteria.

7. The computer-implemented method of claim 5, wherein:

the data object store assigned to the account is maintained by a computing resource service provider that provides the software image to one or more customers of the computing resource service provider; and

as a result of determining that the first set of the image layers is associated with the match to the specified criteria, the method further comprises;

determining a set of the one or more customers having instances in which the software image has been launched; and

notifying the set of the one or more customers of a potential vulnerability found with the software image.

8. The computer-implemented method of claim 5, wherein determining the set of un-referenceable layers includes:

searching the metadata of the structured data store for one or more layers of the image layers that are unassociated with a tagged software image; and

identifying, for deletion, the one or more layers as the second set of layers.

9. The computer-implemented method of claim 5, wherein the event is one of an instruction from an entity associated with the account received through an application programming interface or a clock event wherein a current time corresponds to a predetermined schedule for performing a deletion operation.

10. The computer-implemented method of claim 5, wherein the metadata includes a manifest for the software image that includes, for each image layer of the image layers, a content-addressable identifier that uniquely corresponds to the image layer and a checksum for verifying integrity of the image layer.

11. The computer-implemented method of claim 5, wherein:

the data object store includes a set of repositories associated with the account;

the method further comprises;

receiving, through an application programming interface, an indication from a customer of a computing resource service provider associated with the account, a selection of one or more repositories of the set of repositories for the scan; and

determining that the first set of the image layers is associated with the match to the specified criteria further includes determining the first set of the image layers from image layers of the image layers that are stored in the one or more repositories.

12. The computer-implemented method of claim 5, wherein:

the method further comprises receiving, through an application programming interface, an indication from a customer of one or more levels of vulnerabilities for the scan; and

determining that the first set of the image layers is associated with the match to the specified criteria further includes determining that the match to the specified criteria is associated with the one or more levels of vulnerabilities indicated by the customer.

13. A one or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, as a result of execution by one or more processors of a distributed computer system, cause the distributed computer system to at least:

receive a request to scan a software image for specified criteria, the software image comprising layers stored in a first data store;

search through metadata in a second data store, different from the first data store, to obtain information corresponding to the software image;

determine, based at least in part on the information, a first set of the layers that matches the specified criteria;

mark the first set of the layers as un-referenceable; and

asynchronous to fulfillment of the request;

determine, based at least in part on the metadata, a second set of the layers to be deleted, the second set of the layers including layers marked as un-referenceable; and

delete the second set of the layers.

14. The one or more non-transitory computer-readable storage media of claim 13, wherein the executable instructions further include executable instructions that cause the distributed computer system to:

determine, based at least in part on the metadata, a third set of layers from one or more layers of the layers that are unlinked to an image that has a tag; and

delete the third set of layers.

15. The one or more non-transitory computer-readable storage media of claim 13, wherein an event that triggers the distributed computer system to determine the second set of the layers to be deleted is one of:

receiving, from a device associated with a customer of a computing resource service provider hosting the distributed computer system, an application programming interface request to clean a repository of the customer, the repository located in the second data store,

receiving, from the device associated with the customer, an application programming interface request to delete a particular version of the software image from the repository, or

an occurrence of a current time that corresponds to a predefined schedule for performing garbage collection.

16. The one or more non-transitory computer-readable storage media of claim 13, wherein the executable instructions further include executable instructions that cause the distributed computer system to, in response to receiving a second request to launch the software image to run as a software container in an instance:

make a determination whether the layers includes a layer that has been marked as un-referenceable; and

based at least in part on the determination, deny the second request.

17. The one or more non-transitory computer-readable storage media of claim 13, wherein:

the first data store is a data object store that stores the layers as a set of data objects; and

the second data store is structured data storage that hosts a scalable, distributed database for storing metadata about images.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein:

the specified criteria include at least one specified content-addressable identifier of a layer; and

the executable instructions that cause the distributed computer system to determine the first set of the layers include executable instructions that cause the distributed computer system to;

search the metadata of the second data store for one or more layers of set of the layers that have content-addressable identifiers that match the at least one specified content-addressable identifier; and

identify the one or more layers as the first set of the layers.

19. The one or more non-transitory computer-readable storage media of claim 13, wherein:

the specified criteria include reference criteria for identifying one or more files in a layer; and

the executable instructions that cause the distributed computer system to determine the first set of the layers include executable instructions that cause the distributed computer system to;

obtain the software image from the first data store;

open the layers of the software image to form opened layers; and

determine the first set of the layers at least in part from layers of the opened layers that contain one or more files that match the reference criteria.

20. The one or more non-transitory computer-readable storage media of claim 19, wherein:

the software image is stored in the first data store as an encrypted software image; and

the executable instructions that cause the distributed computer system to determine the first set of the layers include executable instructions that cause the distributed computer system to;

decrypt the encrypted software image using a cryptographic key shared between an entity and the distributed computer system to form decrypted layers, the entity being associated with the encrypted software image through an account that is hosted by a computing resource service provider that hosts the distributed computer system; and

determine the first set of the layers at least in part from layers of the decrypted layers that contain the one or more files that match the reference criteria.