Provisioning of a shippable storage device and ingesting data from the shippable storage device

• US 10,372,922 B2
• Filed: 04/02/2018
• Issued: 08/06/2019
• Est. Priority Date: 12/18/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Paterra, Frank, Basarir, Firat
• Primary Examiner(s): Le, Chau
• Application Number:
  US15/943,627
• Publication Number:
  US 20180225464A1
• Time in Patent Office: 491 Days
• CPC Class Codes: G06F 21/44 Program or device authentication
• CPC Class Codes: G06F 21/606 by securing the transmission between two devices or processes
• CPC Class Codes: G06F 21/80 in storage media based on magnetic or optical technology, e.g. disks with sectors preventing unauthorised reproduction or copying of disc-type recordable media G11B20/00
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/06314 Calendaring for a resource
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 9/0822 using key encryption key
• CPC Class Codes: H04L 9/0897 involving additional devices, e.g. trusted platform module [TPM], smartcard or USB
• CPC Class Codes: H04L 9/3263 involving certificates, e.g. public key certificate [PKC] or attribute certificate [AC]; Public key infrastructure [PKI] arrangements network architectures or network communication protocols for supporting authentication of entities using certificates in a packet data network H04L63/0823

Abstract

When a client requests a data import job, a remote storage service provider provisions a shippable storage device that will be used to transfer client data from the client to the service provider for import. The service provider generates security information for the data import job, provisions the shippable storage device with the security information, and sends the shippable storage device to the client. The service provider also sends client-keys to the client, separate from the shippable storage device (e.g., via a network). The client receives the device, encrypts the client data and keys, transfers the encrypted data and keys onto the device, and ships it back to the service provider. The remote storage service provider authenticates the storage device, decrypts client-generated keys using the client-keys stored at the storage service provider, decrypts the data using the decrypted client-side generated keys, and imports the decrypted data.

Claims

1. A system, comprising:

one or more computing devices connected to a network of a storage service provider;

at least one shippable storage device received from a client of the storage service provider, wherein the at least one shippable storage device is attached to the network of the storage service provider; and

a data ingestion service implemented on at least one of the one or more computing devices, wherein the data ingestion service is configured to;

determine information for a data import job associated with the shippable storage device;

obtain, based on the information for the data import job, security information and one or more stored keys stored by the storage service provider;

authenticate the shippable storage device based on the security information;

obtain encrypted keys and encrypted data from the shippable storage device;

decrypt one or more of the encrypted keys using the one or more keys stored by the storage service provider to generate decrypted keys;

decrypt the encrypted data based on usage of the decrypted keys to generate decrypted data; and

store the decrypted data at one or more locations at the storage service provider indicated by the information for the data import job.

2. The system as recited in claim 1, wherein the encrypted keys comprise one or more encrypted chunk keys and the decrypted keys comprise one or more file keys, and wherein to decrypt the encrypted data based on usage of the decrypted keys, the data ingestion service is further configured to:

decrypt the one or more encrypted chunk keys using the one or more file keys to generate one or more chunk keys; and

decrypt one or more encrypted chunks of data using the one or more chunk keys to generate the decrypted data.

3. The system as recited in claim 1, wherein the data ingestion service is further configured to:

combine two or more of the decrypted chunks of data to form a file.

4. The system as recited in claim 1, wherein the data ingestion service is further configured to:

receive configuration information from a trusted platform module of the shippable storage device; and

determine, based on the configuration information, that a configuration of the shippable storage device has not changed since the shippable storage device was provisioned and sent to the client.

5. The system as recited in claim 1, wherein the data ingestion service is further configured to:

erase the shippable storage device to remove operating software and data associated with the client from the shippable storage device.

6. The system as recited in claim 1, wherein the decrypted data comprises a shard, and wherein the data ingestion service is configured to:

reconstruct data based on the shard and at least one other shard; and

store the reconstructed data at the one or more locations at the storage service provider.

7. A method, comprising:

performing, by a data ingestion service implemented on one or more computing devices of a storage service provider;

determining a data import job associated with a shippable storage device received from a client, wherein the shippable storage device is attached to a network of the storage service provider;

obtaining, based on the data import job, one or more stored keys stored by the storage service provider;

obtaining encrypted keys and encrypted data from the shippable storage device;

decrypting one or more of the encrypted keys using the one or more stored keys to generate one or more decrypted keys;

decrypting the encrypted data based at least on usage of the one or more decrypted keys to generate decrypted data; and

storing the decrypted data at one or more locations at the storage service provider.

8. The method as recited in claim 7, wherein the encrypted keys comprise one or more encrypted chunk keys and the one or more decrypted keys comprise one or more file keys, and further comprising:

decrypting the one or more encrypted chunk keys using the one or more file keys to generate one or more chunk keys; and

decrypting one or more encrypted chunks of data using the one or more chunk keys to generate one or more decrypted chunks of data.

9. The method as recited in claim 8, further comprising:

combining two or more of the decrypted chunks of data to form a file.

10. The method as recited in claim 9, further comprising:

calculating a checksum for the file;

comparing the calculated checksum to a checksum stored by the storage service provider; and

verifying the integrity of the file in response to determining that the calculated checksum matches the stored checksum.

11. The method as recited in claim 7, wherein the decrypted data comprises a shard, and further comprising:

reconstructing data based on the shard and at least one other shard; and

storing the reconstructed data at the one or more locations at the storage service provider.

12. The method as recited in claim 7, further comprising:

receiving configuration information from a trusted platform module of the shippable storage device; and

determining, based on the configuration information, that a configuration of the shippable storage device has not changed since the shippable storage device was provisioned and sent to the client.

13. The method as recited in claim 7, further comprising:

erasing the shippable storage device to remove data associated with the client from the shippable storage device.

14. A non-transitory computer-accessible storage medium storing program instructions that, when executed on one or more processors of a data ingestion service cause the one or more processors to implement a data ingestion service to perform:

determining a data import job associated with a shippable storage device received from a client, wherein the shippable storage device is attached to a network of the storage service provider;

obtaining, based on the data import job, one or more stored keys stored by the storage service provider;

obtaining encrypted keys and encrypted data from the shippable storage device;

decrypting one or more of the encrypted keys using the one or more stored keys to generate one or more decrypted keys;

decrypting the encrypted data based at least on usage of the one or more decrypted keys to generate decrypted data; and

storing the decrypted data at one or more locations at the storage service provider.

15. The computer-accessible storage medium as recited in claim 14, wherein the encrypted keys comprise one or more encrypted chunk keys and the one or more decrypted keys comprise one or more file keys, and wherein the program instructions when executed cause the one or more processors to perform:

decrypting the one or more encrypted chunk keys using the one or more file keys to generate one or more chunk keys; and

decrypting one or more encrypted chunks of data using the one or more chunk keys to generate one or more decrypted chunks of data.

16. The computer-accessible storage medium as recited in claim 15, wherein the program instructions when executed cause the one or more processors to perform:

combining two or more of the decrypted chunks of data to form a file.

17. The computer-accessible storage medium as recited in claim 16, wherein the program instructions when executed cause the one or more processors to perform:

calculating a checksum for the file;

comparing the calculated checksum to a checksum stored by the storage service provider; and

verifying the integrity of the file in response to determining that the calculated checksum matches the stored checksum.

18. The computer-accessible storage medium as recited in claim 14, wherein the decrypted data comprises a shard, and wherein the program instructions when executed cause the one or more processors to perform:

reconstructing data based on the shard and at least one other shard; and

storing the reconstructed data at the one or more locations at the storage service provider.

19. The computer-accessible storage medium as recited in claim 14, wherein the program instructions when executed cause the one or more processors to perform:

receiving configuration information from a trusted platform module of the shippable storage device; and

determining, based on the configuration information, that a configuration of the shippable storage device has not changed since the shippable storage device was provisioned and sent to the client.

20. The computer-accessible storage medium as recited in claim 14, the program instructions when executed cause the one or more processors to perform:

obtaining, based on the data import job, a root certificate or another key stored by the storage service provider; and

authenticating the shippable storage device based at least on the root certificate or the other key.