Distributed data set encryption and decryption

• US 9,990,367 B2
• Filed: 09/01/2017
• Issued: 06/05/2018
• Est. Priority Date: 07/27/2015
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Bowman, Brian Payton, Gass, III, Mark Kuebler
• Primary Examiner(s): Rigol, Yaima
• Assistant Examiner(s): WOOLWINE, SHANE D
• Application Number:
  US15/694,217
• Publication Number:
  US 20180011865A1
• Time in Patent Office: 277 Days
• Field of Search: 711173, 711135
• CPC Class Codes: G06F 12/0292 using tables or multilevel address translation means G06F12/023 takes precedence; address translation in virtual memory systems G06F12/10
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/1056 Simplification
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/262 configured as RAID
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455

Abstract

An apparatus including a processor caused to: receive sizes and data block encryption data for multiple encrypted data blocks from multiple node devices, wherein data block encryption data is separately generated and used by each node device to encrypt a portion of a data set to generate one of the multiple encrypted data blocks; for each encrypted data block, generate a corresponding map entry within map data to include size and data block encryption data; and in response to receiving size and data block encryption data for all encrypted data blocks, encrypt a portion of the map data to generate an encrypted map base, wherein the portion of map data includes at least a subset of the multiple map entries, and transmit the encrypted map base to one or more storage devices to be stored within a data file along with the multiple encrypted data blocks.

Claims

1. An apparatus comprising a processor component and a storage to store instructions that, when executed by the processor component, cause the processor component to perform operations comprising:

transmit, to multiple node devices, a command to encrypt a data set distributed among the multiple node devices for storage;

receive, from at least one node device of the multiple node devices, at least a portion of metadata indicative of organization of data within the data set, wherein;

the data set is to be stored as multiple encrypted data blocks within a data file maintained by one or more storage devices;

each node device of the multiple node devices is to provide at least one encrypted data block of the multiple encrypted data blocks to be stored;

the organization of the multiple encrypted data blocks within the data file is to be indicated in map data that is to comprise multiple map entries;

each map entry of the multiple map entries is to correspond to an encrypted data block of the multiple encrypted data blocks; and

at least a subset of the multiple node devices are to each encrypt a portion of the data of the data set to generate at least one of the multiple encrypted data blocks at least partially in parallel;

receive, from each node device of the multiple node devices, an indication of a size of one of the multiple encrypted data blocks and data block encryption data, wherein the data block encryption data is generated by the node device and is used by the node device to encrypt a portion of the data set to generate the one of the multiple encrypted data blocks;

for each encrypted data block of the multiple encrypted data blocks for which an indication of size and data block encryption data is received, generate a corresponding one of the multiple map entries within the map data to include the indication of size and the data block encryption data; and

in response to receipt of indications of size and data block encryption data for the encrypted data blocks of the multiple encrypted data blocks, perform operations comprising;

use, by the processor component, metadata block encryption data to encrypt the metadata to generate an encrypted metadata block;

add the metadata block encryption data to the map data;

transmit the encrypted metadata block to the one or more storage devices to be stored at a first predetermined location within the data file;

use, by the processor component, first map block encryption data to encrypt a first portion of the map data to generate an encrypted map base, wherein the first portion of the map data includes at least a first subset of the multiple map entries and the metadata block encryption data; and

transmit the encrypted map base to the one or more storage devices to be stored at a second predetermined location within the data file.

2. The apparatus of claim 1, wherein the processor component is caused to transmit a pass phrase value to each of the multiple node devices to use to encrypt a portion of the data of the data set to generate at least one of the multiple encrypted data blocks at least partially in parallel, wherein no indication of the pass phrase value is to be transmitted to the one or more storage devices.

3. The apparatus of claim 1, wherein:

the data set comprises partitioned data, wherein;

the data of the data set is divided into multiple partitions that are each assigned a partition label;

each encrypted data block comprises at least one data sub-block that includes data of the data set that belongs to a single partition of the multiple partitions; and

the indication of a size of one of the multiple encrypted data blocks received from each node device comprises an indication of a sub-block size indicative of a quantity of the data of the data set within each data sub-block within the one of the encrypted data blocks; and

in response to an indication received from the at least one node device that the data set comprises partitioned data, the processor component is caused to perform operations comprising;

generate within each map entry a sub-block count indicative of a quantity of the data sub-blocks within the corresponding encrypted data block; and

generate within each map entry a separate map sub-entry for each data sub-block within the corresponding encrypted data block, wherein;

each map sub-entry comprises the sub-block size of the corresponding data sub-block and a hashed identifier; and

the hashed identifier is derived from the partition label of the partition to which the data of the data set that is within the corresponding data sub-block belongs.

4. The apparatus of claim 1, wherein:

the processor component is caused to perform operations comprising;

randomly generate a first salt value as part of the metadata block encryption data;

use, by the processor component, the metadata block encryption data and a pass phrase to generate a first encryption cipher;

use, by the processor component, the first encryption cipher and a size of the metadata to encrypt the metadata;

randomly generate a second salt value as part of the first map block encryption data;

use, by the processor component, the first map block encryption data and the pass phrase to generate a second encryption cipher; and

use, by the processor component, the second encryption cipher and a size of the first portion of the map data to encrypt the first portion of the map data wherein the pass phrase is not to be transmitted to the one or more storage devices;

each data block encryption data is separately generated by one of the multiple node devices for a corresponding one of the multiple encrypted data blocks, and comprises another salt value that is randomly generated by the one of the multiple node devices; and

more than one node device of the multiple node devices generates more than one encrypted data block of the multiple encrypted data blocks at least partially in parallel.

5. The apparatus of claim 1, wherein the processor component is caused to perform operations comprising:

perform an XOR operation with the first map block encryption data; and

following the performance of the XOR operation, transmit the first map block encryption data to the one or more storage devices to be stored at a third predetermined location within the data file.

6. The apparatus of claim 1, wherein the processor component is caused, prior to encryption of the first portion of the data map, to perform operations comprising:

use, by the processor component, the second map block encryption data to encrypt a second portion of the map data to generate a first encrypted map extension, wherein the second portion of the map data includes a second subset of the multiple map entries;

add the second map block encryption data to the first portion of the map data prior to encryption of the first portion of the map data; and

transmit the first encrypted map extension to the one or more storage devices to be stored at a third predetermined location within the data file.

7. The apparatus of claim 6, wherein the processor component is caused to generate multiple encrypted map extensions, wherein:

the multiple encrypted map extensions comprises the first encrypted map extension; and

each successive one of the multiple encrypted map extensions following the first encrypted map extension, in a pre-defined order, is up to twice the size of the preceding one of the multiple encrypted map extensions.

8. The apparatus of claim 1, wherein:

the multiple node devices are capable of communication with the one or more storage devices; and

the processor component is caused to perform operations comprising;

receive, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, a request for a pointer to a location within the data file at which to store the one of the multiple encrypted data blocks; and

for each encrypted data block for which a request for a pointer to a location is received, determine the location within the data file at which the corresponding node device is to store the encrypted data block, and transmit, to the corresponding node device, a pointer to the location within the data file.

9. The apparatus of claim 1, wherein:

at least one node device of the multiple node devices is not capable of communication with the one or more storage devices; and

the processor component is caused to perform operations comprising;

receive, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, the one of the multiple encrypted data blocks; and

for each encrypted data block received from a corresponding one of the multiple node devices, determine the location within the data file at which to store the encrypted data block, and transmit, to the one or more storage devices, the encrypted data block to be stored at the location within the data file.

10. The apparatus of claim 1, wherein:

the apparatus comprises an additional node device; and

the processor component is caused to perform operations comprising;

perform a task with a portion of the data set at least partially in parallel with one or more of the multiple node devices;

generate data block encryption data to encrypt the portion of the data set to generate one of the multiple encrypted data blocks; and

generate a corresponding one of the multiple map entries to include an indication of a size of the one of the multiple encrypted data blocks and the data block encryption data.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor component to perform operations comprising:

transmit, to multiple node devices, a command to encrypt a data set distributed among the multiple node devices for storage;

receive, from at least one node device of the multiple node devices, at least a portion of metadata indicative of organization of data within the data set, wherein;

the data set is to be stored as multiple encrypted data blocks within a data file maintained by one or more storage devices;

each node device of the multiple node devices is to provide at least one encrypted data block of the multiple encrypted data blocks to be stored;

the organization of the multiple encrypted data blocks within the data file is to be indicated in map data that is to comprise multiple map entries;

each map entry of the multiple map entries is to correspond to an encrypted data block of the multiple encrypted data blocks; and

at least a subset of the multiple node devices are to each encrypt a portion of the data of the data set to generate at least one of the multiple encrypted data blocks at least partially in parallel;

receive, from each node device of the multiple node devices, an indication of a size of one of the multiple encrypted data blocks, and data block encryption data, wherein the data block encryption data is generated by the node device and is used by the node device to encrypt a portion of the data set to generate the one of the multiple encrypted data blocks;

for each encrypted data block of the multiple encrypted data blocks for which an indication of size and data block encryption data is received, generate a corresponding one of the multiple map entries within the map data to include the indication of size and the data block encryption data; and

in response to receipt of indications of size and data block encryption data for the encrypted data blocks of the multiple encrypted data blocks, perform operations comprising;

use, by the processor component, metadata block encryption data to encrypt the metadata to generate an encrypted metadata block;

add the metadata block encryption data to the map data;

transmit the encrypted metadata block to the one or more storage devices to be stored at a first predetermined location within the data file;

use, by the processor component first map block encryption data to encrypt a first portion of the map data to generate an encrypted map base, wherein the first portion of the map data includes at least a first subset of the multiple map entries and the metadata block encryption data; and

transmit the encrypted map base to the one or more storage devices to be stored at a second predetermined location within the data file.

12. The computer-program product of claim 11, wherein the processor component is caused to transmit a pass phrase value to each of the multiple node devices to use to encrypt a portion of the data of the data set to generate at least one of the multiple encrypted data blocks at least partially in parallel, wherein no indication of the pass phrase value is to be transmitted to the one or more storage devices.

13. The computer-program product of claim 11, wherein:

the data set comprises partitioned data, wherein;

the data of the data set is divided into multiple partitions that are each assigned a partition label;

each encrypted data block comprises at least one data sub-block that includes data of the data set that belongs to a single partition of the multiple partitions; and

the indication of a size of one of the multiple encrypted data blocks received from each node device comprises an indication of a sub-block size indicative of a quantity of the data of the data set within each data sub-block within the one of the encrypted data blocks; and

in response to an indication received from the at least one node device that the data set comprises partitioned data, the processor component is caused to perform operations comprising;

generate within each map entry a sub-block count indicative of a quantity of the data sub-blocks within the corresponding encrypted data block; and

generate within each map entry a separate map sub-entry for each data sub-block within the corresponding encrypted data block, wherein;

each map sub-entry comprises the sub-block size of the corresponding data sub-block and a hashed identifier; and

the hashed identifier is derived from the partition label of the partition to which the data of the data set that is within the corresponding data sub-block belongs.

14. The computer-program product of claim 11, wherein:

the processor component is caused to perform operations comprising;

randomly generate a first salt value as part of the metadata block encryption data;

use, by the processor component, the metadata block encryption data and a pass phrase to generate a first encryption cipher;

use, by the processor component, the first encryption cipher and a size of the metadata to encrypt the metadata;

randomly generate a second salt value as part of the first map block encryption data;

use, by the processor component, the first map block encryption data and the pass phrase to generate a second encryption cipher; and

use, by the processor component, the second encryption cipher and a size of the first portion of the map data to encrypt the first portion of the map data wherein the pass phrase is not to be transmitted to the one or more storage devices;

each data block encryption data is separately generated by one of the multiple node devices for a corresponding one of the multiple encrypted data blocks, and comprises another salt value that is randomly generated by the one of the multiple node devices; and

more than one node device of the multiple node devices generates more than one encrypted data block of the multiple encrypted data blocks at least partially in parallel.

15. The computer-program product of claim 11, wherein the processor component is caused to perform operations comprising:

perform an XOR operation with the first map block encryption data; and

following the performance of the XOR operation, transmit the first map block encryption data to the one or more storage devices to be stored at a third predetermined location within the data file.

16. The computer-program product of claim 11, wherein the processor component is caused, prior to encryption of the first portion of the data map, to perform operations comprising:

use, by the processor component, the second map block encryption data to encrypt a second portion of the map data to generate a first encrypted map extension, wherein the second portion of the map data includes a second subset of the multiple map entries;

add the second map block encryption data to the first portion of the map data prior to encryption of the first portion of the map data; and

transmit the first encrypted map extension to the one or more storage devices to be stored at a third predetermined location within the data file.

17. The computer-program product of claim 16, wherein the processor component is caused to generate multiple encrypted map extensions, wherein:

the multiple encrypted map extensions comprises the first encrypted map extension; and

each successive one of the multiple encrypted map extensions following the first encrypted map extension, in a pre-defined order, is up to twice the size of the preceding one of the multiple encrypted map extensions.

18. The computer-program product of claim 11, wherein:

the multiple node devices are capable of communication with the one or more storage devices; and

the processor component is caused to perform operations comprising;

receive, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, a request for a pointer to a location within the data file at which to store the one of the multiple encrypted data blocks; and

for each encrypted data block for which a request for a pointer to a location is received, determine the location within the data file at which the corresponding node device is to store the encrypted data block, and transmit, to the corresponding node device, a pointer to the location within the data file.

19. The computer-program product of claim 11, wherein:

at least one node device of the multiple node devices is not capable of communication with the one or more storage devices; and

the processor component is caused to perform operations comprising;

receive, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, the one of the multiple encrypted data blocks; and

for each encrypted data block received from a corresponding one of the multiple node devices, determine the location within the data file at which to store the encrypted data block, and transmit, to the one or more storage devices, the encrypted data block to be stored at the location within the data file.

20. The computer-program product of claim 11, wherein:

the processor component is incorporated into an additional node device; and

the processor component is caused to perform operations comprising;

perform a task with a portion of the data set at least partially in parallel with one or more of the multiple node devices;

generate data block encryption data to encrypt the portion of the data set to generate one of the multiple encrypted data blocks; and

generate a corresponding one of the multiple map entries to include an indication of a size of the one of the multiple encrypted data blocks and the data block encryption data.

21. A computer-implemented method comprising:

transmitting, to multiple node devices via a network, a command to encrypt a data set distributed among the multiple node devices for storage;

receiving, from at least one node device of the multiple node devices, at least a portion of metadata indicative of organization of data within the data set, wherein;

the data set is to be stored as multiple encrypted data blocks within a data file maintained by one or more storage devices;

each node device of the multiple node devices is to provide at least one encrypted data block of the multiple encrypted data blocks to be stored;

the organization of the multiple encrypted data blocks within the data file is to be indicated in map data that is to comprise multiple map entries;

each map entry of the multiple map entries is to correspond to an encrypted data block of the multiple encrypted data blocks; and

at least a subset of the multiple node devices are to each encrypt a portion of the data of the data set to generate at least one of the multiple encrypted data blocks at least partially in parallel;

receiving, from each node device of the multiple node devices, an indication of a size of one of the multiple encrypted data blocks, and data block encryption data, wherein the data block encryption data is generated by the node device and is used by the node device to encrypt a portion of the data set to generate the one of the multiple encrypted data blocks;

for each encrypted data block of the multiple encrypted data blocks for which an indication of size and data block encryption data is received, generating, by a processor component, a corresponding one of the multiple map entries within the map data to include the indication of size and the data block encryption data; and

in response to receipt of indications of size and data block encryption data for the encrypted data blocks of the multiple encrypted data blocks, performing operations comprising;

using, by the processor component, the metadata block encryption data to encrypt the metadata to generate an encrypted metadata block;

adding, by the processor component, the metadata block encryption data to the map data;

transmitting, via the network, the encrypted metadata block to the one or more storage devices to be stored at a first predetermined location within the data file;

using, by the processor component, the first map block encryption data to encrypt a first portion of the map data to generate an encrypted map base, wherein the first portion of the map data includes at least a first subset of the multiple map entries and the metadata block encryption data; and

transmitting, via the network, the encrypted map base to the one or more storage devices to be stored at a second predetermined location within the data file.

22. The computer-implemented method of claim 21, comprising transmitting, via the network, a pass phrase value to each of the multiple node devices to use to encrypt a portion of the data of the data set to generate, within each of the multiple node devices, at least one of the multiple encrypted data blocks at least partially in parallel, wherein no indication of the pass phrase value is to be transmitted to the one or more storage devices.

23. The computer-implemented method of claim 21, wherein:

the data set comprises partitioned data, wherein;

the data of the data set is divided into multiple partitions that are each assigned a partition label;

each encrypted data block comprises at least one data sub-block that includes data of the data set that belongs to a single partition of the multiple partitions; and

the indication of a size of one of the multiple encrypted data blocks received from each node device comprises an indication of a sub-block size indicative of a quantity of the data of the data set within each data sub-block within the one of the encrypted data blocks; and

in response to an indication received from the at least one node device that the data set comprises partitioned data, performing operations comprising;

generating, by the processor component, within each map entry a sub-block count indicative of a quantity of the data sub-blocks within the corresponding encrypted data block; and

generating, by the processor component, within each map entry a separate map sub-entry for each data sub-block within the corresponding encrypted data block, wherein;

each map sub-entry comprises the sub-block size of the corresponding data sub-block and a hashed identifier; and

the hashed identifier is derived from the partition label of the partition to which the data of the data set that is within the corresponding data sub-block belongs.

24. The computer-implemented method of claim 21, wherein:

the method comprises performing operations comprising;

randomly generating, by the processor component, a first salt value as part of the metadata block encryption data;

using, by the processor component, the metadata block encryption data and a pass phrase to generate a first encryption cipher;

using, by the processor component, the first encryption cipher and a size of the metadata to encrypt the metadata;

randomly generating, by the processor component, a second salt value as part of the first map block encryption data;

using, by the processor component, the first map block encryption data and the pass phrase to generate a second encryption cipher; and

using, by the processor component, the second encryption cipher and a size of the first portion of the map data to encrypt the first portion of the map data wherein the pass phrase is not to be transmitted to the one or more storage devices;

each data block encryption data is separately generated by one of the multiple node devices for a corresponding one of the multiple encrypted data blocks, and comprises another salt value that is randomly generated by the one of the multiple node devices; and

more than one node device of the multiple node devices generates more than one encrypted data block of the multiple encrypted data blocks at least partially in parallel.

25. The computer-implemented method of claim 21, comprising:

performing, by the processor component, an XOR operation with the first map block encryption data; and

following the performance of the XOR operation, transmitting, via the network, the first map block encryption data to the one or more storage devices to be stored at a third predetermined location within the data file.

26. The computer-implemented method of claim 21, comprising, prior to encryption of the first portion of the data map, performing operations comprising:

using, by the processor component, the second map block encryption data to encrypt a second portion of the map data to generate a first encrypted map extension, wherein the second portion of the map data includes a second subset of the multiple map entries;

adding, by the processor component, the second map block encryption data to the first portion of the map data prior to encryption of the first portion of the map data; and

transmitting, via the network, the first encrypted map extension to the one or more storage devices to be stored at a third predetermined location within the data file.

27. The computer-implemented method of claim 26, comprising generating, by the processor component, multiple encrypted map extensions, wherein:

the multiple encrypted map extensions comprises the first encrypted map extension; and

each successive one of the multiple encrypted map extensions following the first encrypted map extension in a pre-defined order, is up to twice the size of the preceding one of the multiple encrypted map extensions.

28. The computer-implemented method of claim 21, wherein:

the multiple node devices are capable of communication with the one or more storage devices; and

the method comprises performing operations comprising;

receiving, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, a request for a pointer to a location within the data file at which to store the one of the multiple encrypted data blocks; and

for each encrypted data block for which a request for a pointer to a location is received, determining, by the processor component, the location within the data file at which the corresponding node device is to store the encrypted data block, and transmitting, via the network to the corresponding node device, a pointer to the location within the data file.

29. The computer-implemented method of claim 21, wherein:

at least one node device of the multiple node devices is not capable of communication with the one or more storage devices; and

the method comprises performing operations comprising;

receiving, from each node device of the multiple node devices, and with the indication of size and the data block encryption data for one of the multiple encrypted data blocks, the one of the multiple encrypted data blocks; and

for each encrypted data block received from a corresponding one of the multiple node devices, determining, by the processor component, the location within the data file at which to store the encrypted data block, and transmitting, via the network to the one or more storage devices, the encrypted data block to be stored at the location within the data file.

30. The computer-implemented method of claim 21, wherein:

the processor component is incorporated into an additional node device; and

the method comprises performing operations comprising;

performing, by the processor component, a task with a portion of the data set at least partially in parallel with one or more of the multiple node devices;

generating, by the processor component, data block encryption data to encrypt the portion of the data set to generate one of the multiple encrypted data blocks; and

generating, by the processor component, a corresponding one of the multiple map entries to include an indication of a size of the one of the multiple encrypted data blocks and the data block encryption data.