DISTRIBUTED STORAGE PROCESSING MODULE

• US 20100332751A1
• Filed: 05/06/2010
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: DHUSE, GREG, QUIGLEY, JOHN
• Granted Patent: US 10,230,692 B2
• Time in Patent Office: Days
• US Class Current: 711/114
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/08 Protocols for interworking; Protocol conversion

Abstract

A dispersed storage (DS) processing module may include a gateway module operable to communicate data and/or corresponding information with a user device and may include an access module operable to segment outbound data of the data into one or more outbound data segments and aggregate one or more inbound data segments into inbound data of the data. The DS processing module may include a grid module operable to encode an outbound data segment of the one or more outbound data segments into a plurality of outbound encoded data slices and decode a plurality of inbound encoded data slices into an inbound data segment of the one or more inbound data segments. The DS processing module may include a storage module operable to output the plurality of outbound encoded data slices to a plurality of DS storage units and receive the plurality of inbound encoded data slices from the plurality of DS storage units.

Claims

1. A dispersed storage (DS) processing module comprises:

a gateway module operable to communicate data and corresponding information with a user device;

an access module operable to;

segment outbound data of the data into one or more outbound data segments in accordance with a data storage protocol; and

aggregate one or more inbound data segments into inbound data of the data in accordance with the data storage protocol;

a grid module operable to;

encode an outbound data segment of the one or more outbound data segments into a plurality of outbound encoded data slices in accordance with an error coding dispersal storage function; and

decode a plurality of inbound encoded data slices into an inbound data segment of the one or more inbound data segments in accordance with the error coding dispersal storage function; and

a storage module operable to;

output the plurality of outbound encoded data slices to a plurality of DS storage units; and

receive the plurality of inbound encoded data slices from the plurality of DS storage units.

2. In the DS processing module of claim 1, wherein the gateway module function comprises at least one of:

message conversion;

protocol conversion;

electrical conversion;

optical conversion;

access control;

user identification;

user information retrieval;

traffic monitoring;

statistics generation;

configuration;

management; and

source name determination.

3. In the DS processing module of claim 1, wherein the data storage protocol comprises at least one of:

a file storage system;

a block storage system; and

an aggregated block storage system.

4. In the DS processing module of claim 1, wherein the corresponding information comprises at least one of:

metadata;

a file system directory;

a block number;

a transaction message;

a user device identity (ID);

a data object identifier;

a source name; and

user information.

5. The DS processing module of claim 4 further comprises:

the access module further operable to;

segment outbound corresponding information of the corresponding information into one or more outbound corresponding information segments in accordance with the data storage protocol; and

aggregate one or more inbound corresponding information segments into inbound corresponding information of the corresponding information in accordance with the data storage protocol;

the grid module further operable to;

encode an outbound corresponding information segment of the one or more outbound corresponding information segments into a plurality of outbound encoded corresponding information slices in accordance with the error coding dispersal storage function; and

decode a plurality of inbound encoded corresponding information slices into an inbound corresponding information segment of the one or more inbound corresponding information segments in accordance with the error coding dispersal storage function; and

the storage module further operable to;

output the plurality of outbound encoded corresponding information slices to the plurality of DS storage units; and

receive the plurality of inbound encoded corresponding information slices from the plurality of DS storage units.

6. In the DS processing module of claim 1, wherein the error coding dispersal storage function includes one or more of:

a pre-slice data manipulation function and operational parameters of the pre-slice data manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

an encoding function and operational parameters of the encoding function including at least one of a pillar width, a read threshold, a write threshold, a caching setting, and a parallelism setting;

a slicing function and operational parameters of the slicing function including at least one of a pillar width, a caching setting, a slicing parameter, and a parallelism settings;

a post-slice data manipulation function and operational parameters of the post-slice data manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

a post-slice data de-manipulation function and operational parameters of the post-slice data de-manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

a de-slicing function and operational parameters of the de-slicing function including at least one of a pillar width, a caching setting, a slicing parameter, and a parallelism settings;

a decoding function and operational parameters of the decoding function including at least one of a pillar width, a read threshold, a write threshold, a caching setting, and a parallelism setting; and

a pre-slice data de-manipulation function and operational parameters of the pre-slice data de-manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting.

7. The DS processing module of claim 1 further comprises:

the grid module further operable to;

generate a plurality of slice names corresponding to the plurality of outbound encoded data slices in accordance with the error coding dispersal storage function; and

the storage module further operable to;

identify the plurality of DS storage units based on the plurality of slice names.

8. The DS processing module of claim 1 further comprises:

the gateway module operable to communicate second data and second corresponding information with the user device, wherein at least a portion of the second corresponding information is substantially identical to at least a portion of the corresponding information;

the access module operable to;

segment outbound second data of the second data into one or more outbound second data segments in accordance with the data storage protocol; and

aggregate one or more inbound second data segments into inbound second data of the second data in accordance with the data storage protocol;

the grid module operable to;

encode an outbound second data segment of the one or more outbound second data segments into a plurality of outbound encoded second data slices in accordance with the error coding dispersal storage function; and

decode a plurality of inbound encoded second data slices into an inbound second data segment of the one or more inbound second data segments in accordance with the error coding dispersal storage function; and

the storage module operable to;

output the plurality of outbound encoded second data slices to the plurality of DS storage units; and

receive the plurality of inbound encoded second data slices from the plurality of DS storage units.

9. A dispersed storage (DS) processing module comprises:

a gateway module operable to communicate data and corresponding information; and

an access module operable to;

segment outbound data of the data into one or more outbound data segments in accordance with a data storage protocol; and

aggregate one or more inbound data segments into inbound data of the data in accordance with the data storage protocol.

10. In the DS processing module of claim 9, wherein the gateway module function comprises at least one of:

message conversion;

protocol conversion;

electrical conversion;

optical conversion;

access control;

user identification;

user information retrieval;

traffic monitoring;

statistics generation;

configuration;

management; and

source name determination.

11. In the DS processing module of claim 9, wherein the data storage protocol comprises at least one of:

a file storage system;

a block storage system; and

an aggregated block storage system.

12. In the DS processing module of claim 9, wherein the corresponding information comprises at least one of:

a process identifier (ID);

metadata;

a file system directory;

a block number;

a transaction message;

a user device identity (ID);

a data object identifier;

a source name; and

user information.

13. The DS processing module of claim 9 further comprises:

the gateway module operable to communicate second data and second corresponding information, wherein at least a portion of the second corresponding information is substantially identical to at least a portion of the corresponding information; and

the access module operable to;

segment outbound second data of the second data into one or more outbound second data segments in accordance with the data storage protocol; and

aggregate one or more inbound second data segments into inbound second data of the second data in accordance with the data storage protocol.

14. A dispersed storage (DS) processing module comprises:

a grid module operable to;

encode an outbound data segment of one or more outbound data segments into a plurality of outbound encoded data slices in accordance with an error coding dispersal storage function; and

decode a plurality of inbound encoded data slices into an inbound data segment of one or more inbound data segments in accordance with the error coding dispersal storage function; and

a storage module operable to;

output the plurality of outbound encoded data slices to a plurality of DS storage units; and

receive the plurality of inbound encoded data slices from the plurality of DS storage units.

15. The DS processing module of claim 14 further comprises:

the grid module further operable to;

encode an outbound corresponding information segment of one or more outbound corresponding information segments into a plurality of outbound encoded corresponding information slices in accordance with the error coding dispersal storage function; and

decode a plurality of inbound encoded corresponding information slices into an inbound corresponding information segment of one or more inbound corresponding information segments in accordance with the error coding dispersal storage function; and

the storage module further operable to;

output the plurality of outbound encoded corresponding information slices to the plurality of DS storage units; and

receive the plurality of inbound encoded corresponding information slices from the plurality of DS storage units.

16. In the DS processing module of claim 14, wherein the error coding dispersal storage function includes one or more of:

a pre-slice data manipulation function and operational parameters of the pre-slice data manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

an encoding function and operational parameters of the encoding function including at least one of a pillar width, a read threshold, a write threshold, a caching setting, and a parallelism setting;

a slicing function and operational parameters of the slicing function including at least one of a pillar width, a caching setting, a slicing parameter, and a parallelism settings;

a post-slice data manipulation function and operational parameters of the post-slice data manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

a post-slice data de-manipulation function and operational parameters of the post-slice data de-manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting;

a de-slicing function and operational parameters of the de-slicing function including at least one of a pillar width, a caching setting, a slicing parameter, and a parallelism settings;

a decoding function and operational parameters of the decoding function including at least one of a pillar width, a read threshold, a write threshold, a caching setting, and a parallelism setting; and

a pre-slice data de-manipulation function and operational parameters of the pre-slice data de-manipulation function including at least one of a compression parameter, an integrity check method, a caching setting, an encryption parameter, and a parallelism setting.

17. The DS processing module of claim 14 further comprises:

the grid module further operable to;

generate a plurality of slice names corresponding to the plurality of outbound encoded data slices in accordance with the error coding dispersal storage function; and

the storage module further operable to;

identify the plurality of DS storage units based on the plurality of slice names.

18. The DS processing module of claim 14 further comprises:

the grid module operable to;

encode an outbound second data segment of one or more outbound second data segments into a plurality of outbound encoded second data slices in accordance with the error coding dispersal storage function; and

decode a plurality of inbound encoded second data slices into an inbound second data segment of one or more inbound second data segments in accordance with the error coding dispersal storage function; and

the storage module operable to;

output the plurality of outbound encoded second data slices to the plurality of DS storage units; and

receive the plurality of inbound encoded second data slices from the plurality of DS storage units.