Sequencing encoded data slices

• US 8,885,821 B2
• Filed: 11/28/2010
• Issued: 11/11/2014
• Est. Priority Date: 01/28/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Lewis, Lisa
• Assistant Examiner(s): Ullah, Sharif E
• Application Number:
  US12/954,875
• Publication Number:
  US 20110182424A1
• Time in Patent Office: 1,444 Days
• Field of Search: 380/45, 380/268, 380/36, 380/37, 380/43, 713/194, 713/189, 717/191, 726/16, 726/20, 726/27
• US Class Current: 380/45
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 21/72 in cryptographic circuits
• CPC Class Codes: G06F 21/80 in storage media based on magnetic or optical technology, e.g. disks with sectors preventing unauthorised reproduction or copying of disc-type recordable media G11B20/00
• CPC Class Codes: G06F 21/85 interconnection devices, e.g. bus-connected or in-line devices
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 2221/2107 File encryption
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 9/3242 involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC
• CPC Class Codes: H04N 21/222 Secondary servers, e.g. proxy server, cable television Head-end provisioning of proxy services in data packet switching networks H04L67/56
• CPC Class Codes: H04N 21/2347 involving video stream encryption
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04W 12/02 Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 12/033 of the user plane, e.g. user's traffic

Abstract

A method begins by a processing module receiving data segments of a data stream to produce received data segments. The method continues with the processing module encrypting a data segment of the received data segments to produce an encrypted data segment and dispersed storage error encoding the encrypted data segment to produce a set of encoded data slices in order of receiving the data segments. The method continues with the processing module buffering encoded data slices of sets of the encoded data slices unit to produce buffered encoded data slices and comparing a number of buffered encoded data slices to a threshold. The method continues with the processing module outputting the encoded data slices of the buffered encoded data slices based on a pseudo-random sequencing order when the number of buffered encoded data slices compares favorably to the threshold.

Claims

1. A method for securing transmission of a data stream, the method comprises:

receiving a data file or a data block;

partitioning the data file or data block into a plurality of data segments;

encrypting respective data segments individually to produce a plurality of encrypted data segments;

error encoding respective encrypted data segments to produce N number of encoded data slices for each encrypted data segment, wherein when a particular N number of encoded data slices for a particular encrypted data segment are dispersed separately in distributed storage, only a read threshold number T of the particular N number of encoded data slices are need to be retrieved to reconstruct the particular encrypted data segment, where T is less than N;

commencing storing of the encoded data slices of the plurality of encrypted data segments into a storage device; and

monitoring the storing of the encoded data slices and when a number of encoded data slices reach a threshold value that encompasses data slices of more than one encrypted data segment, generating an output of the stored encoded data slices in a pseudo-random sequencing order that reorders the stored slices of the more than one encrypted data segment stored in the storage device.

2. The method of claim 1, wherein when partitioning the data file or the data block includes:

determining segment sequencing information; and

sequencing the data segments in accordance with the segment sequencing information to produce the data segments.

3. The method of claim 1, wherein when generating the output of the stored encoded data slices comprises:

converting the stored encoded data slices into a symbol stream in accordance with a wireless communication protocol;

converting the symbol stream into an outbound radio frequency (RF) signal in accordance with the wireless communication protocol; and

transmitting the outbound RF signal.

4. The method of claim 1 further comprises:

appending the pseudo-random sequencing order or an encoded sequencing order to at least one of the stored encoded data slices, where the pseudo-random sequencing order is error encoded to produce the encoded sequencing order.

5. The method of claim 1, wherein the encrypting of the respective data segments comprises at least one of:

encrypting the respective data segments utilizing an all or nothing transformation (AONT) to produce the encrypted data segments;

encrypting the respective data segments utilizing a stored encryption key to produce the encrypted data segments;

encrypting the respective data segments utilizing a random encryption key to produce the encrypted data segments;

encrypting the respective data segments utilizing an encryption key associated with at least one recipient to produce the encrypted data segments;

encrypting the respective data segments utilizing the AONT to produce AONT encrypted data segments and encrypting the AONT encrypted data segments utilizing a stored encryption key to produce the encrypted data segments;

obfuscating the respective data segments utilizing an obfuscating method to produce the encrypted data segments;

selecting an encryption method based on a data type to produce a selected encryption method and encrypting the respective data segments utilizing the selected encryption method to produce the encrypted data segments; and

encrypting the respective data segments utilizing a null encryption method to produce the encrypted data segments.

6. The method of claim 1, wherein the pseudo-random sequencing order comprises at least one of:

a sequencing algorithm identifier (ID);

a segment sequencing algorithm ID;

a slice sequencing algorithm ID;

an algorithm seed;

a seed generation algorithm ID;

an encryption key;

a hash algorithm ID;

segment ordering information; and

slice ordering information.

7. The method of claim 1 further comprises:

determining the pseudo-random sequencing order based on at least one of;

a requester identifier (ID);

a vault lookup;

a data object name;

the data;

received sequencing information;

error coding dispersal storage function parameters;

executing a sequencing algorithm utilizing an algorithm seed;

a priority indicator;

a security indicator;

a performance indicator;

a list;

a command;

a predetermination; and

a message.

8. The method of claim 1 further comprises:

when the number of stored encoded data slices compares favorably to the threshold value, comparing, within a given time interval, an ingress number of encoded data slices being stored to an egress number of encoded data slices being outputted;

when the comparing of the ingress number to the egress number is unfavorable with respect to an underflow threshold, adjusting at least one of the encrypting the data segments and the error encoding of the encrypted data segments, such that the comparing of the ingress number to the egress number is favorable with respect to the underflow threshold; and

when the comparing of the ingress number to the egress number is unfavorable with respect to an overflow threshold, adjusting at least one of the encrypting the data segments and the error encoding of the encrypted data segments, such that the comparing of the ingress number to the egress number is favorable with respect to the overflow threshold.

9. The method of claim 1 further comprises:

generating a slice name for each of the encoded data slices to produce a set of slice names;

storing the set of slice names; and

when a number of stored encoded data slices compares favorably to the threshold value, outputting the slice names based on the pseudo-random sequencing order.

10. The method of claim 9, wherein the generating the slice names comprises:

generating corresponding slice information; and

encrypting the slice information to produce the slice names.

11. A computer comprises:

an interface;

a storage device; and

a processing module, having a processor, operable to;

receive, via the interface, a data file or a data block;

partition the data file or data block into a plurality of data segments;

encrypt respective data segments individually to produce a plurality of encrypted data segments;

error encode respective encrypted data segments to produce N number of encoded data slices for each encrypted data segment, wherein when a particular N number of encoded data slices for a particular encrypted data segment are dispersed separately in distributed storage, only a read threshold number T of the particular N number of encoded data slices are need to be retrieved to reconstruct the particular encrypted data segment, where T is less than N;

commence storing of the encoded data slices of the plurality of encrypted data segments into the storage device; and

monitor the storing of the encoded data slices and when a number of encoded data slices reach a threshold value that encompasses data slices of more than one encrypted data segment, to generate an output of the stored encoded data slices in a pseudo-random sequencing that reorders the stored slices of the more than one encrypted data segment stored in the storage device.

12. The computer of claim 11, wherein the processing module further functions to:

determine segment sequencing information; and

sequence the data segments in accordance with the segment sequencing information to produce the data segments.

13. The computer of claim 11, wherein the processing module further functions to generate the output the stored encoded data slices by:

converting the stored encoded data slices into a symbol stream in accordance with a wireless communication protocol;

converting the symbol stream into an outbound radio frequency (RF) signal in accordance with the wireless communication protocol; and

transmitting the outbound RF signal.

14. The computer of claim 11, wherein the processing module further functions to:

append the pseudo-random sequencing order or an encoded sequencing order to at least one of the stored encoded data slices, where the pseudo-random sequencing order is error encoded to produce the encoded sequencing order.

15. The computer of claim 11, wherein the processing module further functions to encrypt the respective data segments by at least one of:

encrypting the respective data segments utilizing an all or nothing transformation (AONT) to produce the encrypted data segments;

encrypting the respective data segments utilizing a stored encryption key to produce the encrypted data segments;

encrypting the respective data segments utilizing a random encryption key to produce the encrypted data segments;

encrypting the respective data segments utilizing an encryption key associated with at least one recipient to produce the encrypted data segments;

encrypting the respective data segments utilizing the AONT to produce AONT encrypted data segments and encrypting the AONT encrypted data segments utilizing a stored encryption key to produce the encrypted data segments;

obfuscating the respective data segments utilizing an obfuscating method to produce the encrypted data segments;

selecting an encryption method based on a data type to produce a selected encryption method and encrypting the respective data segments utilizing the selected encryption method to produce the encrypted data segments; and

encrypting the respective data segments utilizing a null encryption method to produce the encrypted data segments.

16. The computer of claim 11, wherein the pseudo-random sequencing order comprises at least one of:

a sequencing algorithm identifier (ID);

a segment sequencing algorithm ID;

a slice sequencing algorithm ID;

an algorithm seed;

a seed generation algorithm ID;

an encryption key;

a hash algorithm ID;

segment ordering information; and

slice ordering information.

17. The computer of claim 11, wherein the processing module further functions to:

determine the pseudo-random sequencing order based on at least one of;

a requester identifier (ID);

a vault lookup;

a data object name;

the data;

received sequencing information;

error coding dispersal storage function parameters;

executing a sequencing algorithm utilizing an algorithm seed;

a priority indicator;

a security indicator;

a performance indicator;

a list;

a command;

a predetermination; and

a message.

18. The computer of claim 11, wherein the processing module further functions to:

compare, within a given time interval, an ingress number of encoded data slices being stored to an egress number of encoded data slices being outputted when a number of the stored encoded data slices compares favorably to the threshold value;

adjust at least one of the encrypting the data segments and the error encoding of the encrypted data segments, such that the comparing of the ingress number to the egress number is favorable with respect to an underflow threshold; and

adjust at least one of the encrypting the data segments and the error encoding of the encrypted data segments, such that the comparing of the ingress number to the egress number is favorable with respect to an overflow threshold.

19. The computer of claim 11, wherein the processing module further functions to:

generate a slice name for each of the encoded data slices to produce a set of slice names;

store the set of slice names; and

output, via the interface, the slice names based on the pseudo-random sequencing order when a number of stored encoded data slices compares favorably to the threshold value.

20. The computer of claim 19, wherein the processing module further functions to generate the slice names by:

generating corresponding slice information; and

encrypting the slice information to produce the slice name.