Data transmission utilizing route selection and dispersed storage error encoding

• US 8,839,068 B2
• Filed: 12/16/2013
• Issued: 09/16/2014
• Est. Priority Date: 10/06/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W., Gladwin, S. Christopher, Dhuse, Greg, Baptist, Andrew, Volvovski, Ilya, Resch, Jason K.
• Primary Examiner(s): Rizk, Sam
• Application Number:
  US14/107,204
• Publication Number:
  US 20140108882A1
• Time in Patent Office: 274 Days
• Field of Search: 714/752, 714/755, 714/758, 714/763, 711/118, 380/201
• US Class Current: 714/752
• CPC Class Codes: G06F 11/1028 Adjacent errors, e.g. error in n-bit (n>1) wide storage units, i.e. package error
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/29 combining two or more codes or code structures, e.g. product codes, generalised product codes, concatenated codes, inner and outer codes
• CPC Class Codes: H03M 13/373 with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 45/24 Multipath

Abstract

A method begins by a processing module obtaining a set of encoded data slices for transmission to a receiving entity via a network, wherein the set of encoded data slices represents data that has been dispersed storage error encoded. The method continues with the processing module dividing the set into a plurality of sub-sets of encoded data slices in accordance with an error coding distributed routing protocol. The method continues with the processing module determining a plurality of routing paths within the network in accordance with the error coding distributed routing protocol. The method continues with the processing module transmitting the plurality of sub-sets of encoded data slices via the plurality of routing paths to the receiving entity in accordance with the error coding distributed routing protocol.

Claims

1. A method for execution by a processing module, the method comprises:

dividing data into different priority level portions in accordance with a data partitioning scheme;

selecting first error coding dispersal storage function parameters for first priority level portions of the data;

selecting second error coding dispersal storage function parameters for second priority level portions of the data, wherein the first priority level portions having a higher priority than the second priority level portions and the first error coding dispersal storage function parameters provides a higher level of dispersed storage error encoding than the second error coding dispersal storage function parameters;

dispersed storage error encoding the first priority level portions of the data in accordance with the first error coding dispersal storage function parameters to produce a first plurality of encoded data slices;

dispersed storage error encoding the second priority level portions of the data in accordance with the second error coding dispersal storage function parameters to produce a second plurality of encoded data slices; and

outputting the first and second plurality of encoded data slices.

2. The method of claim 1, wherein the dividing data into different priority level portions comprises:

dividing the data into a plurality of data segments in accordance with a data type; and

dividing the plurality of data segments into a first priority level set of data segments and a second priority level set of data segments in accordance with data segment content priority that includes at a fundamental data component and an enhancement data component.

3. The method of claim 1, wherein the dividing data into different priority level portions comprises:

dividing the data into a plurality of data segments in accordance with a data type, wherein a data segment of the plurality of data segments includes first priority data and second priority data;

combining the first priority data of two or more data segments of the plurality of data segments to produce one of the first priority level portions; and

combining the second priority data of the two or more data segments of the plurality of data segments to produce one of the second priority level portions.

4. The method of claim 1 further comprises:

determining the data partitioning scheme based on one or more of a data type indicator, a data analysis, a data size indicator, a priority indicator, a security indicator, buffer bits, authentication indicator, performance indicator, a lookup, a message, and a predetermination.

5. The method of claim 1, wherein the selecting the first and second error coding dispersal storage function parameters comprises:

determining each of the first and second error coding dispersal storage function parameters based on one or more of a routing path quality of service information, candidate routing paths, selected routing paths, a communications requirement, a data type indicator, a capacity estimate of the selected routing paths, a priority indicator, a security indicator, a performance indicator, an estimated routing path performance indicator, a lookup, and a message.

6. The method of claim 1, wherein the dividing data into different priority level portions comprises:

dividing, as the data, video data into standard definition video portions and high definition video portions, wherein the standard definition video portions are the first priority level portions of the data and the high definition video portions are the second priority level portions of the data.

7. The method of claim 1, wherein the dividing data into different priority level portions comprises:

dividing, as the data, video data into base frames of video and change frames of video, wherein the base frames of video are the first priority level portions of the data and the change frames of video are the second priority level portions of the data.

8. The method of claim 1, wherein the dividing data into different priority level portions comprises:

dividing, as the data, audio data into fundamental audio data components and enhancement audio data components, wherein the fundamental audio data components are the first priority level portions of the data and the enhancement audio data components are the second priority level portions of the data.

9. The method of claim 1, wherein the outputting the first and second plurality of encoded data slices comprises:

selecting routing paths from a plurality of routing paths based on one or more of communication requirements for transmitting the data and routing path quality of service information; and

transmitting the first and second plurality of encoded data slices via the selected routing paths.

10. A dispersed storage module comprises:

a first module, when operable within a computer, causes the computer to divide data into different priority level portions in accordance with a data partitioning scheme;

a second module, when operable within the computer, causes the computer to;

select first error coding dispersal storage function parameters for first priority level portions of the data;

select second error coding dispersal storage function parameters for second priority level portions of the data, wherein the first priority level portions having a higher priority than the second priority level portions and the first error coding dispersal storage function parameters provides a higher level of dispersed storage error encoding than the second error coding dispersal storage function parameters;

a third module, when operable within the computer, causes the computer to;

dispersed storage error encode the first priority level portions of the data in accordance with the first error coding dispersal storage function parameters to produce a first plurality of encoded data slices; and

dispersed storage error encode the second priority level portions of the data in accordance with the second error coding dispersal storage function parameters to produce a second plurality of encoded data slices; and

a fourth module, when operable within the computer, causes the computer to output the first and second plurality of encoded data slices.

11. The dispersed storage module of claim 10, wherein the first module further causes the computer to divide the data into different priority level portions by:

dividing the data into a plurality of data segments in accordance with a data type; and

dividing the plurality of data segments into a first priority level set of data segments and a second priority level set of data segments in accordance with data segment content priority that includes at a fundamental data component and an enhancement data component.

12. The dispersed storage module of claim 10, wherein the first module further causes the computer to divide the data into different priority level portions by:

dividing the data into a plurality of data segments in accordance with a data type, wherein a data segment of the plurality of data segments includes first priority data and second priority data;

combining the first priority data of two or more data segments of the plurality of data segments to produce one of the first priority level portions; and

combining the second priority data of the two or more data segments of the plurality of data segments to produce one of the second priority level portions.

13. The dispersed storage module of claim 10, wherein the first module further causes the computer to:

determine the data partitioning scheme based on one or more of a data type indicator, a data analysis, a data size indicator, a priority indicator, a security indicator, buffer bits, authentication indicator, performance indicator, a lookup, a message, and a predetermination.

14. The dispersed storage module of claim 10, wherein the second module further causes the computer to select the first and second error coding dispersal storage function parameters by:

determining each of the first and second error coding dispersal storage function parameters based on one or more of a routing path quality of service information, candidate routing paths, selected routing paths, a communications requirement, a data type indicator, a capacity estimate of the selected routing paths, a priority indicator, a security indicator, a performance indicator, an estimated routing path performance indicator, a lookup, and a message.

15. The dispersed storage module of claim 10, wherein the first module further causes the computer to divide the data into different priority level portions by:

dividing, as the data, video data into standard definition video portions and high definition video portions, wherein the standard definition video portions are the first priority level portions of the data and the high definition video portions are the second priority level portions of the data.

16. The dispersed storage module of claim 10, wherein the first module further causes the computer to divide the data into different priority level portions by:

dividing, as the data, video data into base frames of video and change frames of video, wherein the base frames of video are the first priority level portions of the data and the change frames of video are the second priority level portions of the data.

17. The dispersed storage module of claim 10, wherein the first module further causes the computer to divide the data into different priority level portions by:

dividing, as the data, audio data into fundamental audio data components and enhancement audio data components, wherein the fundamental audio data components are the first priority level portions of the data and the enhancement audio data components are the second priority level portions of the data.

18. The dispersed storage module of claim 10, wherein the third module further causes the computer to output the first and second plurality of encoded data slices by:

selecting routing paths from a plurality of routing paths based on one or more of communication requirements for transmitting the data and routing path quality of service information; and

transmitting the first and second plurality of encoded data slices via the selected routing paths.