DATA TRANSMISSION UTILIZING DATA PROCESSING AND DISPERSED STORAGE ERROR ENCODING

• US 20120089887A1
• Filed: 10/03/2011
• Published: 04/12/2012
• Est. Priority Date: 10/06/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: DHUSE, GREG, BAPTIST, ANDREW, VOLVOVSKI, ILYA, GRUBE, GARY W., MARKISON, TIMOTHY W., GLADWIN, S. CHRISTOPHER, RESCH, JASON K.
• Granted Patent: US 8,918,693 B2
• Time in Patent Office: Days
• US Class Current: 714/752
• CPC Class Codes: G06F 11/1028 Adjacent errors, e.g. error in n-bit (n>1) wide storage units, i.e. package error
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H03M 13/29 combining two or more codes or code structures, e.g. product codes, generalised product codes, concatenated codes, inner and outer codes
• CPC Class Codes: H03M 13/373 with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 45/24 Multipath

Abstract

A method begins by a processing module dispersed storage error encoding fundamental component data of data in accordance with dispersed storage error coding parameters to produce a plurality of sets of encoded data slices, wherein the data includes the fundamental component data and enhancement component data. The method continues with the processing module transmitting a set of the plurality of sets of encoded data slices and transmitting a corresponding portion of the enhancement component data substantially concurrently with the transmitting of the set of the plurality of sets of encoded data slices.

Claims

1. A method comprises:

dispersed storage error encoding fundamental component data of data in accordance with dispersed storage error coding parameters to produce a plurality of sets of encoded data slices, wherein the data includes the fundamental component data and enhancement component data;

transmitting a set of the plurality of sets of encoded data slices; and

transmitting a corresponding portion of the enhancement component data substantially concurrently with the transmitting of the set of the plurality of sets of encoded data slices.

2. The method of claim 1 further comprises:

processing the data in accordance with a processing function to produce the fundamental component data and the enhancement component data.

3. The method of claim 2, wherein the processing function comprises one or more of:

a data component priority segregation function;

a spectral analysis function; and

a data filter function.

4. The method of claim 1 further comprises:

the fundamental component data including first level of fundamental component data and second level of fundamental component data;

the dispersed storage error coding parameters including first dispersed storage error coding parameters and second dispersed storage error coding parameters;

the plurality of sets of encoded data slices including a first plurality of sets of encoded data slices and a second plurality of sets of encoded data slices;

dispersed storage error encoding the first fundamental component data in accordance with the first dispersed storage error coding parameters to produce the first plurality of sets of encoded data slices; and

dispersed storage error encoding the second fundamental component data in accordance with the second dispersed storage error coding parameters to produce the second plurality of sets of encoded data slices.

5. The method of claim 1 further comprises:

the enhancement component data including a first level of enhancement component data and a second level of enhancement component data.

6. The method of claim 1 further comprises:

determining the dispersed storage error coding parameters based on at least one of metadata associated with the data and metadata associated with the fundamental component data.

7. The method of claim 1 further comprises:

dispersed storage error encoding the enhancement component data in accordance with second dispersed storage error coding parameters to produce a second plurality of sets of encoded data slices; and

transmitting, as the corresponding portion of the enhancement component data, a set of the second set of encoded data slices.

8. The method of claim 1 further comprises:

the data including video data; and

processing the video data in accordance with a video processing function to produce the fundamental component data and the enhancement component data.

9. A method comprises:

receiving sets of encoded data slices, wherein fundamental component data of data was encoded in accordance with dispersed storage error coding parameters to produce the sets of encoded data slices, wherein the data includes the fundamental component data and enhancement component data;

receiving corresponding portions of the enhancement component data substantially concurrently with the receiving of the sets of encoded data slices;

dispersed storage error decoding the sets of encoded data slices to reproduce the fundamental component data; and

processing the fundamental component data and the enhancement component data in accordance with a processing function to reproduce the data.

10. The method of claim 9, wherein the processing function comprises one or more of:

a data component priority segregation function;

a spectral analysis function; and

a data filter function.

11. The method of claim 9 further comprises:

the fundamental component data including first level of fundamental component data and second level of fundamental component data;

the sets of encoded data slices including a first plurality of sets of encoded data slices and a second plurality of sets of encoded data slices;

dispersed storage error decoding the first plurality of sets of encoded data slices in accordance with first dispersed storage error coding parameters to reproduce the first level of fundamental component data; and

dispersed storage error decoding the second plurality of sets of encoded data slices in accordance with second dispersed storage error coding parameters to reproduce the second level of fundamental component data.

12. The method of claim 9 further comprises:

receiving, as the enhancement component data, a second plurality of sets of encoded data slices; and

dispersed storage error decoding the enhancement component data in accordance with dispersed storage error coding parameters to reproduce the enhancement component data.

13. The method of claim 9 further comprises:

the data including video data; and

processing the fundamental component data and the enhancement component data in accordance with a video processing function to reproduce the video data.

14. A computer comprises:

an interface;

a memory; and

a processing module operable to;

dispersed storage error encode fundamental component data of data in accordance with dispersed storage error coding parameters to produce a plurality of sets of encoded data slices, wherein the data includes the fundamental component data and enhancement component data;

transmit, via the interface, a set of the plurality of sets of encoded data slices; and

transmit, via the interface, a corresponding portion of the enhancement component data substantially concurrently with the transmitting of the set of the plurality of sets of encoded data slices.

15. The computer of claim 14, wherein the processing module further functions to:

process the data in accordance with a processing function to produce the fundamental component data and the enhancement component data.

16. The computer of claim 15, wherein the processing function comprises one or more of:

a data component priority segregation function;

a spectral analysis function; and

a data filter function.

17. The computer of claim 14 further comprises:

the fundamental component data including first level of fundamental component data and second level of fundamental component data;

the dispersed storage error coding parameters including first dispersed storage error coding parameters and second dispersed storage error coding parameters;

the plurality of sets of encoded data slices including a first plurality of sets of encoded data slices and a second plurality of sets of encoded data slices; and

the processing module further functions to;

dispersed storage error encode the first fundamental component data in accordance with the first dispersed storage error coding parameters to produce the first plurality of sets of encoded data slices; and

dispersed storage error encode the second fundamental component data in accordance with the second dispersed storage error coding parameters to produce the second plurality of sets of encoded data slices.

18. The computer of claim 14 further comprises:

the enhancement component data including a first level of enhancement component data and a second level of enhancement component data.

19. The computer of claim 14, wherein the processing module further functions to:

determine the dispersed storage error coding parameters based on at least one of metadata associated with the data and metadata associated with the fundamental component data.

20. The computer of claim 14, wherein the processing module further functions to:

dispersed storage error encode the enhancement component data in accordance with second dispersed storage error coding parameters to produce a second plurality of sets of encoded data slices; and

transmit, via the interface, as the corresponding portion of the enhancement component data, a set of the second set of encoded data slices.

21. The computer of claim 14 further comprises:

the data including video data; and

the processing module further functions to;

process the video data in accordance with a video processing function to produce the fundamental component data and the enhancement component data.

22. A computer comprises:

an interface;

a memory; and

a processing module operable to;

receive, via the interface, sets of encoded data slices, wherein fundamental component data of data was encoded in accordance with dispersed storage error coding parameters to produce the sets of encoded data slices, wherein the data includes the fundamental component data and enhancement component data;

receive, via the interface, corresponding portions of the enhancement component data substantially concurrently with the receiving of the sets of encoded data slices;

dispersed storage error decode the sets of encoded data slices to reproduce the fundamental component data; and

process the fundamental component data and the enhancement component data in accordance with a processing function to reproduce the data.

23. The computer of claim 22, wherein the processing function comprises one or more of:

a data component priority segregation function;

a spectral analysis function; and

a data filter function.

24. The computer of claim 22 further comprises:

the fundamental component data including first level of fundamental component data and second level of fundamental component data;

the sets of encoded data slices including a first plurality of sets of encoded data slices and a second plurality of sets of encoded data slices; and

the processing module further functions to;

dispersed storage error decode the first plurality of sets of encoded data slices in accordance with first dispersed storage error coding parameters to reproduce the first level of fundamental component data; and

dispersed storage error decode the second plurality of sets of encoded data slices in accordance with second dispersed storage error coding parameters to reproduce the second level of fundamental component data.

25. The computer of claim 22, wherein the processing module further functions to:

receive, via the interface, as the enhancement component data, a second plurality of sets of encoded data slices; and

dispersed storage error decode the enhancement component data in accordance with dispersed storage error coding parameters to reproduce the enhancement component data.

26. The computer of claim 22 further comprises:

the data including video data; and

the processing module further functions to;

process the fundamental component data and the enhancement component data in accordance with a video processing function to reproduce the video data.