DISPERSAL OF PRIORITY DATA IN A DISPERSED STORAGE NETWORK

• US 20110228931A1
• Filed: 12/31/2010
• Published: 09/22/2011
• Est. Priority Date: 03/16/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Markison, Timothy W., Grube, Gary W.
• Granted Patent: US 8,938,013 B2
• Time in Patent Office: Days
• US Class Current: 380/201
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H03M 13/35 Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H04N 21/2181 comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097

Abstract

A method begins by a processing module encoding data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters and determining priority data segments of the data. The method continues with the processing module encoding the priority data segments in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of priority encoded data slices. The method continues with the processing module outputting the plurality of sets of encoded data slices and the plurality of sets of priority encoded data slices to a dispersed storage network memory for storage therein.

Claims

1. A method comprises:

encoding data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters;

determining priority data segments of the data;

encoding the priority data segments in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of priority encoded data slices, wherein the first error coding dispersal storage function parameters include a first pillar width and a first decode threshold optimized for data recovery speed and non-optimal for data recovery reliability and the second error coding dispersal storage function parameters include a second pillar width and a second decode threshold optimized for data recovery reliability and non-optimal for data recovery speed; and

outputting the plurality of sets of encoded data slices and the plurality of sets of priority encoded data slices to a dispersed storage network (DSN) memory for storage therein.

2. The method of claim 1 further comprises at least one of:

updating a storage location table to associate the plurality of sets of encoded data slices with the data; and

updating the storage location table to associate the plurality of sets of priority encoded data slices with the priority data segments.

3. The method of claim 1, wherein the determining priority data segments of the data comprises at least one of:

identifying content of the data having a desired priority level, wherein data segments containing the content are identified as the priority data segments;

determining a desired relationship between the data recovery speed and the data recovery reliability, and determining the priority data segments based on the desired relationship;

receiving an indicator that identifies the priority data segments; and

accessing a table regarding the data to identify the priority data segments.

4. The method of claim 1 further comprises:

encoding video data into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters, wherein the plurality of set of encoded data slices is retrieved in response to a video on demand request; and

determining key frames of the video data as the priority data segments.

5. The method of claim 1, wherein the encoding the data into a plurality of sets of encoded data slices further comprises:

identifying non-priority data segments of the data; and

encoding the non-priority data segments into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters.

6. A method comprises:

retrieving a set of encoded data slices from a dispersed storage network (DSN) memory, wherein a data segment was encoded in accordance with first error coding dispersal storage function parameters to produce the set of encoded data slices;

determining whether the data segment is a priority data segment;

when the data segment is a priority data segment, determining whether to decode the set of encoded data slices in accordance with the first error coding dispersal storage function parameters or to decode a second set of encoded data slices in accordance with second error coding dispersal storage function parameters, wherein the data segment was encoded in accordance with the second error coding dispersal storage function parameters to produce the second set of encoded data slices, wherein the first error coding dispersal storage function parameters include a first pillar width and a first decode threshold optimized for data recovery speed and non-optimal for data recovery reliability and the second error coding dispersal storage function parameters include a second pillar width and a second decode threshold optimized for data recovery reliability and non-optimal for data recovery speed; and

when it is determined to decode the second set of encoded data slices;

retrieving the second set of encoded data slices from the DSN memory; and

decoding the second set of encoded data slices in accordance with the second error coding dispersal storage function parameters to produce a recovered data segment.

7. The method of claim 6 further comprises at least one of:

accessing a storage location table to identify the set of encoded data slices with the data; and

accessing the storage location table to identify the second set of encoded data slices.

8. The method of claim 6, wherein the determining whether to decode the set of encoded data slices or the second set of encoded data slices comprises at least one of:

determining a desired relationship between the data recovery speed and the data recovery reliability, decoding the set of encoded data slices when the desired relationship compares favorably to a relationship threshold, and decoding the second set of encoded data slices when the desired relationship compares unfavorably to the relationship threshold;

receiving an indicator that indicates whether to decode the set of encoded data slices or the second set of encoded data slices; and

accessing a table based on identity of the data segment to determine whether to decode the set of encoded data slices or the second set of encoded data slices.

9. The method of claim 6 further comprises:

retrieving the set of encoded data slices in response to a video on demand request;

determining whether the data segment corresponds to a key frame of video data; and

when the data segment corresponds to the key frame;

identifying the data segment as a priority data segment; and

determining to decode the second set of encoded data slices.

10. The method of claim 6 further comprises:

when the data segment is not the priority data segment, decoding the first set of encoded data slices.

11. A computer comprises:

an interface;

a memory; and

a processing module operable to;

encode data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters;

determine priority data segments of the data;

encode the priority data segments in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of priority encoded data slices, wherein the first error coding dispersal storage function parameters include a first pillar width and a first decode threshold optimized for data recovery speed and non-optimal for data recovery reliability and the second error coding dispersal storage function parameters include a second pillar width and a second decode threshold optimized for data recovery reliability and non-optimal for data recovery speed; and

output, via the interface, the plurality of sets of encoded data slices and the plurality of sets of priority encoded data slices to a dispersed storage network (DSN) memory for storage therein.

12. The computer of claim 11, wherein the processing module further functions to:

update a storage location table to associate the plurality of sets of encoded data slices with the data; and

update the storage location table to associate the plurality of sets of priority encoded data slices with the priority data segments.

13. The computer of claim 11, wherein the processing module functions to determine the priority data segments of the data by at least one of:

identifying content of the data having a desired priority level, wherein data segments containing the content are identified as the priority data segments;

determining a desired relationship between the data recovery speed and the data recovery reliability, and determining the priority data segments based on the desired relationship;

receiving, via the interface, an indicator that identifies the priority data segments; and

accessing a table regarding the data to identify the priority data segments.

14. The computer of claim 11, wherein the processing module further functions to:

encode video data into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters, wherein the plurality of set of encoded data slices is retrieved in response to a video on demand request; and

determine key frames of the video data as the priority data segments.

15. The computer of claim 11, wherein the processing module further functions to encode the data into the plurality of sets of encoded data slices by:

identifying non-priority data segments of the data; and

encoding the non-priority data segments into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters.

16. A computer comprises:

an interface;

a memory; and

a processing module operable to;

retrieve, via the interface, a set of encoded data slices from a dispersed storage network (DSN) memory, wherein a data segment was encoded in accordance with first error coding dispersal storage function parameters to produce the set of encoded data slices;

determine whether the data segment is a priority data segment;

determine whether to decode the set of encoded data slices in accordance with the first error coding dispersal storage function parameters or to decode a second set of encoded data slices in accordance with second error coding dispersal storage function parameters when the data segment is a priority data segment, wherein the data segment was encoded in accordance with the second error coding dispersal storage function parameters to produce the second set of encoded data slices, wherein the first error coding dispersal storage function parameters include a first pillar width and a first decode threshold optimized for data recovery speed and non-optimal for data recovery reliability and the second error coding dispersal storage function parameters include a second pillar width and a second decode threshold optimized for data recovery reliability and non-optimal for data recovery speed; and

when it is determined to decode the second set of encoded data slices;

retrieve, via the interface, the second set of encoded data slices from the DSN memory; and

decode the second set of encoded data slices in accordance with the second error coding dispersal storage function parameters to produce a recovered data segment.

17. The computer of claim 16, wherein the processing module further functions to:

access a storage location table to identify the set of encoded data slices with the data; and

access the storage location table to identify the second set of encoded data slices.

18. The computer of claim 16, wherein the processing module further functions to determine whether to decode the set of encoded data slices or the second set of encoded data slices by at least one of:

determining a desired relationship between the data recovery speed and the data recovery reliability, decoding the set of encoded data slices when the desired relationship compares favorably to a relationship threshold, and decoding the second set of encoded data slices when the desired relationship compares unfavorably to the relationship threshold;

receiving, via the interface, an indicator that indicates whether to decode the set of encoded data slices or the second set of encoded data slices; and

accessing a table based on identity of the data segment to determine whether to decode the set of encoded data slices or the second set of encoded data slices.

19. The computer of claim 16, wherein the processing module further functions to:

retrieve the set of encoded data slices in response to a video on demand request;

determine whether the data segment corresponds to a key frame of video data; and

when the data segment corresponds to the key frame;

identify the data segment as a priority data segment; and

determine to decode the second set of encoded data slices.

20. The computer of claim 16, wherein the processing module further functions to:

decode the first set of encoded data slices when the data segment is not the priority data segment.