Dispersal of priority data in a dispersed storage network

• US 8,938,013 B2
• Filed: 12/31/2010
• Issued: 01/20/2015
• Est. Priority Date: 03/16/2010
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Grube, Gary W., Markison, Timothy W.
• Primary Examiner(s): Abrishamkar, Kaveh
• Assistant Examiner(s): PARSONS, THEODORE C
• Application Number:
  US12/983,209
• Publication Number:
  US 20110228931A1
• Time in Patent Office: 1,481 Days
• Field of Search: 726/1, 713/189, 380/201, 375/240.27
• US Class Current: 375/240.27
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: H03M 13/35 Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
• CPC Class Codes: H03M 13/356 Unequal error protection [UEP]
• CPC Class Codes: H04N 21/2181 comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097

Abstract

A processing module encodes data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters optimized for data recovery speed and non-optimal for data recovery reliability. The processing module encodes priority data segments of the data in accordance with second error coding dispersal storage function parameters to produce a plurality of sets of priority encoded data slices optimized for data recovery reliability and non-optimal for data recovery speed. The module outputs the plurality of sets of encoded data slices and the plurality of sets of priority encoded data slices to a dispersed storage network memory for storage therein.

Claims

1. A method comprises:

encoding data segments of data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters, wherein the data includes non-priority data portions and priority data portions, wherein data boundaries of the non-priority data portions and priority data portions are not aligned with boundaries of the data segments;

determining priority data segments of the data segments, wherein one of the priority data segments includes at least a fraction of one of the priority data portions;

redundantly encoding the priority data segments in accordance with second error coding dispersal storage function parameters to produce sets of priority encoded data slices, wherein the first error coding dispersal storage function parameters prioritize data recovery speed over data recovery reliability and the second error coding dispersal storage function parameters prioritize data recovery reliability over data recovery speed, wherein a substantially identical representation of a pre-encoded priority data segment of the priority data segments is recoverable from a corresponding set of the plurality of sets of encoded data slices or from a corresponding set of the sets of priority encoded data slices; and

outputting the plurality of sets of encoded data slices and the sets of priority encoded data slices to a dispersed storage network (DSN) memory for storage therein.

2. The method of claim 1 further comprises at least one of:

updating a storage location table to associate the plurality of sets of encoded data slices with the data; and

updating the storage location table to associate the sets of priority encoded data slices with the priority data segments.

3. The method of claim 1, wherein the determining priority data segments of the data comprises at least one of:

identifying content of the data having a desired priority level, wherein data segments containing the content are identified as the priority data segments;

determining a desired relationship between the data recovery speed and the data recovery reliability, and determining the priority data segments based on the desired relationship;

receiving an indicator that identifies the priority data segments; and

accessing a table regarding the data to identify the priority data segments.

4. The method of claim 1 further comprises:

encoding video data into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters, wherein the plurality of set of encoded data slices is retrieved in response to a video on demand request; and

determining key frames of the video data as the priority data segments.

5. A method comprises:

retrieving a set of encoded data slices of a data segment from a dispersed storage network (DSN) memory, wherein the data segment was encoded in accordance with first error coding dispersal storage function parameters to produce a the set of encoded data slices;

when a substantially identical representation of a pre-encoded data segment cannot be recovered from the set of encoded data slices;

determining whether the data segment is a priority data segment and, when the data segment is a priority data segment, the data segment is redundantly encoded in accordance with a second error coding dispersed storage function parameters to produce a second set of encoded data slices; and

when the data segment is the priority data segment;

retrieving the second set of encoded data slices from the DSN memory; and

decoding the second set of encoded data slices in accordance with the second error coding dispersal storage function parameters to recapture the substantially identical representation of the pre-encoded data segment.

6. The method of claim 5 further comprises at least one of:

accessing a storage location table to identify the set of encoded data slices with the data; and

accessing the storage location table to identify the second set of encoded data slices.

7. The method of claim 5 further comprises:

retrieving the set of encoded data slices in response to a video on demand request;

determining whether the data segment corresponds to a key frame of video data; and

when the data segment corresponds to the key frame;

identifying the data segment as a priority data segment.

8. A computer comprises:

an interface;

a memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

encode data segments of data into a plurality of sets of encoded data slices in accordance with first error coding dispersal storage function parameters, wherein the data includes non-priority data portions and priority data portions, wherein data boundaries of the non-priority data portions and priority data portions are not aligned with boundaries of the data segments;

determine priority data segments of the data segments, wherein one of the priority data segments includes at least a fraction of one of the priority data portions;

redundantly encode the priority data segments in accordance with second error coding dispersal storage function parameters to produce sets of priority encoded data slices, wherein the first error coding dispersal storage function parameters prioritize data recovery speed over data recovery reliability and the second error coding dispersal storage function parameters prioritize data recovery reliability over data recovery speed, wherein a substantially identical representation of a pre-encoded priority data segment of the priority data segments is recoverable from a corresponding set of the plurality of sets of encoded data slices or from a corresponding set of the sets of priority encoded data slices; and

output, via the interface, the plurality of sets of encoded data slices and the sets of priority encoded data slices to a dispersed storage network (DSN) memory for storage therein.

9. The computer of claim 8, wherein the processing module further functions to:

update a storage location table to associate the plurality of sets of encoded data slices with the data; and

update the storage location table to associate the sets of priority encoded data slices with the priority data segments.

10. The computer of claim 8, wherein the processing module functions to determine the priority data segments of the data by at least one of:

identifying content of the data having a desired priority level, wherein data segments containing the content are identified as the priority data segments;

determining a desired relationship between the data recovery speed and the data recovery reliability, and determining the priority data segments based on the desired relationship;

receiving, via the interface, an indicator that identifies the priority data segments; and

accessing a table regarding the data to identify the priority data segments.

11. The computer of claim 8, wherein the processing module further functions to:

encode video data into the plurality of sets of encoded data slices in accordance with the first error coding dispersal storage function parameters, wherein the plurality of set of encoded data slices is retrieved in response to a video on demand request; and

determine key frames of the video data as the priority data segments.

12. A computer comprises:

an interface;

a memory; and

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

retrieve, via the interface, a set of encoded data slices of a data segment from a dispersed storage network (DSN) memory, wherein the data segment was encoded in accordance with first error coding dispersal storage function parameters to produce a the set of encoded data slices;

when a substantially identical representation of a pre-encoded data segment cannot be recovered from the set of encoded data slices;

determine whether the data segment is a priority data segment and, when the data segment is a priority data segment, the data segment is redundantly encoded in accordance with a second error coding dispersed storage function parameters to produce a second set of encoded data slices; and

when the data segment is the priority data segment;

retrieve, via the interface, the second set of encoded data slices from the DSN memory; and

decode the second set of encoded data slices in accordance with the second error coding dispersal storage function parameters to recapture the substantially identical representation of the pre-encoded data segment.

13. The computer of claim 12, wherein the processing module further functions to:

access a storage location table to identify the set of encoded data slices with the data; and

access the storage location table to identify the second set of encoded data slices.

14. The computer of claim 12, wherein the processing module further functions to:

retrieve the set of encoded data slices in response to a video on demand request;

determine whether the data segment corresponds to a key frame of video data; and

when the data segment corresponds to the key frame;

identify the data segment as a priority data segment.