Prioritized deleting of slices stored in a dispersed storage network

• US 8,756,480 B2
• Filed: 05/04/2012
• Issued: 06/17/2014
• Est. Priority Date: 06/06/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gladwin, S. Christopher, Shirley, Thomas F. Jr., Resch, Jason K.
• Primary Examiner(s): TRIMMINGS, JOHN P
• Application Number:
  US13/464,846
• Publication Number:
  US 20120311403A1
• Time in Patent Office: 774 Days
• US Class Current: 714/763
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/1402 Saving, restoring, recovering or retrying
• CPC Class Codes: G06F 2211/1035 Keeping track, i.e. keeping track of data and parity changes
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 9/06 using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
• CPC Class Codes: H03M 13/1105 Decoding
• CPC Class Codes: H04L 65/00 Network arrangements, protocols or services for supporting real-time applications in data packet communication real-time or near real-time messaging, e.g. instant messaging [IM] H04L51/04; selective video distribution H04N21/00
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 69/00 Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass networks security protocols H04L9/40; wireless communication networks H04W
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02

Abstract

A method begins by a dispersed storage (DS) processing module monitoring storage of data, wherein the data is encoded using a dispersed storage error coding function to produce a plurality of sets of encoded data slices and is stored as the plurality of sets of encoded data slices. The method continues with the DS processing module determining analysis priority of the data in accordance with an analysis prioritization protocol. When the analysis priority of the data compares unfavorably to a first priority threshold, the method continues with the DS processing module issuing a command to delete an encoded data slice from each set of at least some of the plurality of sets of encoded data slices.

Claims

1. A method comprises:

monitoring storage of data, wherein the data is encoded using a dispersed storage error coding function to produce a plurality of sets of encoded data slices and is stored as the plurality of sets of encoded data slices;

determining analysis priority of the data in accordance with an analysis prioritization protocol; and

when the analysis priority of the data compares unfavorably to a first priority threshold, issuing a command to delete an encoded data slice from each set of at least some of the plurality of sets of encoded data slices.

2. The method of claim 1 further comprises:

when the analysis priority of the data compares unfavorably to a second priority threshold, issuing a second command to delete the encoded data slice and at least one other encoded data slice from each set of the at least some of the plurality of sets of encoded data slices.

3. The method of claim 1 further comprises:

determining, in accordance with the analysis prioritization protocol, that analysis of the data is no longer desired; and

when the analysis of the data is no longer desired, issuing a second command to delete the plurality of sets of encoded data slices.

4. The method of claim 1 further comprises:

analyzing the data;

determining, based on the analyzing the data, whether the data can be deleted; and

when the data can be deleted, issuing a second command to delete the plurality of sets of encoded data slices.

5. The method of claim 1, wherein the analysis priority protocol comprises one or more of:

a time based scaling factor;

a data source based scaling factor;

a data content based scaling factor; and

a data location origination based scaling factor, wherein the analysis priority protocol is a function of the scaling factors.

6. The method of claim 1 further comprises:

determining a data characterization of the data, wherein the data characterization includes at least one of a time factor, a data source factor, a data content factor, and a data location origination factor;

determining a priority level of the data based on the data characterization and the analysis priority protocol; and

selecting parameters for the dispersed storage error coding function based on the priority level.

7. The method of claim 1 further comprises:

when the analysis priority of the data compares unfavorably to the first priority threshold;

issuing a no-rebuilding command regarding the encoded data slices from each set of the at least some of the plurality of sets of encoded data slices; and

updating parameters of the dispersed storage error coding function to reflect the deletion of encoded data slices from each set of the at least some of the plurality of sets of encoded data slices.

8. A dispersed storage (DS) processing module comprises:

a first module, when operable within a computing device, causes the computing device to;

monitor storage of data, wherein the data is encoded using a dispersed storage error coding function to produce a plurality of sets of encoded data slices and is stored as the plurality of sets of encoded data slices;

a second module, when operable within the computing device, causes the computing device to;

determine analysis priority of the data in accordance with an analysis prioritization protocol; and

a third module, when operable within the computing device, causes the computing device to;

when the analysis priority of the data compares unfavorably to a first priority threshold, issue a command to delete an encoded data slice from each set of at least some of the plurality of sets of encoded data slices.

9. The DS processing module of claim 8, wherein the third module is further operable to:

when the analysis priority of the data compares unfavorably to a second priority threshold, issue a second command to delete the encoded data slice and at least one other encoded data slice from each set of the at least some of the plurality of sets of encoded data slices.

10. The DS processing module of claim 8, wherein the third module is further operable to:

determine, in accordance with the analysis prioritization protocol, that analysis of the data is no longer desired; and

when the analysis of the data is no longer desired, issue a second command to delete the plurality of sets of encoded data slices.

11. The DS processing module of claim 8 further comprises:

a fourth module, when operable within the computing device, causes the computing device to;

analyze the data; and

the third module, when operable within the computing device, further causes the computing device to;

determine, based on the analyzing the data, whether the data can be deleted; and

when the data can be deleted, issue a second command to delete the plurality of sets of encoded data slices.

12. The DS processing module of claim 8, wherein the analysis priority protocol comprises at least one of:

a time based scaling factor;

a data source based scaling factor;

a data content based scaling factor; and

a data location origination based scaling factor, wherein the analysis priority protocol is a function of the scaling factors.

13. The DS processing module of claim 8, wherein the first module is further operable to:

determine a data characterization of the data, wherein the data characterization includes at least one of a time factor, a data source factor, a data content factor, and a data location origination factor;

determine a priority level of the data based on the data characterization and the analysis priority protocol; and

select parameters for the dispersed storage error coding function based on the priority level.

14. The DS processing module of claim 8, wherein the third module is further operable to:

when the analysis priority of the data compares unfavorably to the first priority threshold;

issue a no-rebuilding command regarding the encoded data slices from each set of the at least some of the plurality of sets of encoded data slices; and

update parameters of the dispersed storage error coding function to reflect the deletion of encoded data slices from each set of the at least some of the plurality of sets of encoded data slices.