Metadata management system for an information dispersed storage system

• US 8,019,960 B2
• Filed: 07/07/2009
• Issued: 09/13/2011
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gopala Krishna Kapila Lakshmana Harsha, Dhanvi, Hendrickson, Dustin M., Mark, Zachary J., Resch, Jason K., Gladwin, S. Christopher, England, Matthew M., Thornton, Vance T.
• Primary Examiner(s): BATAILLE, PIERRE MICHE
• Application Number:
  US12/499,013
• Publication Number:
  US 20100023529A1
• Time in Patent Office: 798 Days
• Field of Search: 711/114, 711/112, 711/170, 711/156, 711/E12.001
• US Class Current: 711/171
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

An apparatus includes a processing module and a network interface. The processing module is operably coupled to obtain; a data storage transaction request regarding a file; determine identity of a user based on the data storage transaction request; access file level metadata based on the identity of the user to determine a plurality of file slices associated with the file; access file slice metadata to identify at least one of a plurality of storage nodes that stores or is to store, a data slice and coded subsets of at least one of the plurality of file slices; and generate a data storage transaction message to facilitate a data storage transaction regarding the at least one of the plurality of file slices. The network interface is operably coupled to convert the data storage transaction message into a network data storage message and transmit it.

Claims

1. An apparatus comprises:

a processing module operably coupled to;

obtain a data storage transaction request regarding a file;

determine identity of a user based on the data storage transaction request;

access file level metadata based on the identity of the user to determine a plurality of file slices associated with the file;

access file slice metadata associated with the plurality of file slices to identify at least one of a plurality of storage nodes that stores, or is to store, a data slice and coded subsets of at least one of the plurality of file slices; and

generate a data storage transaction message to facilitate a data storage transaction of the data storage transaction request regarding the at least one of the plurality of file slices with the at least one of the plurality of storage nodes; and

a network interface operably couple to;

convert the data storage transaction message into a network data storage message; and

transmit the network data storage message.

2. The apparatus of claim 1, wherein the file level metadata comprises at least one of:

a transaction data sources table; and

an applications table.

3. The apparatus of claim 1, wherein the file slice level metadata comprises at least one of:

a data sources table; and

a data space table.

4. The apparatus of claim 1, wherein the processing module further determines the identity of the user by:

accessing data space level metadata to determine to a data space of the user based on an account identifier of the user, wherein the data storage transaction request includes the account identifier; and

accessing user level metadata based on the account identifier and the data space to determine file transaction information for a file associated with the data storage transaction request, wherein the file transaction information includes reference to the file level metadata.

5. The apparatus of claim 4, wherein the system level metadata comprises at least one of:

a data space directory map;

a data space volume map; and

an account data space map.

6. The apparatus of claim 4, wherein the user level metadata comprises at least one of:

a transaction context table; and

a list of files.

7. The apparatus of claim 4, wherein the processing module further functions to:

close the file; and

when closing the file, update file transaction metadata of the user level metadata.

8. The apparatus of claim 1, wherein the processing module further functions to:

receive, via the network interface, a committed message from the at least one of the plurality of storage nodes;

update the file slice metadata for the at least one of the plurality of file slices; and

update the file level metadata in accordance with updating of the file slice metadata.

9. The apparatus of claim 1, wherein the processing module obtains the data storage transaction request by at least one of:

receiving the data storage transaction request from another processing module;

receiving the data storage transaction request from an application being supported by the processing module;

receiving the data storage transaction request from an application being supported by the other processing module; and

receiving the data storage transaction request during a log-in of the user.

10. The apparatus of claim 1, wherein the data storage transaction request comprises at least one of:

a read request;

a write request;

an edit request;

a rebuild request, and

a deletion request.

11. The apparatus of claim 1 further comprises at least one of:

a computer;

a plurality of computers;

an application running on the computer; and

the application running of the plurality of computers.

12. An apparatus comprises:

a processing module operably coupled to;

receive a request to create a file, wherein the request includes an identity of a user;

create, or update, system level metadata based on the identity of the user;

create, or update, user level metadata of the user to include data regarding the file;

create file level metadata regarding the file that includes a plurality of file slices;

create slice level metadata regarding the plurality of file slices; and

generate a data storage message that identifies at least one of the plurality of storage nodes that will store at least one of the plurality of file slices; and

a network interface operably couple to;

convert the data storage message into a network data storage message; and

transmit the network data storage message.

13. The apparatus of claim 12, wherein the file level metadata comprises at least one of:

a transaction data sources table; and

an applications table.

14. The apparatus of claim 12, wherein the file slice level metadata comprises at least one of:

a data sources table; and

a data space table.

15. The apparatus of claim 12, wherein the system level metadata comprises at least one of:

a data space directory map;

a data space volume map; and

an account data space map.

16. The apparatus of claim 12, wherein the user level metadata comprises at least one of:

a transaction context table; and

a list of files.

17. The apparatus of claim 12 further comprises at least one of:

a computer;

a plurality of computers;

an application running on the computer; and

the application running of the plurality of computers.

18. An data storage node comprises:

a network interface operably coupled to;

receive a network data storage message; and

convert the network data storage message into a data storage message;

memory; and

a processing module operably coupled to;

interpret the data storage message to identify at least one of a plurality of file slices and to determine a file slice level data storage transaction request;

accessing data space metadata to determine whether file slice specific metadata exists for the at least one of the plurality of file slices;

when the file slice specific metadata exists for the at least one of the plurality of file slices, access the memory to execute the file slice level data storage transaction request on the at least one of the plurality of file slices; and

update the file slice specific metadata when the file slice level data storage transaction request has been executed.

19. The data storage node of claim 18, wherein the data space metadata comprises at least one of:

an identification code for the data storage node;

a listing of file slices; and

status for at least one file slice of the listing of file slices.