Storing directory metadata in a dispersed storage network

• US 8,832,493 B2
• Filed: 12/01/2011
• Issued: 09/09/2014
• Est. Priority Date: 12/22/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Volvovski, Ilya, Baptist, Andrew, Leggette, Wesley
• Primary Examiner(s): PATEL, KAMINI B
• Application Number:
  US13/308,724
• Publication Number:
  US 20120166867A1
• Time in Patent Office: 1,013 Days
• Field of Search: 714/6.24
• US Class Current: 714/6.24
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 12/1009 using page tables, e.g. page table structures
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a processing module dispersed storage error encoding data to produce encoded data slices and updating directory metadata regarding storing the data in a dispersed storage network (DSN) memory to produce updated directory metadata. The method continues with the processing module dispersed storage error encoding the updated directory metadata to produce encoded directory metadata slices and transmitting one or more data slice write requests to the DSN that each include a first transaction number. The method continues with the processing module transmitting one or more directory metadata write requests to the DSN memory, wherein the one or more directory metadata write requests includes a second transaction number, and when a favorable write response condition exists, transmitting a commit request to the DSN memory to commit storage of at least one of: the encoded data slices and the encoded directory metadata slices.

Claims

1. A method comprises:

dispersed storage error encoding data to produce a plurality of sets of encoded data slices;

updating directory metadata regarding storing the data as the plurality of sets of encoded data slices in a dispersed storage network (DSN) memory to produce updated directory metadata;

dispersed storage error encoding the updated directory metadata to produce a plurality of sets of encoded directory metadata slices;

transmitting one or more data slice write requests to the DSN memory regarding storing the plurality of sets of encoded data slices, wherein the one or more data slice write requests includes a first transaction number;

transmitting one or more directory metadata write requests to the DSN memory regarding storing the plurality of sets of encoded directory metadata slices, wherein the one or more directory metadata write requests includes a second transaction number; and

when a favorable write response condition exists for both of the one or more data slice write requests and the one or more directory metadata write requests, transmitting one or more common commit requests to the DSN memory to commit storage of both of the plurality of sets of encoded data slices and the plurality of sets of encoded directory metadata slices, wherein the one or more common commit requests includes the first transaction number and the second transaction number.

2. The method of claim 1, wherein the transmitting the one or more data slice write requests comprises:

generating a set of data slice write requests, wherein each of the data slice write requests includes the first transaction number regarding storage of a respective group of encoded data slices of the plurality of sets of encoded data slices; and

transmitting the set of data slice write requests to a set of dispersed storage (DS) units of the DSN memory.

3. The method of claim 1, wherein the transmitting the one or more data slice write requests comprises:

generating a data slice write request, wherein the data slice write request includes the first transaction number regarding storage of the plurality of sets of encoded data slices; and

transmitting the set of data slice write requests to a set of dispersed storage (DS) units of the DSN memory, wherein a DS unit of the set of DS units is targeted to store a respective group of encoded data slices of the plurality of sets of encoded data slices.

4. The method of claim 1, wherein the updating the directory metadata comprises:

retrieving the directory metadata from at least one of the DSN memory and a local memory;

determining a DSN memory storage location information for the plurality of sets of encoded data slices, wherein the DSN memory storage location information includes one or more of;

a source name corresponding to a DSN memory storage location, a data size indicator, a data type indicator, snapshot information, a timestamp; and

a segment allocation table; and

modifying the directory metadata to include a filename associated with the data and the DSN memory storage location to produce the updated directory metadata.

5. The method of claim 1 further comprises one or more of:

receiving at least a threshold number of data slices write responses regarding the one or more data slice write requests and generating a commit request as the one or more commit requests; and

receiving at least a threshold number of directory metadata write responses regarding the one or more directory metadata write requests and generating the commit request.

6. The method of claim 1 further comprises one or more of:

receiving at least a first threshold number of data slices write responses regarding writing a first set of encoded data slices of the plurality of sets of encoded data slices and generating a first commit request as one of the one or more commit requests;

receiving at least a first threshold number of directory metadata write responses regarding writing a first set of encoded directory metadata slices of the plurality of sets of encoded directory metadata slices and generating the first commit request; and

receiving at least a second threshold number of data slices write responses regarding writing a second set of encoded data slices of the plurality of sets of encoded data slices and generating a second commit request as a second one of the one or more commit requests.

7. The method of claim 1, wherein the transmitting one or more common commit requests comprises one of:

generating a common commit request as the one or more common commit requests to include a common first transaction number and a common second transaction number regarding committing storage of the plurality of sets of encoded data slices and the plurality of sets of encoded directory metadata slices; and

generating a group of commit requests as the one or more common commit requests, wherein a commit request of the group of commit requests includes a unique first transaction number and a unique second transaction number regarding committing storage of a respective group of encoded data slices of the plurality of sets of encoded data slices.

8. A dispersed storage (DS) module comprises:

at least one memory;

a first module for dispersed storage error encoding data to produce a plurality of sets of encoded data slices;

a second module for;

updating directory metadata regarding storing the data as the plurality of sets of encoded data slices in a dispersed storage network (DSN) memory to produce updated directory metadata; and

dispersed storage error encoding the updated directory metadata to produce a plurality of sets of encoded directory metadata slices;

a third module for;

facilitating transmitting one or more data slice write requests to the DSN memory regarding storing the plurality of sets of encoded data slices, wherein the one or more data slice write requests includes a first transaction number; and

facilitating transmitting one or more directory metadata write requests to the DSN memory regarding storing the plurality of sets of encoded directory metadata slices, wherein the one or more directory metadata write requests includes a second transaction number; and

a fourth module for transmitting one or more common commit requests to the DSN memory, when a favorable write response condition exists, for both of the one or more data slice write requests and the one or more directory metadata write requests, to commit storage of both of the plurality of sets of encoded data slices and the plurality of sets of encoded directory metadata slices, wherein the common commit request includes the first transaction number and the second transaction number.

9. The DS module of claim 8, wherein the third module is operable to transmit the one or more data slice write requests by:

generating a set of data slice write requests, wherein each of the data slice write requests includes the first transaction number regarding storage of a respective group of encoded data slices of the plurality of sets of encoded data slices; and

transmitting the set of data slice write requests to a set of dispersed storage (DS) units of the DSN memory.

10. The DS module of claim 8, wherein the third module is further operable to transmit the one or more data slice write requests by:

generating a data slice write request, wherein the data slice write request includes the first transaction number regarding storage of the plurality of sets of encoded data slices; and

transmitting the set of data slice write requests to a set of dispersed storage (DS) units of the DSN memory, wherein a DS unit of the set of DS units is targeted to store a respective group of encoded data slices of the plurality of sets of encoded data slices.

11. The DS module of claim 8, wherein the second module is operable to update the directory metadata by:

retrieving the directory metadata from at least one of the DSN memory and a local memory;

determining a DSN memory storage location information for the plurality of sets of encoded data slices, wherein the DSN memory storage location information includes one or more of;

a source name corresponding to a DSN memory storage location, a data size indicator, a data type indicator, snapshot information, a timestamp; and

a segment allocation table; and

modifying the directory metadata to include a filename associated with the data and the DSN memory storage location to produce the updated directory metadata.

12. The DS module of claim 8 further comprises:

a fifth module for at least one of;

receiving at least a threshold number of data slices write responses regarding the one or more data slice write requests and generating a commit request as the one or more commit requests; and

receiving at least a threshold number of directory metadata write responses regarding the one or more directory metadata write requests and generating the commit request.

13. The DS module of claim 8 further comprises:

a fifth module for at least one of;

receiving at least a first threshold number of data slices write responses regarding writing a first set of encoded data slices of the plurality of sets of encoded data slices and generating a first commit request as one of the one or more commit requests;

receiving at least a first threshold number of directory metadata write responses regarding writing a first set of encoded directory metadata slices of the plurality of sets of encoded directory metadata slices and generating the first commit request; and

receiving at least a second threshold number of data slices write responses regarding writing a second set of encoded data slices of the plurality of sets of encoded data slices and generating a second commit request as a second one of the one or more commit requests.

14. The DS module of claim 8, wherein the fourth module is operable to transmit one or more common commit requests by one of:

generating a common commit request as the one or more common commit requests to include a common first transaction number and a common second transaction number regarding committing storage of the plurality of sets of encoded data slices and the plurality of sets of encoded directory metadata slices; and

generating a group of commit requests as the one or more common commit requests, wherein a commit request of the group of commit requests includes a unique first transaction number and a unique second transaction number regarding committing storage of a respective group of encoded data slices of the plurality of sets of encoded data slices.