Distributed deletion of a file and directory hierarchy

• US 10,678,452 B2
• Filed: 01/31/2017
• Issued: 06/09/2020
• Est. Priority Date: 09/15/2016
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Karr, Ronald, Lee, Robert, Ostrovsky, Igor
• Primary Examiner(s): Kuddus, Daniel A
• Application Number:
  US15/421,284
• Publication Number:
  US 20180075053A1
• Time in Patent Office: 1,225 Days
• CPC Class Codes: G06F 12/0253 Garbage collection, i.e. reclamation of unreferenced memory
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 2212/7205 Cleaning, compaction, garbage collection, erase control
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/0637 Permissions
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A method of distributed file deletion, performed by a storage system, is provided. The method includes receiving, at the storage system, a request to delete a directory and contents of the directory and adding the directory to a first set, listed in a memory in the storage system. The method includes operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory. The method includes deleting in a distributed manner across the storage system without concern for order, contents of directories, and the directories, listed in the second set.

Claims

1. A method, comprising:

receiving, at a storage system, a request to delete a directory and contents of the directory;

adding the directory to a first set, listed in a memory in the storage system;

operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory, responsive to the request to delete the directory and the contents of the directory;

deleting in a distributed manner across the storage system, contents of directories, and the directories, listed in the second set, responsive to the request to delete the directory and the contents of the directory; and

delaying for a time span, responsive to receiving the request, wherein the adding, the operating, and the deleting occur upon expiration of the time span, and wherein the contents of the directory are retrievable during the time span.

2. The method of claim 1, wherein the receiving the request comprises:

receiving a request to move the directory under a named directory that conceals the directory and contents to a client.

3. The method of claim 1, further comprising:

communicating a plurality of batch lists to a plurality of processors in the storage system, each of the plurality of batch lists listing a subset of the directory, wherein the deleting the directory is in accordance with the plurality of batch lists.

4. The method of claim 1, further comprising:

determining an amount of memory space made available by the request to delete the directory; and

scheduling a plurality of writes to the storage system, in accordance with the determining and in parallel with the deleting.

5. The method of claim 1, wherein:

the operating on the first set further comprises examining each directory in the first set to identify files, and adding each identified file to the second set; and

the deleting in a distributed manner across the storage system, the contents of the directories further comprises deleting data and records of the files listed in the second set and wherein the deleting is performed without concern for order.

6. The method of claim 1, wherein the deleting comprises:

coordinating the deleting of the directory with garbage collection in each of a plurality of solid-state storage units of the storage system.

7. The method of claim 1, further comprising:

writing directory names of the directories listed in the second set and all files of the directories listed in the second set to a trash list in NVRAM (nonvolatile random-access memory) of the storage system;

flushing the trash list from the NVRAM to solid-state storage memory in the storage system, responsive to detecting a power loss in the storage system; and

recovering the trash list from the solid-state storage memory, to continue the deleting upon restart after the power loss.

8. A tangible, non-transitory, computer-readable media having instructions thereupon which, when executed by a processor, cause the processor to perform a method comprising:

receiving, at the storage system, a request to delete a directory and contents of the directory;

adding the directory to a first set, listed in a memory in the storage system;

operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory, responsive to the request to delete the directory and the contents of the directory;

deleting in a distributed manner across the storage system, contents of directories, and the directories, listed in the second set, responsive to the request to delete the directory and the contents of the directory; and

delaying for a time span, responsive to receiving the request, wherein the adding, the operating, and the deleting occur upon expiration of the time span, and wherein the contents of the directory are retrievable during the time span.

9. The computer-readable media of claim 8, wherein the receiving the request comprises:

receiving a request to move the directory under a named directory that conceals the directory and contents to a client.

10. The computer-readable media of claim 8, wherein the method further comprises:

communicating a plurality of batch lists to a plurality of processors in the storage system, each of the plurality of batch lists listing a subset of the directory, wherein the deleting the directory is in accordance with the plurality of batch lists.

11. The computer-readable media of claim 8, wherein the method further comprises:

determining an amount of memory space made available by the request to delete the directory; and

scheduling a plurality of writes to the storage system, in accordance with the determining and in parallel with the deleting.

12. The computer-readable media of claim 8, wherein the listing for deletion, and the deleting, are performed by a plurality of authorities in the storage system, with each inode, range of data, and sub-directory owned by an authority.

13. The computer-readable media of claim 8, wherein the deleting comprises:

coordinating the deleting of the directory with garbage collection in each of a plurality of solid-state storage units of the storage system and wherein the deleting is performed without concern for order.

14. A storage system, comprising:

storage memory; and

a plurality of storage nodes, each having one or more processors, the plurality of storage nodes configurable to cooperate to perform a method comprising;

receiving, at the storage system, a request to delete a directory and contents of the directory;

adding the directory to a first set, listed in a memory in the storage system;

operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory, responsive to the request to delete the directory and the contents of the directory;

deleting in a distributed manner across the storage system, contents of directories, and the directories, listed in the second set, responsive to the request to delete the directory and the contents of the directory; and

delaying for a time span, responsive to receiving the request, wherein the adding, the operating, and the deleting occur upon expiration of the time span, and wherein the contents of the directory are retrievable during the time span.

15. The storage system of claim 14, wherein the receiving the request comprises:

receiving a request to move the directory under a named directory that conceals the directory and contents to a client, and wherein the deleting is performed without concern for order.

16. The storage system of claim 14, wherein the method further comprises:

communicating a plurality of batch lists to a plurality of processors in the storage system, each of the plurality of batch lists listing a subset of the directory, wherein the deleting the directory is in accordance with the plurality of batch lists.

17. The storage system of claim 14, wherein the method further comprises:

determining an amount of memory space made available by the request to delete the directory rectory; and

scheduling a plurality of writes to the storage system, in accordance with the determining and in parallel with the deleting.

18. The storage system of claim 14, wherein the listing for deletion, and the deleting, are performed by a plurality of authorities in the storage system, with each inode, range of data, and sub-directory owned by an authority.

19. The storage system of claim 14, wherein the deleting comprises:

coordinating the deleting of the directory with garbage collection in each of a plurality of solid-state storage units of the storage system.

20. The storage system of claim 14, wherein the method further comprises:

writing directory names of the directories listed in the second set and all files of the directories listed in the second set to a trash list in NVRAM (nonvolatile random-access memory) of the storage system;

flushing the trash list from the NVRAM to solid-state storage memory in the storage system, responsive to detecting a power loss in the storage system; and

recovering the trash list from the solid-state storage memory, to continue the deleting upon restart after the power loss.