File system enabling fast purges and file access

• US 9,128,944 B2
• Filed: 02/13/2013
• Issued: 09/08/2015
• Est. Priority Date: 02/13/2013
• Status: Active Grant

• Current Assignee: Edgio Incorporated
• Original Assignee: EdgeCast Networks (Verizon Communications Inc.)
• Inventors: Shiell, Derek, Peters, Robert J.
• Primary Examiner(s): Chen, Susan
• Application Number:
  US13/766,717
• Publication Number:
  US 20140229513A1
• Time in Patent Office: 937 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

Some embodiments provide a file system for purging content based on a single traversal of the file system that identifies the directory containing the content without performing a secondary traversal within the directory to target the operation to only the file that are associated with content such that other files contained in the directory are unaffected. The file system supplements traditional directory structures with file-level directories. Each file-level directory is created to contain a root file associated with particular content, different variants of the particular content, and supporting files. Consequently, the file system can complete an operation targeting particular content by performing that operation on the file-level directory containing the particular content, thereby eliminating the need to conduct a file-by-file traversal of the containing directory as a prerequisite to identifying the files associated with the particular content and performing the operation on the files individually.

Claims

1. A computer-implemented method for organizing files on a computer file system, the computer-implemented method comprising:

receiving a first storage operation identifying a directory path as a storage location for a first file with a first filename;

generating, in response to the first storage operation, a first file-level directory as a subdirectory within the directory path identified in the first storage operation based on the first filename, wherein the first file-level directory isolates variants of the first file from other files identified to be in the same directory path, wherein said variants comprise different versions, encodings, resolutions, or compressions of the first file;

storing the first file in response to the first storage operation by accessing the directory path identified in the first storage operation, further accessing the first file-level directory not identified in the first storage operation, and generating a copy of the first file in the first file-level directory isolating the first file and variants of the first file from other files identified to be in the same directory path as the first file;

receiving a second storage operation identifying the same directory path as a storage location for a second file with a second filename;

generating a second file-level directory as a subdirectory within the directory path based on the second filename, wherein the second file-level directory is a different subdirectory under the directory path than the first file-level directory with the second file-level directory isolating variants of the second file from other files including the first file identified to be in the same directory path; and

storing the second file in response to the second storage operation by accessing the directory path identified in the second storage operation, further accessing the second file-level directory not identified in the second storage operation, and generating a copy of the second file in the second file-level directory isolating the second file and variants of the second file from other files including the first file identified to be in the same directory path as the second file.

2. The computer-implemented method of claim 1, wherein generating the first file-level directory comprises (i) producing a name for the first file-level directory by appending an escape modifier to the first filename and (ii) appending the name for the first file-level directory to the directory path.

3. The computer-implemented method of claim 1 further comprising receiving an operation to purge the first file from the computer file system and, in response to said operation, deleting the first-file directory without querying for the first file in the directory path or the first file-level directory.

4. The computer-implemented method of claim 1 further comprising receiving an operation to purge content encoded to the first file and, in response to said operation, traversing the computer file system to the first file-level directory and removing the first file-level directory without querying within the directory path or the first file-level directory for the first file.

5. The computer-implemented method of claim 1 further comprising receiving a third storage operation for a third file representing a variant of the first file, the third storage operation identifying the same directory path and a third filename that is derived from the first filename.

6. The computer-implemented method of claim 5 further comprising storing the third file in the first file-level directory and not in the directory path identified by the third storage operation.

7. The computer-implemented method of claim 5, wherein the third storage operation specifies a URL, the URL comprising the directory path, the first file filename, and a set of query string parameters, wherein the third filename is identified as a result of combining the first file filename with a hash of the query string parameters.

8. The computer-implemented method of claim 1, wherein the first file-level directory and the second file-level directory are not specified in the directory path and wherein the first file-level directory and the second file-level directory are automatically generated by the computer file system as part of storing the first file and the second file to the computer file system.

9. A computer-implemented method for efficiently purging cached content from a file system, the computer-implemented method comprising:

identifying a common directory path that is specified as a location for a plurality of files encoding a plurality of different content;

generating a new subdirectory within the directory path for each file of the plurality of files encoding different content of the plurality of content;

for each set of the plurality of files encoding different variants of same content and identified to be in the common directory path, storing the set of files to a subdirectory within the directory path generated for that same content, wherein said variants comprise different versions, encodings, resolutions, or compressions of the same content;

receiving a purge operation directed to particular content of the plurality of different content; and

purging a particular subdirectory within the directory path that contains the set of files encoding different variants of the particular content identified in the purge operation without searching within the directory path or the particular subdirectory and individually selecting files encoding the particular content for deletion.

10. The computer-implemented method of claim 9, wherein each subdirectory is automatically generated by the file system by appending the subdirectory to the particular directory path based on a filename for at least one file stored to the subdirectory.

11. The computer-implemented method of claim 9, wherein purging the particular subdirectory comprises performing a first purge during a period of heavy load and performing a second purge during a period of light load.

12. The computer-implemented method of claim 11, wherein performing the first purge comprises renaming the particular subdirectory and performing the second purge comprises physically deleting the particular subdirectory and any contained files from the file system.

13. The computer-implemented method of claim 9, wherein generating the subdirectory within the directory path comprises identifying a filename for one file encoding the particular content, generating a subdirectory name based on the filename, and appending the subdirectory name to the directory path.

14. A computer-implemented method comprising:

receiving a request for content, the request comprising a URL identifying the content by way of a filename that follows a terminal directory of a directory path;

generating a directory name to a new subdirectory not specified as part of the URL directory path using the filename from the request;

appending the directory name to the terminal directory of the directory path; and

traversing the directory path in response to said request, wherein said traversing comprises accessing each directory in the directory path up to the terminal directory and further accessing the subdirectory with the directory name not specified as part of said request; and

serving a file having the filename from the subdirectory in response to the request when the file is stored to the subdirectory.

15. The computer-implemented method of claim 14 further comprising retrieving the file having the filename from an origin server when the file is not stored to the subdirectory.

16. The computer-implemented method of claim 15 further comprising storing the file to the subdirectory not specified as part of the URL instead of the terminal directory of the directory path specified as part of the URL.

17. The computer-implemented method of claim 14, wherein the URL includes query string parameters identifying a variant of the content, wherein said variant comprises a different version, encoding, resolution, or compression of the content.

18. The computer-implemented method of claim 17, wherein the file is a first file, the computer-implemented method further comprising generating a second file storing the variant of the content based on the first file having the filename and the query string parameters when the first file is stored to the subdirectory.

19. The computer-implemented method of claim 18, wherein serving the file in response to the request comprises serving the second file.

20. The computer-implemented method of claim 18, wherein generating the second file comprises hashing the query string parameters and combining a result of the hashing with the filename to produce a name for identifying the second file.

21. The computer-implemented method of claim 14 further comprising receiving a request to purge the content identified by the URL.

22. The computer-implemented method of claim 21 further comprising purging the subdirectory under the terminal directory of the directory path without scanning any files of the terminal directory and the subdirectory.