File System Enabling Fast Purges and File Access

• US 20140229513A1
• Filed: 02/13/2013
• Published: 08/14/2014
• Est. Priority Date: 02/13/2013
• Status: Active Grant

• Current Assignee: Edgio Incorporated
• Original Assignee: EdgeCast Networks (Verizon Communications Inc.)
• Inventors: Shiell, Derek, Peters, Robert J.
• Granted Patent: US 9,128,944 B2
• Time in Patent Office: Days
• US Class Current: 707/803
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11

Abstract

Some embodiments provide a file system for purging content based on a single traversal of the file system that identifies the directory containing the content without performing a secondary traversal within the directory to target the operation to only the file that are associated with content such that other files contained in the directory are unaffected. The file system supplements traditional directory structures with file-level directories. Each file-level directory is created to contain a root file associated with particular content, different variants of the particular content, and supporting files. Consequently, the file system can complete an operation targeting particular content by performing that operation on the file-level directory containing the particular content, thereby eliminating the need to conduct a file-by-file traversal of the containing directory as a prerequisite to identifying the files associated with the particular content and performing the operation on the files individually.

Claims

1. A computer-implemented method for organizing files on a computer file system, the computer-implemented method comprising:

receiving a first request for first content, the first request identifying a directory path to the first content and a first filename for a first file associated with the first content;

generating a first file-level directory that is a subdirectory within the directory path based on the first filename, wherein the first file-level directory isolates files that are associated with the first content from files that are associated with other content;

storing the first file to the first file-level directory;

receiving a second request for second content that is different than the first content, the second request identifying the same directory path and a second filename for a second file associated with the second content;

generating a second file-level directory that is a subdirectory within the directory path based on the second filename; and

storing the second file to the second file-level directory.

2. The computer-implemented method of claim 1, wherein generating the first file-level directory comprises (i) producing a name for the first file-level directory by appending an escape modifier to the first filename and (ii) appending the name for the first file-level directory to the directory path.

3. The computer-implemented method of claim 1 further comprising receiving an operation to purge the first content from the computer file system.

4. The computer-implemented method of claim 3 further comprising purging the first content by traversing the computer file system to the first file-level directory and purging the first file-level directory without conducting a file-by-file analysis of any files within the first file-level directory.

5. The computer-implemented method of claim 1 further comprising receiving a third request for a third file representing a variant of the first content, the third request identifying the same directory path and a third filename that is derived from the first filename.

6. The computer-implemented method of claim 5 further comprising storing the third file in the first file-level directory.

7. The computer-implemented method of claim 5, wherein the third request specifies a URL, the URL comprising the directory path, the first file filename, and a set of query string parameters, wherein the third filename is identified as a result of combining the first file filename with a hash of the query string parameters.

8. The computer-implemented method of claim 1, wherein the first file-level directory and the second file-level directory are not specified in the directory path and wherein the first file-level directory and the second file-level directory are automatically generated by the computer file system as part of storing the first content and the second content to the computer file system.

9. A computer-implemented method for efficiently purging cached content from a file system, the computer-implemented method comprising:

identifying a common directory path that is specified as a location for a plurality of files associated with a plurality of different content;

generating a subdirectory within the directory path for each item of content of the plurality of content;

for each set of the plurality of files that is associated with same content, storing the set of files to a subdirectory within the directory path that is generated for that same content;

receiving a purge operation directed to particular content of the plurality of different content; and

purging a particular subdirectory within the directory path that contains the set of files that are associated with the particular content without analyzing any files in the directory path or the particular subdirectory.

10. The computer-implemented method of claim 9, wherein each subdirectory is automatically generated by the file system by appending the subdirectory to the particular directory path based on a filename for at least one file stored to the subdirectory.

11. The computer-implemented method of claim 9, wherein purging the particular subdirectory comprises performing a first purge during a period of heavy load and performing a second purge during a period of light load.

12. The computer-implemented method of claim 11, wherein performing the first purge comprises renaming the particular subdirectory and performing the second purge comprises physically deleting the particular subdirectory and any contained files from the file system.

13. The computer-implemented method of claim 9, wherein generating the subdirectory within the directory path for the particular content of the plurality of content comprises identifying a filename for one file of the particular content, generating a subdirectory name for the particular content based on the filename, and appending the subdirectory name to the directory path.

14. The computer-implemented method of claim 9, wherein the set of files associated with the particular content comprises files representing different variants of the particular content.

15. For a caching server that distributes content in response to requests for said content, a computer-implemented method for organizing and managing cached content, the computer-implemented method comprising:

receiving a request for content, the request comprising a URL identifying the content by way of a filename that follows a terminal directory of a directory path;

generating a directory name to a subdirectory within the terminal directory of the directory path;

appending the directory name to the terminal directory of the directory path;

determining if a file having the filename is stored to the subdirectory; and

serving the file in response to the request when the file is stored to the subdirectory.

16. The computer-implemented method of claim 15 further comprising retrieving the file having the filename from an origin server when the file is not stored to the subdirectory.

17. The computer-implemented method of claim 16 further comprising storing the file to the subdirectory instead of the terminal directory of the directory path specified as part of the URL.

18. The computer-implemented method of claim 15, wherein the URL includes query string parameters identifying a variant of the content.

19. The computer-implemented method of claim 18, wherein the file is a first file, the computer-implemented method further comprising generating a second file storing the variant of the content based on the first file having the filename and the query string parameters when the first file is stored to the subdirectory.

20. The computer-implemented method of claim 19, wherein serving the file in response to the request comprises serving the second file.

21. The computer-implemented method of claim 19, wherein generating the second file comprises hashing the query string parameters and combining a result of the hashing with the filename to produce a name for identifying the second file.

22. The computer-implemented method of claim 15 further comprising receiving a request to purge the content identified by the URL.

23. The computer-implemented method of claim 22 further comprising purging the subdirectory under the terminal directory of the directory path without scanning any files of the terminal directory and the subdirectory.