PERFORMING ANTI-VIRUS CHECKS FOR A DISTRIBUTED FILESYSTEM

• US 20140007239A1
• Filed: 09/05/2013
• Published: 01/02/2014
• Est. Priority Date: 05/03/2010
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Sharpe, Richard, Chou, Randy Yen-pang
• Granted Patent: US 9,811,662 B2
• Time in Patent Office: Days
• US Class Current: 726/24
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: G06F 21/561 Virus type analysis

Abstract

The disclosed embodiments disclose techniques that facilitate the process of performing anti-virus checks for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a write request from a client system that seeks to store a target file in the distributed system. A scan is then performed for this target file. For instance, the scan may be an anti-virus scan that ensures that viruses are not spread to the distributed filesystem or the clients of the distributed filesystem.

Claims

1. A computer-implemented method for performing file checks for a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

caching data for the distributed filesystem in each cloud controller;

receiving at a cloud controller a write request from a client system that seeks to store a target file in the distributed filesystem; and

performing a scan of the target file.

2. The computer-implemented method of claim 1, wherein the scan is an anti-virus scan.

3. The computer-implemented method of claim 2,

wherein the metadata for each file in the distributed filesystem includes a clean field that is marked when the file has been scanned and determined to be clean of viruses;

wherein performing the anti-virus scan of the target file involves marking the clean field for the target file if no virus is found for the target file; and

wherein a second cloud controller receiving a client request to access the target file confirms that the clean field for the target file is marked before allowing access to the target file.

4. The computer-implemented method of claim 3, wherein performing the anti-virus scan of the target file further comprises:

storing one or more cloud files that contain data blocks for the target file in a cloud storage system;

sending an incremental metadata snapshot containing metadata for the target file from the cloud controller to the other cloud controllers for the distributed filesystem;

receiving the incremental metadata snapshot at a third cloud controller that is associated with anti-virus scanning operations; and

if no virus is found for the target file, sending a subsequent incremental metadata snapshot from the third cloud controller in which the clean field for the target file is marked.

5. The computer-implemented method of claim 4,

wherein the third cloud controller is co-located with the cloud storage system and an anti-virus service in a cloud computing environment;

wherein the third cloud controller downloads the target file by downloading the one or more cloud files containing the data blocks for the target file from the cloud storage system;

wherein the third cloud controller transfers the target file to the anti-virus service; and

wherein after performing the anti-virus scan for the target file the anti-virus service sends the results for the anti-virus scan to the third cloud controller.

6. The computer-implemented method of claim 5, wherein the anti-virus scan of the target file is initiated at the time that the target file is written to the distributed filesystem.

7. The computer-implemented method of claim 5,

wherein the anti-virus scan of the target file is initiated at the time that the target file is accessed by the client request via the second cloud controller;

and

wherein the anti-virus scan executes in parallel with the download of the one or more cloud files that contain the data blocks for the target file from the cloud storage system.

8. The computer-implemented method of claim 5,

wherein a first anti-virus scan of the target file is initiated at the time that the target file is written to the distributed filesystem;

wherein the metadata for the target file includes a version field that tracks the version of the virus definitions that was used to perform the first anti-virus scan; and

wherein a subsequent anti-virus scan of the target file is initiated if an updated set of virus definitions that are newer than the tracked version are available when the target file is accessed.

9. The computer-implemented method of claim 5,

wherein a first anti-virus scan of the target file is initiated at the time that the target file is written to the distributed filesystem;

and

wherein a subsequent anti-virus scan of the target file is initiated when the anti-virus service receives an updated set of virus definitions.

10. The computer-implemented method of claim 9, wherein subsequent anti-virus scans for the distributed filesystem are configured to selectively re-scan a subset of the distributed filesystem.

11. The computer-implemented method of claim 10, wherein the subset of the distributed filesystem that is re-scanned is selected based on the file type for a file, the owner of the file, and the location of the file in the distributed filesystem hierarchy.

12. The computer-implemented method of claim 5,

wherein the clean field is organized in a manner that allows sub-portions of the target file that have already been scanned to be marked as clean;

and

wherein the second cloud controller already serves portions of the target file that have been marked as clean to a requesting client while the anti-virus service is still scanning other portions of the target file.

13. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for performing anti-virus checks for a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises;

storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems; and

caching data for the distributed filesystem in each cloud controller;

receiving at a cloud controller a write request from a client system that seeks to store a target file in the distributed filesystem; and

performing an anti-virus scan of the target file.

14. The non-transitory computer-readable storage medium of claim 13,

wherein the metadata for each file in the distributed filesystem includes a clean field that is marked when the file has been scanned and determined to be clean of viruses;

wherein performing the anti-virus scan of the target file involves marking the clean field for the target file if no virus is found for the target file; and

wherein a second cloud controller receiving a client request to access the target file confirms that the clean field for the target file is marked before allowing access to the target file.

15. The non-transitory computer-readable storage medium of claim 14, wherein performing the anti-virus scan of the target file further comprises:

storing one or more cloud files that contain data blocks for the target file in a cloud storage system;

sending an incremental metadata snapshot containing metadata for the target file from the cloud controller to the other cloud controllers for the distributed filesystem;

receiving the incremental metadata snapshot at a third cloud controller that is associated with anti-virus scanning operations; and

if no virus is found for the target file, sending a subsequent incremental metadata snapshot from the third cloud controller in which the clean field for the target file is marked.

16. The non-transitory computer-readable storage medium of claim 15,

wherein the third cloud controller is co-located with the cloud storage system and an anti-virus service in a cloud computing environment;

wherein the third cloud controller downloads the target file by downloading the one or more cloud files containing the data blocks for the target file from the cloud storage system;

wherein the third cloud controller transfers the target file to the anti-virus service; and

wherein after performing the anti-virus scan for the target file the anti-virus service sends the results for the anti-virus scan to the third cloud controller.

17. The non-transitory computer-readable storage medium of claim 16, wherein the anti-virus scan of the target file is initiated at the time that the target file is written to the distributed filesystem.

18. The non-transitory computer-readable storage medium of claim 16,

wherein the anti-virus scan of the target file is initiated at the time that the target file is accessed by the client request via the second cloud controller;

and

wherein the anti-virus scan executes in parallel with the download of the one or more cloud files that contain the data blocks for the target file from the cloud storage system.

19. The non-transitory computer-readable storage medium of claim 16,

wherein a first anti-virus scan of the target file is initiated at the time that the target file is written to the distributed filesystem;

wherein the metadata for the target file includes a version field that tracks the version of the virus definitions that was used to perform the first anti-virus scan; and

wherein a subsequent anti-virus scan of the target file is initiated if an updated set of virus definitions that are newer than the tracked version are available when the target file is accessed.

20. A cloud controller that initiates anti-virus checks for a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem; and

wherein the cloud controller is configured to;

receive an incremental metadata snapshot containing metadata for a new file that has been written to the distributed filesystem by another cloud controller for the distributed filesystem;

use the incremental metadata snapshot to download the new file from the cloud storage system;

transfer the target file to an anti-virus service to perform an anti-virus scan for the target file;

receive the results for the anti-virus scan; and

if no virus was found for the target file, send a subsequent incremental metadata snapshot that indicates that the target file is clean to the other cloud controllers for the distributed filesystem.