APPARATUS AND METHODS FOR SCANNING DATA IN A CLOUD STORAGE SERVICE

• US 20150331905A1
• Filed: 07/28/2015
• Published: 11/19/2015
• Est. Priority Date: 12/22/2008
• Status: Active Grant

• Current Assignee: CTERA Networks Limited
• Original Assignee: CTERA Networks Limited
• Inventors: Brand, Aron
• Granted Patent: US 10,521,423 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: H04L 63/20 for managing network security; network security policies in general filtering policies H04L63/0227
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods and apparati for scanning objects stored in a cloud storage system are disclosed. In an embodiment, the method includes determining at least one object that requires scanning; scanning each of the at least one object, wherein the scanning is performed using at least one scanning engine; and synchronizing the scanning results of the at least one object with a plurality of devices connected to the cloud store system.

Claims

1. A method for scanning objects stored in a cloud storage system, comprising:

determining at least one object that requires scanning;

scanning each of the at least one object, wherein the scanning is performed using at least one scanning engine; and

synchronizing the scanning results of the at least one object with a plurality of devices connected to the cloud store system.

2. The method of claim 1, wherein the determining if the at least one object containing at least one object requires scanning further comprises:

checking a scanning policy for the at least one object, wherein the scanning policy defines at least a type of the at least one scanning engine.

3. The method of claim 1, wherein at least one scanning engine is a blocking scan, wherein the blocking scan requires completion of the scanning prior to allowing access to the object.

4. The method of claim 1, wherein the scanning further comprises:

scanning at least one object using any one of;

at least one scanning engine running as a background process and at least one scanning engine running as a foreground process, wherein the foreground process is triggered by an access attempt to the at least one object.

5. The method of claim 1, wherein the at least one scanning engine is any one of:

a full-text indexing engine;

an image cataloguing and metadata extraction engine, a content type detection engine, an e-discovery and data mining engine, an unstructured data analytics engine.

6. The method of claim 1, wherein the at least one scanning engine is a threat detection engine.

7. The method of claim 6, wherein upon determining by the at least one scanning engine that at least one object is a threat, neutralizing the threat from the at least one object from the cloud store system and any device synchronized with the cloud storage system, wherein neutralizing the threat includes any one of:

deleting the object, quarantining the object, and replacing the object by a safe version of the object.

8. The method of claim 1, further comprising:

caching the at least one scanned object based on a hash function computed over any one of;

the at least one object and a map of the at least one object.

9. The method of claim 8, wherein the cached object is stored in a distributed hash table.

10. The method of claim 1, further comprising:

tracking objects that have been scanned by;

assigning a monotonically increasing sequence number to each object;

setting a high watermark value, wherein the high watermark value is a highest sequence number that has been scanned, thereby the high watermark value determines in part if subsequent scanning for the at least one object is required; and

storing the high watermark value.

11. The method of claim 1, wherein the at least one object contains unstructured data.

12. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 1.

13. A method for block-level scanning of objects stored in a cloud storage system, comprising:

selecting at least one object stored in a cloud storage system for scanning;

analyzing each new block in the at least one object, wherein the analysis is performed, in part, on each new block based on existing blocks having a previously determined safety level indication, wherein each block is a portion of the at least one object;

determining, respective of the analysis, a safety level indication for each analyzed block; and

handling each analyzed block respective of the determined safety level indication.

14. The method of claim 13, wherein analyzing each new block in the at least one object further comprises:

checking if an identical block having a previously determined safety level indication exists;

scanning each block with at least one threat scanning engine, if an identical block does not exist; and

storing for each block the safety level indication determined by the at least one threat scanning engine.

15. The method of claim 14, wherein the safety level indication is any one of:

safe, threat, and suspect.

16. The method of claim 15, wherein upon determining that the safety level indication of a new block is suspect further comprising:

matching between data of the new block and at least one known threat pattern.

17. The method of claim 16, wherein the safety level indication of the new block is determined to be threat upon a complete matching with the at least one known threat pattern.

18. The method of claim 16, wherein the safety level indication of the new block is determined to be suspect upon a partial matching with the at least one known threat pattern.

19. The method of claim 18, further comprising:

searching for a complete matching of at least one known threat pattern across adjacent blocks determined as suspect.

20. The method of claim 13, wherein the cloud storage system is a block-based deduplication storage system.

21. A non-transitory computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to claim 13.

22. An apparatus for scanning objects stored in a cloud storage system, comprising:

a processing unit; and

a memory coupled to the processing unit, the memory contains instructions that when executed by the processing unit configures the apparatus to;

determine at least one object that requires scanning;

scan each of the at least one object, wherein the scanning is performed using at least one scanning engine; and

synchronize the scanning results of the at least one object with a plurality of devices connected to the cloud store system.

23. An apparatus for block-level scanning of objects stored in a cloud storage system, comprising:

a processing unit; and

a memory coupled to the processing unit, the memory contains instructions that when executed by the processing unit configures the apparatus to;

select at least one object stored in a cloud storage system for scanning;

analyze each new block in the at least one object, wherein the analysis is performed, in part, on each new block based on existing blocks having a previously determined safety level indication, wherein each block is a portion of the at least one object;

determine, respective of the analysis, a safety level indication for each analyzed block; and

handle each analyzed block respective of the determined safety level indication.

24. A method for synchronizing objects between a network attached storage device and a cloud storage service (CSS), wherein the CSS is geographically distributed from the network attached storage device, comprising:

marking all objects that have been changed since the last synchronization;

dividing all marked objects into variable size blocks;

assigning each block with a unique code;

determining which blocks have been modified using the unique codes of the blocks;

synchronizing the modified blocks such that both the CSS and the storage device include a consistent version of each object that has been changed; and

scanning each modified block to detect potential threats.

25. The method of claim 24, wherein scanning each modified block further comprises:

analyzing each modified block based, in part, on existing blocks having a previously determined safety level indication, wherein each block is a portion of the at least one object; and

determining, respective of the analysis, a safety level indication for each analyzed block; and

handling each analyzed block respective of the determined safety level indication.

26. The method of claim 25, wherein analyzing each modified block further comprises:

checking if an identical block having a previously determined safety level indication exists;

scanning each block with at least one threat scanning engine, if an identical block does not exist; and

storing for each block the safety level indication determined by the at least one threat scanning engine, wherein the safety level indication is any one of;

safe, threat, and suspect.

27. The method of claim 26, wherein upon determining that the safety level indication of a modified block is suspect, further comprises:

matching between data of the new block and at least one known threat pattern.