Preventing Excessive Hydration In A Storage Virtualization System

• US 20180060345A1
• Filed: 04/28/2017
• Published: 03/01/2018
• Est. Priority Date: 08/25/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Christiansen, Neal R., Pudipeddi, Ravisankar V., Xie, Ping, Barkhouse, Craig Ashley, Novak, Michael J.
• Granted Patent: US 11,061,623 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1734 Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Storage virtualization techniques allow files and directories to be stored remotely, for example, by a cloud storage provider, but in a manner that appears to a user or application running on a local computing device as if the files are stored locally—even though the data of those files and directories may not be resident on the local computing device. That is, the contents of files and directories that may exist in the cloud look and behave as if they were stored locally on a computing device. The level of hydration of placeholders may be monitored and operations may be performed to reduce the impact of excessive hydration on the resources of the computing device.

Claims

1. In a computing device comprising a processor, memory, and secondary storage, the memory storing computer-executable instructions that, when executed by the processor, implement a file system for managing the storage of files on the secondary storage, a method for storage virtualization of files, comprising:

storing, on the secondary storage, a plurality of placeholder for a respective plurality of files, each file comprising data which is stored on a network remotely from the secondary storage, each placeholder containing metadata associated with its respective file, a sparse data stream containing none or some data of the respective file, and information which enables any remotely stored data of the respective file to be retrieved from the network;

receiving, from one or more applications executing on the processor of the computing device, one or more requests, each request comprising a request to access a file for which a placeholder is stored on the secondary storage;

determining, for each received request to access a file, whether the request requires that data of the file be retrieved from remote storage on the network;

monitoring a level of such requests requiring retrieval of data from the remote storage;

determining that the monitored level of requests has satisfied a threshold; and

performing operations, in response to determining that the threshold has been satisfied, to reduce an impact of such level of requests on resources of the computing device.

2. The method recited in claim 1, wherein the resources may comprise one or more of available storage space on the secondary storage of the computing device and network bandwidth.

3. The method recited in claim 1, wherein determining that the threshold is satisfied is based on a number of received requests requiring data to be retrieved from remote storage.

4. The method recited in claim 1, wherein determining that the threshold is satisfied is based on one of a fixed amount or a percentage of available resources of the computing device.

5. The method recited in claim 4, wherein determining that the threshold is satisfied is further based on an indication of a user intent associated with the one or more requests.

6. The method recited in claim 1, the operations are performed on a per-application basis.

7. The method recited in claim 1, the operations comprising preventing an application from opening a file for which a placeholder has been created where the data of the file does not reside in the placeholder on the secondary storage.

8. The method recited in claim 1, the operations comprising, for at least one of the requests determined to require retrieval of data from the remote storage:

transmitting to a storage virtualization provider one or more corresponding requests to retrieve the remotely stored data;

receiving, from the storage virtualization provider, the requested data; and

providing the requested data directly to the application requesting the data, without writing the received data to the secondary storage.

9. The method recited in claim 1, the operations comprising, for any placeholders on the secondary storage that contain at least some data of the file they represent, selecting at least one of the placeholders and moving the data contained in the at least one selected placeholder from the secondary storage to remote storage.

10. The method recited in claim 9, the at least one of the placeholders being selected based on criteria including one or more of age of the data, the time the data was last accessed, whether the data is synchronized with the remote storage, and an intent of a user.

11. The method recited in claim 1, the operations comprising, for any files that are fully stored on the secondary storage, converting at least one of those files to a placeholder and moving at least some of the data of the file to remote storage.

12. A computing device comprising a processor, a memory, and secondary storage, the memory storing executable instructions that, when executed by the processor, cause the computing device to perform operations comprising:

storing, on the secondary storage, a plurality of placeholder for a respective plurality of files, each file comprising data which is stored on a network remotely from the secondary storage, each placeholder containing metadata associated with its respective file, a sparse data stream containing none or some data of the respective file, and information which enables any remotely stored data of the respective file to be retrieved from the network;

receiving, from one or more applications executing on the processor of the computing device, one or more requests, each request comprising a request to access a file for which a placeholder is stored on the secondary storage;

determining, for each received request to access a file, whether the request requires that data of the file be retrieved from remote storage on the network;

monitoring a level of such requests requiring retrieval of data from the remote storage;

determining that the monitored level of requests has satisfied a threshold; and

performing operations, in response to determining that the threshold has been satisfied, to reduce an impact of such level of requests on resources of the computing device.

13. The computing device recited in claim 12, wherein determining that the threshold is satisfied is based on a number of received requests requiring data to be retrieved from remote storage.

14. The computing device recited in claim 12, wherein determining that the threshold is satisfied is based on one of a fixed amount or a percentage of available resources of the computing device.

15. The computing device recited in claim 14, wherein determining that the threshold is satisfied is further based on an indication of a user intent associated with the one or more requests.

16. The computing device recited in claim 12, the operations comprising preventing an application from opening a file for which a placeholder has been created where the data of the file does not reside in the placeholder on the secondary storage.

17. The computing device recited in claim 12, the operations comprising, for at least one of the requests determined to require retrieval of data from the remote storage:

transmitting to a storage virtualization provider one or more corresponding requests to retrieve the remotely stored data;

receiving, from the storage virtualization provider, the requested data; and

providing the requested data directly to the application requesting the data, without writing the received data to the secondary storage.

18. The computing device recited in claim 12, the operations comprising, for any placeholders on the secondary storage that contain at least some data of the file they represent, selecting at least one of the placeholders and moving the data contained in the at least one selected placeholder from the secondary storage to remote storage.

19. The computing device recited in claim 18, the at least one of the placeholders being selected based on criteria including one or more of age of the data, the time the data was last accessed, whether the data is synchronized with the remote storage, and an intent of a user.

20. The computing device recited in claim 12, the operations comprising, for any files that are fully stored on the secondary storage, converting at least one of those files to a placeholder and moving at least some of the data of the file to remote storage.