Systems and methods for managing an upload of files in a shared cache storage system

• US 8,549,229 B2
• Filed: 09/30/2010
• Issued: 10/01/2013
• Est. Priority Date: 08/19/2010
• Status: Expired due to Fees

• Current Assignee: Western Digital Israel Limited (Western Digital Corporation)
• Original Assignee: SanDisk IL Ltd. (Western Digital Corporation)
• Inventors: Meza, Joseph R., Hahn, Judah Gamliel, Hutton, Henry, Sherry, Leah
• Primary Examiner(s): CHOE, YONG J
• Application Number:
  US12/895,397
• Publication Number:
  US 20120047331A1
• Time in Patent Office: 1,097 Days
• Field of Search: 711/134, 711/118, 711/E12.022
• US Class Current: 711/134
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 2212/171 Portable consumer electronics, e.g. mobile phone
• CPC Class Codes: G06F 2212/463 File
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics

Abstract

Systems and methods for managing a storage device are disclosed. Generally, in a host to which a storage device is operatively coupled, wherein the storage device includes a cache for storing one or more discardable files, a file is identified to be uploaded to an external location. A determination is made whether sufficient free space exists in the cache to pre-stage the file for upload to the external location and the file is stored in the cache upon determining that sufficient free space exists in the cache to pre-stage the file for upload to the external location, wherein pre-stating prepares a file for opportunistically uploading such file in accordance with an uploading policy.

Claims

1. A method for managing files with a storage device, the method comprising:

in a host to which a storage device is operatively coupled, wherein the storage device includes a cache for storing one or more discardable files;

identifying a file to be uploaded to an external location;

determining whether sufficient free space exists in the cache to pre-stage the file for upload to the external location;

determining that pre-staging the file for uploading to the external location requires more space than free space exists in the cache;

associating discarding criteria with the file;

determining whether one or more discardable files are stored in the cache that are associated with discarding criteria that is higher than that of the file to be pre-staged;

removing one or more discardable files from the cache that are associated with discarding criteria that is higher than that of the file to be pre-staged to free UP space in the cache for the file to be pre-staged for uploading, starting with the discardable file associated with the highest discarding criteria and proceeding with the discardable file associated with the next highest discarding criteria, in sequence, wherein no more discardable files are removed from the cache than are needed to free up space in the cache for the file to be pre-staged; and

storing the file in the cache upon determining that sufficient free space exists in the cache to pre-stage the file for upload to the external location, wherein pre-staging prepares a file for opportunistically uploading such file in accordance with an uploading policy.

2. The method of claim 1, wherein discardable files in the cache include pre-staged files and unsolicited files, wherein all discardable files can be discarded without notice but pre-staged files are discarded only after they are uploaded to an external location.

3. The method of claim 1, wherein the uploading policy comprises uploading when communication with the external location is off-peak.

4. The method of claim 1, wherein uploading of pre-staged files frees up space in the cache.

5. The method of claim 1, wherein the discarding criteria comprises a priority level.

6. The method of claim 1, further comprising:

determining that, after removing one or more discardable files from the cache, sufficient free space does not exist in the cache to pre-stage the file for upload to the external location;

determining whether one or more pre-staged files already stored in the cache may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location;

uploading the one or more pre-staged files already stored in the cache upon determining that the one or more pre-staged files may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location; and

storing the file in the cache after uploading the one or more pre-staged files to external locations.

7. The method of claim 6, further comprising:

determining whether the file can be uploaded immediately to the external location when pre-staged files are not stored in the cache that may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location;

uploading the file to the external location upon determining the file can be uploaded immediately; and

generating a failed upload indicator upon determining the file can not be uploaded immediately.

8. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

determining whether the one or more pre-staged files may be uploaded to external locations based on an amount of network bandwidth available to upload the one or more pre-staged files to external locations.

9. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

determining whether the one or more pre-staged files may be uploaded to external locations based on a power level associated with a host device.

10. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

determining whether the one or more pre-staged files may be uploaded to external locations based on at least one of a number of read requests, a number of erases, or a number of writes associated with the cache.

11. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

determining whether the one or more pre-staged files may be uploaded to external locations based on a security parameter associated with the storage device.

12. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

determining whether the one or more pre-staged files may be uploaded to external locations based on a number of host applications currently accessing the cache.

13. The method of claim 6, wherein determining whether the one or more pre-staged files already stored in the cache may be uploaded to external locations comprises:

querying the storage device for one or more metrics associated with the cache; and

determining whether the one or more pre-staged files may be uploaded to external locations based on the one or more metrics.

14. The method of claim 13, wherein the one or more metrics comprise at least one of a number of read requests, a number of erases, or a number of writes associated with the cache.

15. A storage system comprising:

a communication interface;

a processor on a host for managing a cache of a storage device, wherein the cache is configured to store one or more discardable files and wherein the processor is configured to;

identify a file to be uploaded over the communication interface to an external location;

determine whether sufficient free space exists in the cache to pre-stage the file for upload to the external location;

determine that pre-staging the file for upload to the external location requires more space than free space exists in the cache;

associate discarding criteria with the file;

determine whether one or more discardable files are stored in the cache that are associated with discarding criteria that is higher than that of the file to be pre-staged;

remove one or more discardable files from the cache that are associated with discarding criteria that is higher than that of the file to be pre-staged to free UP space in the cache for the file to be pre-staged for uploading, starting with the discardable file associated with the highest discarding criteria and proceeding with the discardable file associated with the next highest discarding criteria, in sequence, wherein no more discardable files are removed from the cache than are needed to free up space in the cache for the file to be pre-staged; and

store the file in the cache upon determining that sufficient free space exists in the cache to pre-stage the file for upload to the external location, wherein pre-staging prepares a file for opportunistically uploading such file in accordance with an uploading policy.

16. The storage system of claim 15, wherein discardable files in the cache include pre-staged files and unsolicited files, wherein all discardable files can be discarded without notice but pre-staged files are discarded only after they are uploaded to an external location.

17. The storage system of claim 15, wherein the uploading policy comprises uploading when communication with the external location is off-peak.

18. The storage system of claim 15, wherein uploading of pre-staged files frees up space in the cache.

19. The storage system of claim 15, wherein the discarding criteria comprises a priority level.

20. The storage system of claim 15, wherein the processor is further configured to:

determine that, after removing one or more discardable files from the cache, sufficient free space does not exist in the cache to pre-stage the file for upload to the external location;

determine whether one or more pre-staged files already stored in the cache may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location;

upload the one or more pre-staged files already stored in the cache upon determining that the one or more pre-staged files may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location; and

store the file in the cache after uploading the one or more pre-staged files to external locations.

21. The storage system of claim 20, wherein the processor is further configured to:

determine whether the file can be uploaded immediately to the external location when pre-staged files are not stored in the cache that may be uploaded to external locations to create sufficient free space in the cache to pre-stage the file for upload to the external location;

upload the file to the external location upon determining the file can be uploaded immediately; and

generate a failed upload indicator upon determining the file can not be uploaded immediately.

22. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

determine whether the one or more pre-staged files may be uploaded to external locations based on an amount of network bandwidth available to upload the one or more pre-staged files to external locations.

23. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

determine whether the one or more pre-staged files may be uploaded to external locations based on a power level associated with a host device.

24. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

determine whether the one or more pre-staged files may be uploaded to external locations based on at least one of a number of read requests, a number of erases, or a number of writes associated with the cache.

25. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

determine whether the one or more pre-staged files may be uploaded to external locations based on a security parameter associated with the storage device.

26. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

determine whether the one or more pre-staged files may be uploaded to external locations based on a number of host applications currently accessing the cache.

27. The storage system of claim 20, wherein to determine whether the one or more pre-staged files already stored in the cache may be uploaded to external locations, the processor is further configured to:

query the storage device for one or more metrics associated with the cache; and

determine whether the one or more pre-staged files may be uploaded to external locations based on the one or more metrics.

28. The storage system of claim 27, wherein the one or more metrics comprise at least one of a number of read requests, a number of erases, or a number of writes associated with the cache.