Dynamic selection of storage tiers

• US 10,642,491 B2
• Filed: 09/27/2016
• Issued: 05/05/2020
• Est. Priority Date: 11/13/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Thomas, Nathan Bartholomew
• Primary Examiner(s): Farrokh, Hashem
• Application Number:
  US15/277,588
• Publication Number:
  US 20180314429A1
• Time in Patent Office: 1,316 Days
• Field of Search: 711117
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An operating system is configured to receive a request to store an object that does not specify the location at which the object should be stored. The request might also include an optimization factor and one or more object location factors. The operating system might also generate object location factors or retrieve object location factors from one or more external locations. Object location factors might also be utilized that are based upon properties of the object to be stored. Utilizing the object location factors, and the optimization factor if provided, the operating system dynamically selects an appropriate storage tier for storing the object. The tiers might include a local storage tier, a local network storage tier, a remote network storage tier, and other types of storage tiers. The object is then stored on the selected storage tier. The object may be retrieved from the storage tier at a later time.

Claims

1. A computer-implemented method for dynamically selecting a storage tier for storing an object from a plurality of storage tiers, the method comprising performing computer-implemented operations for:

receiving, via an Application Programming Interface (API) exposed by an operating system executing on a computing device, a request to store an object at an unspecified location, the request being received from an application executing on the computing device and the request comprising the object to be stored, one or more object location factors, and an optimization factor that identifies one or more attributes associated with selecting the storage tier for storing the object;

monitoring a plurality of object location factors for the plurality of storage tiers, the plurality of storage tiers including a local storage tier including a memory of a local storage device and a mass storage device of the local storage device, a local network storage tier, and a remote network storage tier that comprises one or more storage resources provided by one or more data centers, wherein monitoring the plurality of object location factors includes monitoring a usage pattern associated with the object and the one or more object location factors; and

in response to receiving the request, selecting by way of the operating system the selected storage tier from the plurality of storage tiers for storing the object based at least in part upon a type of the object, at least one of the plurality of the object location factors and the one or more object location factors, and the optimization factor;

causing the object to be stored at the selected storage tier;

creating data that indicates the object is stored at the selected storage tier;

receiving a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilizing the data to identify the selected storage tier for the object, retrieving the object from the selected storage tier, and providing the object in response to the request to retrieve the object.

2. The computer-implemented method of claim 1, wherein the selection of the storage tier for storing the object is further based upon the optimization factor received via the API, and one or more security requirements.

3. The computer-implemented method of claim 1, further comprising:

storing historical data that indicates historical usage patterns associated with the object and wherein the selecting by way of the operating system the selected storage tier from the plurality of storage tiers for storing the object is further based at least in part upon at least a portion of the historical data.

4. The computer-implemented method of claim 1, wherein the operating system executes on a virtual machine instance that executes in a distributed execution environment.

5. The computer-implemented method of claim 1, wherein the remote network storage tier comprises a storage service accessible to the computing device by way of a web service.

6. The computer-implemented method of claim 5, wherein one or more of the object location factors are retrieved from the storage service.

7. An apparatus, comprising:

a processor; and

a non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the apparatus to;

receive, via an Application Programming Interface (API) exposed by an operating system executing on the apparatus, a request to store an object at an unspecified location, the request being received from an application executing on the apparatus and including the object to be stored, one or more object location factors, and an optimization factor that identifies one or more attributes associated with selecting the storage tier for storing the object;

in response to receiving the request, select, by the operating system, a selected storage location for storing the object based at least in part upon a type of the object, the one or more object location factors, and the optimization factor, wherein the selected storage location is selected from a plurality of storage locations including a local storage location and a network storage location, wherein the local storage location comprises one or more memories and one or more mass storage devices of the apparatus and the network storage location comprises one or more storage resources provided by one or more data centers;

cause the object to be stored at the selected storage location;

create data that indicates the object is stored at the selected storage location;

receive a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilize the data to identify the selected storage location for the object, retrieve the object from the selected storage location, and provide the object in response to the request to retrieve the object.

8. The apparatus of claim 7, wherein selecting the storage location is based at least in part on historical data that indicates historical usage patterns associated with the object, and the one or more object location factors that include one or more of a sharing requirement or a security requirement for the object.

9. The apparatus of claim 8, wherein the network storage location comprises a storage service accessible to the apparatus by way of a web service.

10. The apparatus of claim 9, wherein at least one of the one or more object location factors are retrieved from the storage service.

11. The apparatus of claim 8, wherein the request further comprises at least one of the one or more object location factors.

12. The apparatus of claim 7, wherein at least one of the one or more object location factors are generated by monitoring communication with the local storage location and the network storage location.

13. The apparatus of claim 7, wherein the object location factors comprise one or more of an object type, object security requirements, object sharing requirements, or the compressibility of the object.

14. The apparatus of claim 7, wherein the network storage location comprises one or more storage systems accessible to the apparatus by way of a local area network (LAN).

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to:

receive, via an Application Programming Interface (API) exposed by an operating system executing on the computer, a request to store an object at an unspecified location, the request being received from an application executing on the computer and including the object to be stored, one or more object location factors, and an optimization factor that identifies one or more attributes associated with selecting the storage tier for storing the object;

in response to receiving the request to store the object, determine a selected storage tier for the object based at least in part upon a type of the object, and one or more object location factors, the selected storage tier is selected from a local storage tier that comprises one or more memories of the computer, one or more mass storage devices of the computer, and one or more of a local network storage tier, or a remote network storage tier;

cause the object to be stored at the selected storage tier;

create data that indicates the object is stored at the selected storage tier;

receive a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilize the data to identify the selected storage tier for the object, retrieve the object from the selected storage tier, and provide the object in response to the request to retrieve the object.

16. The non-transitory computer-readable storage medium of claim 15, wherein the local network storage tier comprises one or more storage systems accessible to the computer by way of a local area network (LAN).

17. The non-transitory computer-readable storage medium of claim 15, wherein selecting the selected storage tier from the local storage tier, the local network storage tier, or the remote network storage tier is based at least in part upon historical data that indicates historical usage patterns associated with the object, and the one or more object location factors including one or more of a security requirement or a sharing requirement for the object.

18. The non-transitory computer-readable storage medium of claim 17, wherein the remote network storage tier comprises one or more storage services accessible to the computer.

19. The non-transitory computer-readable storage medium of claim 17, wherein one or more of the object location factors are retrieved from the storage services.

20. The non-transitory computer-readable storage medium of claim 17, wherein the one or more object location factors comprise one or more of security requirements of the object, sharing requirements of the object, or a compressibility of the object.