Dynamic selection of storage tiers

• US 9,460,099 B2
• Filed: 11/13/2012
• Issued: 10/04/2016
• Est. Priority Date: 11/13/2012
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Thomas, Nathan Bartholomew
• Primary Examiner(s): Farrokh, Hashem
• Application Number:
  US13/675,718
• Publication Number:
  US 20140136782A1
• Time in Patent Office: 1,421 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0664 at device level, e.g. emulation of a storage device or system
• CPC Class Codes: G06F 3/0667 at data level, e.g. file, record or object virtualisation
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

An operating system is configured to receive a request to store an object that does not specify the location at which the object should be stored. The request might also include an optimization factor and one or more object location factors. The operating system might also generate object location factors or retrieve object location factors from one or more external locations. Object location factors might also be utilized that are based upon properties of the object to be stored. Utilizing the object location factors, and the optimization factor if provided, the operating system dynamically selects an appropriate storage tier for storing the object. The tiers might include a local storage tier, a local network storage tier, a remote network storage tier, and other types of storage tiers. The object is then stored on the selected storage tier. The object may be retrieved from the storage tier at a later time.

Claims

1. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to:

execute an operating system configured to store objects at a local storage tier, a local network storage tier, or a remote network storage tier, the local storage tier comprising one or more memories and mass storage devices of the computer, and the remote network storage tier comprising one or more storage resources provided by one or more data centers;

receive, at the operating system, an unqualified request to store an object without specifying a location for storing the object, the unqualified request being received from an application executing on the computer where the operating system executes and comprising the object to be stored, an optimization factor and at least one of one or more object location factors;

in response to receiving the request to store the object, utilize at least the one or more object location factors to determine a selected storage tier for the object, the selected storage tier is selected from the local storage tier, the local network storage tier, or the remote network storage tier;

cause the object to be stored at the selected storage tier;

create an entry in a lookup table that indicates the object is stored at the selected storage tier;

receive a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilize the lookup table to identify the selected storage tier for the object, retrieve the object from the selected storage tier, and provide the object in response to the request to retrieve the object.

2. The non-transitory computer-readable storage medium of claim 1, wherein the local network storage tier comprises one or more storage systems accessible to the computer by way of a local area network (LAN).

3. The non-transitory computer-readable storage medium of claim 1, wherein the remote network storage tier comprises one or more storage services accessible to the computer.

4. The non-transitory computer-readable storage medium of claim 3, wherein the storage services are accessible to the computer by way of a web service.

5. The non-transitory computer-readable storage medium of claim 4, wherein one or more of the object location factors are retrieved from the storage services.

6. The non-transitory computer-readable storage medium of claim 1, wherein selecting the selected storage tier from the local storage tier, the local network storage tier, or the remote network storage tier is further based at least in part upon the optimization factor.

7. The non-transitory computer-readable storage medium of claim 1, wherein the one or more object location factors comprise one or more of a type of the object, a size of the object, file attributes of the object, security requirements of the object, sharing requirements of the object, data describing historical usage of the object, or the compressibility of the object.

8. The non-transitory computer-readable storage medium of claim 1, wherein the one or more object location factors comprise one or more of a cost of storing the object at a storage tier, a latency when storing the object or retrieving the object from a storage tier, or available bandwidth to a storage tier.

9. The non-transitory computer-readable storage medium of claim 1, wherein the operating system is executing in a virtual machine instance.

10. The non-transitory computer-readable storage medium of claim 9, wherein the virtual machine instance is executing in a distributed execution environment.

11. A computer-implemented method for dynamically selecting a storage tier for storing an object from a plurality of storage tiers, the method comprising performing computer-implemented operations for:

receiving, at an operating system executing on a computing device, an unqualified request to store an object without specifying a location for storing the object, the unqualified request being received from an application executing on the computing device and comprising the object to be stored, an optimization factor and at least one of one or more object location factors;

in response to receiving the request, selecting by way of the operating system a selected storage tier from a plurality of storage tiers for storing the object based upon the one or more object location factors, the plurality of storage tiers comprising a local storage tier that comprises one or more memories and mass storage devices of the computing device, a local network storage tier, and a remote network storage tier that comprises one or more storage resources provided by one or more data centers;

causing the object to be stored at the selected storage tier;

creating an entry in a lookup table that indicates the object is stored at the selected storage tier;

receiving a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilizing the lookup table to identify the selected storage tier for the object, retrieving the object from the selected storage tier, and providing the object in response to the request to retrieve the object.

12. The computer-implemented method of claim 11, wherein the selection of the storage tier for storing the object is further based upon the optimization factor.

13. The computer-implemented method of claim 11, wherein the operating system is executing in a virtual machine instance.

14. The computer-implemented method of claim 13, wherein the virtual machine instance is executing in a distributed execution environment.

15. The computer-implemented method of claim 11, wherein the remote network storage tier comprises a storage service accessible to the computing device by way of a web service.

16. The computer-implemented method of claim 15, wherein one or more of the object location factors are retrieved from the storage service.

17. An apparatus:

comprising;

a processor; and

a non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the apparatus to;

receive, by an operating system executing on the apparatus, an unqualified request to store an object without specifying a location for storing the object, the unqualified request being received from an application executing on the apparatus and comprising the object to be stored, an optimization factor and at least one of one or more object location factors;

in response to receiving the request, select, by the operating system, a selected storage location for storing the object from a local storage location and a network storage location, wherein selecting the storage location is based at least in part on the one or more object location factors, wherein the local storage location comprises one or more memories and mass storage devices of the apparatus, and wherein the network storage location comprises one or more storage resources provided by one or more data centers;

cause the object to be stored at the selected storage location;

create an entry in a lookup table that indicates the object is stored at the selected storage location;

receive a request to retrieve the object; and

in response to receiving the request to retrieve the object, utilize the lookup table to identify the selected storage location for the object, retrieve the object from the selected storage location, and provide the object in response to the request to retrieve the object.

18. The apparatus of claim 17, wherein the network storage location comprises a storage service accessible to the apparatus by way of a web service.

19. The apparatus of claim 18, wherein one or more of the object location factors are retrieved from the storage service.

20. The apparatus of claim 17, wherein one or more of the object location factors are generated by monitoring communication with the local storage location and the network storage location.

21. The apparatus of claim 17, wherein the object location factors comprise one or more of an object type, an object size, object file attributes, object security requirements, object sharing requirements, data describing historical usage of the object, the compressibility of the object, a cost of storing the object at a storage location, a latency when storing the object or retrieving the object from a storage location, or available bandwidth to a storage location.

22. The apparatus of claim 17, wherein the network storage location comprises one or more storage systems accessible to the apparatus by way of a local area network (LAN).

23. The apparatus of claim 17, wherein the operating system executes in a virtual machine instance.

24. The apparatus of claim 23, wherein the virtual machine instance is executing in a distributed execution environment.