Dynamic data relocation using cloud based ranks

• US 10,372,371 B2
• Filed: 09/14/2017
• Issued: 08/06/2019
• Est. Priority Date: 09/14/2017
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gupta, Lokesh, Borlick, Matthew G.
• Primary Examiner(s): Giardino, Jr., Mark A
• Application Number:
  US15/704,775
• Publication Number:
  US 20190079693A1
• Time in Patent Office: 691 Days
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data

Abstract

An example method for dynamic data relocation using cloud based ranks comprises monitoring accesses to data stored on a plurality of local ranks of an enterprise storage system; identifying data which has not been accessed for a predetermined amount of time based on the monitored accesses; and moving the data which has not been accessed for the predetermined amount of time to one or more cloud based ranks of the enterprise storage system, wherein each cloud based rank comprises storage space on one or more cloud storage devices, the storage space on the one or more cloud storage devices mapped to corresponding virtual local addresses that are grouped as a virtual local rank.

Claims

1. A computer-implemented method for dynamic data relocation, the method comprising:

monitoring accesses to data stored on a plurality of local ranks of an enterprise storage system;

based on the monitored accesses, identifying data which has not been accessed for a predetermined amount of time;

moving the data which has not been accessed for the predetermined amount of time to one or more cloud based ranks of the enterprise storage system, wherein each cloud based rank comprises storage space on one or more cloud storage devices, the storage space on the one or more cloud storage devices mapped to corresponding virtual local addresses that are grouped as a virtual local rank;

wherein the virtual local addresses are memory addresses which appear as addresses of a storage device coupled to a storage controller via a local connection;

in response to an access request directed to the corresponding virtual local addresses for at least part of the moved data, converting the access request to a cloud data access request configured for an application programming interface (API) corresponding to the one or more cloud storage devices of the one or more cloud based ranks;

relocating the at least part of the moved data from the one or more cloud based ranks to at least one of the plurality of local ranks;

determining that all of the moved data has been relocated to at least one of the plurality of local ranks; and

deleting the one or more cloud based ranks.

2. The computer-implemented method of claim 1, wherein the predetermined amount of time comprises one of a predetermined number of weeks or a predetermined number of days.

3. The computer-implemented method of claim 1, wherein moving the data comprises:

in response to determining that a cloud based rank having sufficient storage space is not available, creating one or more new cloud based ranks;

wherein creating each of the one or more new cloud based ranks comprises;

allocating storage space on one or more corresponding cloud storage devices via a cloud interface;

mapping the allocated storage space to corresponding virtual local addresses; and

grouping the virtual local addresses as a virtual local rank.

4. The computer-implemented method of claim 1, wherein moving the data comprises moving the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that the size of the data which has not been accessed for the predetermined amount of time exceeds a predetermined size threshold.

5. The computer-implemented method of claim 1, wherein moving the data comprises moving the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that available space on the plurality of local ranks is less than a predetermined available space threshold.

6. A storage system comprising:

a host adapter having one or more ports configured to communicatively couple the host adapter to one or more host devices;

a storage controller comprising a processor and a memory, the storage controller communicatively coupled to the host adapter; and

a device adapter comprising a processor and a memory, the device adapter communicatively coupled to the storage controller;

wherein the device adapter further comprises a plurality of ports communicatively coupled to a plurality of local storage drives and at least one network port communicatively coupled to a plurality of cloud storage devices via a network;

wherein one of the device adapter and the storage controller is further configured implement a data relocation module configured to;

monitor accesses to data stored on a plurality of local redundant array of independent disks (RAID) arrays of an enterprise storage system;

based on the monitored accesses, identify data which has not been accessed for a predetermined amount of time;

move the data which has not been accessed for the predetermined amount of time to one or more cloud based RAID arrays of the enterprise storage system, wherein each cloud based RAID array comprises storage space on one or more cloud storage devices, the storage space on the one or more cloud storage devices mapped to corresponding virtual local addresses that are grouped as a virtual local RAID array;

wherein the virtual local addresses are grouped such that the storage space on the one or more cloud storage devices appears to the storage controller as a local array of storage devices connected to the storage controller without an intervening wide area network;

in response to an access request directed to the corresponding virtual local addresses for at least part of the moved data, relocate the at least part of the moved data from the one or more cloud based ranks to at least one of the plurality of local ranks; and

delete the one or more cloud based ranks in response to determining that all of the moved data has been relocated to at least one of the plurality of local ranks.

7. The storage system of claim 6, wherein the relocation module is configured to move the data by:

creating one or more new cloud based ranks in response to determining that a cloud based rank having sufficient storage space is not available;

wherein the relocation module is configured to create each of the one or more new cloud based ranks by;

allocating storage space on one or more corresponding cloud storage devices via a cloud interface;

mapping the allocated storage space to corresponding virtual local addresses; and

grouping the virtual local addresses as a virtual local rank.

8. The storage system of claim 6, wherein the relocation module is configured to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that the size of the data which has not been accessed for the predetermined amount of time exceeds a predetermined size threshold.

9. The storage system of claim 6, wherein the relocation module is configured to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that available space on the plurality of local ranks is less than a predetermined available space threshold.

10. The storage system of claim 6, wherein the predetermined amount of time comprises one of a predetermined number of weeks or a predetermined number of days.

11. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed by a processor, causes the processor to:

monitor accesses to data stored on a plurality of local redundant array of independent disks (RAID) arrays of an enterprise storage system;

based on the monitored accesses, identify data which has not been accessed for a predetermined amount of time;

move the data which has not been accessed for the predetermined amount of time to one or more cloud based RAID arrays of the enterprise storage system, wherein each cloud based RAID array comprises storage space on one or more cloud storage devices, the storage space on the one or more cloud storage devices mapped to corresponding virtual local addresses that are grouped as a virtual local RAID array such that the storage space on the one or more cloud storage devices appears to the storage controller as a local RAID array connected to the storage controller via a local connection;

in response to an access request directed to the corresponding virtual local addresses for at least part of the moved data, convert the access request to a cloud data access request configured for an application programming interface (API) corresponding to the one or more cloud storage devices of the one or more cloud based ranks;

relocate the at least part of the moved data from the one or more cloud based ranks to at least one of the plurality of local ranks; and

delete the one or more cloud based ranks in response to determining that all of the moved data has been relocated to at least one of the plurality of local ranks.

12. The computer program product of claim 11, wherein the computer readable program is further configured to cause the processor to move the data by:

creating one or more new cloud based ranks in response to determining that a cloud based rank having sufficient storage space is not available;

wherein the relocation module is configured to create each of the one or more new cloud based ranks by;

allocating storage space on one or more corresponding cloud storage devices via a cloud interface;

mapping the allocated storage space to corresponding virtual local addresses; and

grouping the virtual local addresses as a virtual local rank.

13. The computer program product of claim 11, wherein the computer readable program is further configured to cause the processor to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that the size of the data which has not been accessed for the predetermined amount of time exceeds a predetermined size threshold.

14. The computer program product of claim 11, wherein the computer readable program is further configured to cause the processor to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that available space on the plurality of local ranks is less than a predetermined available space threshold.

15. The computer program product of claim 11, wherein the predetermined amount of time comprises one of a predetermined number of weeks or a predetermined number of days.

16. A computing device comprising:

a network adapter configured to communicatively couple the computing device to one or more cloud storage devices via a network;

a storage medium configured to store data; and

a processor communicatively coupled to the network adapter and to the storage medium, wherein the processor is configured to;

monitor accesses to data stored on a plurality of local ranks of an enterprise storage system;

based on the monitored accesses, identify data which has not been accessed for a predetermined amount of time;

move the data which has not been accessed for the predetermined amount of time to one or more cloud based ranks of the enterprise storage system, wherein each cloud based rank comprises storage space on one or more cloud storage devices, the storage space on the one or more cloud storage devices mapped to corresponding virtual local addresses that are grouped as a virtual local rank such that the storage space on the one or more cloud storage devices appears to a storage controller as a local array of storage devices connected to the storage controller via a local connection;

in response to an access request directed to the corresponding virtual local addresses for at least part of the moved data, convert the access request to a cloud data access request configured for an application programming interface (API) corresponding to the one or more cloud storage devices of the one or more cloud based ranks; and

relocate the at least part of the moved data from the one or more cloud based ranks to at least one of the plurality of local ranks; and

delete the one or more cloud based ranks in response to determining that all of the moved data has been relocated to at least one of the plurality of local ranks.

17. The computing device of claim 16, wherein the processor is further configured to move the data by:

creating one or more new cloud based ranks in response to determining that a cloud based rank having sufficient storage space is not available;

wherein the processor is configured to create each of the one or more new cloud based ranks by;

allocating storage space on one or more corresponding cloud storage devices via an application programming interface (API) associated with the corresponding one or more cloud storage devices; and

generating a map table which maps the allocated storage space to corresponding virtual local addresses and groups the virtual local addresses as a corresponding virtual local rank.

18. The computing device of claim 16, wherein the processor is further configured to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that the size of the data which has not been accessed for the predetermined amount of time exceeds a predetermined size threshold.

19. The computing device of claim 16, wherein the processor is further configured to move the data which has not been accessed for the predetermined amount of time to the one or more cloud based ranks in response to determining that available space on the plurality of local ranks is less than a predetermined available space threshold.

20. The computing device of claim 16, wherein the predetermined amount of time comprises one of a predetermined number of weeks or a predetermined number of days.