Congestion mitigation in a multi-tiered distributed storage system

• US 9,645,761 B1
• Filed: 03/04/2016
• Issued: 05/09/2017
• Est. Priority Date: 01/28/2016
• Status: Active Grant

• Current Assignee: Weka.IO Ltd.
• Original Assignee: Weka.IO Ltd.
• Inventors: Ben Dayan, Maor, Palmon, Omri, Zvibel, Liran, Arditti, Kanael, Filiba, Tomer
• Primary Examiner(s): DOAN, DUC T
• Application Number:
  US15/061,518
• Time in Patent Office: 431 Days
• Field of Search: None
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0673 Single storage device
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: H04L 43/0882 Utilisation of link capacity
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04L 47/12 Avoiding congestion; Recovering from congestion
• CPC Class Codes: H04L 47/6275 based on priority
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/83 : based on usage prediction
• CPC Class Codes: H04L 49/90 : Buffering arrangements
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/06 : specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A distributed electronic storage system (DESS) comprises congestion management circuitry and data migration circuitry. The congestion management circuitry is operable to determine an amount of congestion in the DESS. The data migration circuitry is operable to control migration of data stored in a first tier of storage to a second tier of storage based on the amount of congestion in the DESS, characteristics of the data, and characteristics of the first tier of storage.

Claims

1. A method comprising:

determining, by congestion management circuitry of a distributed electronic storage system (DESS), an amount of congestion in the DESS;

controlling, by data migration circuitry of the DESS, migration of data stored in a first tier of storage of the DESS to a second tier of storage of the DESS based on;

the amount of congestion in the DESS;

characteristics of the data; and

characteristics of the first tier of storage.

2. The method of claim 1, wherein the controlling the migration comprises:

triggering the migration upon concurrent occurrence of;

the amount of congestion in the DESS being less than a determined threshold; and

the data having a first particular characteristic.

3. The method of claim 2, wherein the particular characteristic is one of:

amount of time since creation of the data being greater than a determined threshold;

amount of time since last modification of the data being greater than a determined threshold;

amount of time since last access of the data being greater than a determined threshold;

the data being of a particular size;

the data being of a particular type;

the data having been generated by a particular client process; and

the data having been generated by a particular node of the DESS.

4. The method of claim 1, wherein the determining the amount of congestion in the DESS comprises:

determining a load on a local resource of the DESS; and

determining a load on a remote resource of the DESS based on an indication of the load on the remote resource of the DESS received via a network link.

5. The method of claim 1, wherein the characteristics of the first tier of storage comprise an amount of unused space in the first tier of storage.

6. The method of claim 5, wherein the controlling the migration comprises:

when the amount of unused space in the first tier of storage is below a first threshold, triggering the migration regardless of the amount of congestion of the DESS; and

when the amount of unused space in the first tier of storage is above the first threshold, delaying the migration until the amount of congestion in the DESS is below the first threshold.

7. The method of claim 1, comprising determining, by data migration circuitry of the DESS, whether to store data read from the second tier of storage in the first tier of storage based on the amount of congestion in the DESS.

8. The method of claim 7, comprising storing the data read from the second tier of storage to the first tier of storage only when the amount congestion in the DESS is below a determined threshold.

9. A system comprising:

congestion management circuitry of a distributed electronic storage system (DESS), the congestion management circuitry being operable to determine an amount of congestion in the DESS;

data migration circuitry of the DESS, the data migration circuitry being operable to control migration of data stored in a first tier of storage of the DESS to a second tier of storage of the DESS based on;

the amount of congestion in the DESS;

characteristics of the data; and

characteristics of the first tier of storage.

10. The system of claim 9, wherein, as part of the control of the migration, the data migration circuitry is operable to:

trigger the migration upon concurrent occurrence of;

the amount of congestion in the DESS being less than a determined threshold; and

the data having a first particular characteristic.

11. The system of claim 10, wherein the particular characteristic is one of:

amount of time since creation of the data being greater than a determined threshold;

amount of time since last modification of the data being greater than a determined threshold;

amount of time since last access of the data being greater than a determined threshold;

the data being of a particular size;

the data being of a particular type;

the data having been generated by a particular client process; and

the data having been generated by a particular node of the DESS.

12. The system of claim 9, wherein, as part of the determination of the amount of congestion in the DESS comprises, the congestion management circuitry is operable to:

determine a load on a local resource of the DESS; and

determine a load on a remote resource of the DESS based on an indication of the load on the remote resource of the DESS received via a network link.

13. The system of claim 9, wherein the characteristics of the first tier of storage comprise an amount of unused space in the first tier of storage.

14. The system of claim 13, wherein, as part of the control of the migration, the data migration circuitry is operable to:

when the amount of unused space in the first tier of storage is below a first threshold, trigger the migration regardless of the amount of congestion of the DESS; and

when the amount of unused space in the first tier of storage is above the first threshold, delay the migration until the amount of congestion in the DESS is below the first threshold.

15. The system of claim 9, wherein the data migration circuitry is operable to determine whether to store data read from the second tier of storage in the first tier of storage based on the amount of congestion in the DESS.

16. The system of claim 15, wherein the data migration circuitry is configured to store the data read from the second tier of storage to the first tier of storage only when the amount congestion in the DESS is below a determined threshold.

17. One or more non-transitory machine-readable storage medium having code stored thereon, that, when executed by one or more computing devices of a distributed electronic storage system (DESS), configures the one or more computing devices to comprise DESS congestion management circuitry and DESS data migration circuitry, wherein:

the DESS congestion management circuitry is operable to determine an amount of congestion in the DESS;

the DESS data migration circuitry is operable to control migration of data stored in a first tier of storage of the DESS to a second tier of storage of the DESS based on;

the amount of congestion in the DESS;

characteristics of the data; and

characteristics of the first tier of storage.

18. The one or more non-transitory machine-readable storage medium of claim 17, wherein, as part of the control of the migration, the data migration circuitry is operable to:

trigger the migration upon concurrent occurrence of;

the amount of congestion in the DESS being less than a determined threshold; and

the data having a first particular characteristic.

19. The one or more non-transitory machine-readable storage medium of claim 18, wherein the particular characteristic is one of:

amount of time since creation of the data being greater than a determined threshold;

amount of time since last modification of the data being greater than a determined threshold;

amount of time since last access of the data being greater than a determined threshold;

the data being of a particular size;

the data being of a particular type;

the data having been generated by a particular client process; and

the data having been generated by a particular node of the DESS.

20. The one or more non-transitory machine-readable storage medium of claim 17, wherein, as part of the determination of the amount of congestion in the DESS comprises, the congestion management circuitry is operable to:

determine a load on a local resource of the DESS; and

determine a load on a remote resource of the DESS based on an indication of the load on the remote resource of the DESS received via a network link.

21. The one or more non-transitory machine-readable storage medium of claim 17, wherein the characteristics of the first tier of storage comprise an amount of unused space in the first tier of storage.

22. The one or more non-transitory machine-readable storage medium of claim 21, wherein, as part of the control of the migration, the data migration circuitry is operable to:

when the amount of unused space in the first tier of storage is below a first threshold, trigger the migration regardless of the amount of congestion of the DESS; and

when the amount of unused space in the first tier of storage is above the first threshold, delay the migration until the amount of congestion in the DESS is below the first threshold.

23. The one or more non-transitory machine-readable storage medium of claim 17, wherein the data migration circuitry is operable to determine whether to store data read from the second tier of storage in the first tier of storage based on the amount of congestion in the DESS.

24. The one or more non-transitory machine-readable storage medium of claim 23, wherein the data migration circuitry is configured to store the data read from the second tier of storage to the first tier of storage only when the amount congestion in the DESS is below a determined threshold.