Load balancing of I/O by moving logical unit (LUN) slices between non-volatile storage represented by different rotation groups of RAID (Redundant Array of Independent Disks) extent entries in a RAID extent table of a mapped RAID data storage system

• US 10,318,169 B2
• Filed: 10/27/2017
• Issued: 06/11/2019
• Est. Priority Date: 10/27/2017
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Dalmatov, Nickolay, Wahl, Michael, Gao, Jian
• Primary Examiner(s): Yi, David
• Assistant Examiner(s): Ahmed, Zubair
• Application Number:
  US15/795,655
• Publication Number:
  US 20190129614A1
• Time in Patent Office: 592 Days
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

Load balancing in a mapped RAID data storage system that moves slices of logical unit (LUN) address space between non-volatile storage represented by different rotation groups of RAID extent entries in a RAID extent table. The RAID extent table is divided into RAID extent groups corresponding to partnership groups of drives, and the RAID extent groups are divided into rotation groups. At least one LUN with an address space made up of slices is generated for each RAID extent group. Host data directed to each slice is stored in drive extents indicated by the RAID extent entries in the rotation group to which the slice is mapped. A rebalancing operation modifies the mapping between slices in a LUN and the rotation groups in the corresponding RAID extent group such that at least one slice is remapped from a heavily loaded rotation group to a lightly loaded rotation group.

Claims

1. A method of performing load balancing in a data storage system while providing mapped RAID (Redundant Array of Independent Disks) data protection, wherein the data storage system includes at least one storage processor and an array of data storage drives communicably coupled to the storage processor, the method comprising:

dividing the data storage drives in the array into a plurality of partnership groups of data storage drives, wherein each data storage drive in the array is contained in only one of the partnership groups;

generating a RAID extent table, wherein the RAID extent table contains a plurality of RAID extent entries, wherein each RAID extent entry contained in the RAID extent table indicates a plurality of drive extents, wherein each drive extent comprises a unique contiguous region of non-volatile data storage located on one of the data storage drives and each one of the data storage drives has multiple drive extents located thereon, wherein the RAID extent table contains a plurality of RAID extent groups, wherein each one of the RAID extent groups contains a plurality of RAID extent entries and corresponds to one of the partnership groups, and wherein the RAID extent entries in each RAID extent group only indicate drive extents that are located in data storage drives contained in the corresponding one of the partnership groups;

dividing each RAID extent group into a plurality of rotation groups, wherein each rotation group contains the same number of RAID extent entries, and wherein each RAID extent entry is contained in only one rotation group;

generating at least one corresponding logical unit (LUN) for each one of the RAID extent groups, wherein each LUN is made up of a plurality of slices, wherein host data directed to each slice is stored in drive extents indicated by RAID extent entries in a rotation group to which the slice is mapped according to a mapping between the slices in the LUN and the rotation groups in the corresponding RAID extent group; and

for each RAID extent group, performing a rebalancing operation that includes i) identifying, within the RAID extent group, a heavily loaded rotation group having a high I/O load and a lightly loaded rotation group having a low I/O load, and ii) modifying the mapping between the slices in the corresponding LUN and the rotation groups in the RAID extent group such that at least one slice of the corresponding LUN that is mapped to the heavily loaded rotation group is remapped to the lightly loaded rotation group.

2. The method of performing load balancing as in claim 1, wherein dividing each RAID extent group into a plurality of rotation groups further comprises dividing each RAID extent group into an integral number of rotation groups;

wherein the host data directed to each slice is stored completely in drive extents indicated by the RAID extent entries contained in the rotation group to which the slice is mapped according to the mapping between the slices in the LUN that contains the slice and the rotation groups contained in the RAID extent group corresponding to the LUN that contains the slice; and

wherein for each LUN the mapping between the slices in the LUN and the rotation groups contained in the corresponding RAID extent group maps multiple slices to each rotation group in the corresponding RAID extent group.

3. The method of performing load balancing as in claim 2, wherein dividing each RAID extent group into a plurality of rotation groups further comprises dividing each RAID extent group into a plurality of rotation groups such that no data storage drive that contains a drive extent that is indicated by any one of the RAID extent entries contained in a rotation group contains another drive extent that is indicated by any other RAID extent entry in the same rotation group.

4. The method of performing load balancing as in claim 3, wherein dividing each RAID extent group into a plurality of rotation groups further comprises dividing each RAID extent group into a plurality of rotation groups such that each one of the rotation groups contained within the same RAID extent group contains the same number of RAID extent entries.

5. The method of performing load balancing as in claim 4, wherein dividing each RAID extent group into a plurality of rotation groups further comprises dividing each RAID extent group into a plurality of rotation groups such that each rotation group contained within a RAID extent group contains a number of RAID extent entries that is equal to the total number of drives in the corresponding partnership group integer divided by the total number of drive extents indicated by each RAID extent entry.

6. The method of performing load balancing as in claim 4, wherein dividing each RAID extent group into a plurality of rotation groups further comprises dividing each RAID extent group into a plurality of rotation groups such that the set of drive extents indicated by the RAID extent entries in each rotation group includes one and only one drive extent allocated from each one of the data storage drives in the partnership group of data storage drives corresponding to the RAID extent group that contains the rotation group.

7. The method of performing load balancing as in claim 5, further comprising:

calculating an average I/O load for each rotation group, wherein the average I/O load for each rotation group is equal to a sum of the average numbers of I/O operations received per second for the slices mapped to that rotation group;

calculating an average rotation group I/O load for each RAID extent group that is equal to an average of the average I/O loads for the rotation groups contained within that RAID extent group;

wherein identifying the heavily loaded rotation group in the RAID extent group comprises identifying a rotation group having an average I/O load that exceeds the average rotation group I/O load for the RAID extent group by greater than a threshold percentage of the average rotation group I/O load for the RAID extent group; and

wherein identifying the lightly loaded rotation group in the RAID extent group comprises identifying a rotation group having an average I/O load that is less than the average rotation group I/O load for the RAID extent group by greater than the threshold percentage of the average rotation group I/O load for the RAID extent group.

8. The method of performing load balancing as in claim 7, wherein the at least one slice of the LUN that is mapped to the heavily loaded rotation group that is remapped to the lightly loaded rotation group comprises a slice of the LUN that is mapped to the heavily loaded rotation group that has a higher average number of I/O operations received per second than any other slice of the LUN that is mapped to the heavily loaded rotation group.

9. The method of performing load balancing as in claim 8, wherein the rebalancing operation further comprises modifying the mapping between the slices in the corresponding LUN and the rotation groups in the RAID extent group such that at least one slice of the LUN that is mapped to the lightly loaded rotation group is remapped to the heavily loaded rotation group.

10. The method of performing load balancing as in claim 9, wherein the at least one slice of the LUN that is mapped to the lightly loaded rotation group that is remapped to the heavily loaded rotation group comprises a slice of the LUN that is mapped to the lightly loaded rotation group that has a lower average number of I/O operations received per second than any other slice of the LUN that is mapped to the lightly loaded rotation group.

11. The method of performing load balancing as in claim 10, wherein generating the RAID extent table further comprises generating each RAID extent entry in the RAID extent table such that each RAID extent entry in the RAID extent table indicates the same number of drive extents.

12. The method of performing load balancing as in claim 11, wherein each RAID extent entry represents a RAID stripe and indicates i) a first set of drive extents that are used to persistently store host data directed to the slices that are mapped to the rotation group that contains the RAID extent entry, and ii) a second set of drive extents that are used to store parity information.

13. The method of performing load balancing as in claim 12, wherein the size of each LUN is a sum of the capacities of the drive extents that are indicated by the RAID extent entries in the corresponding RAID extent group that are used to persistently store host data directed to the slices contained in the LUN.

14. The method of performing load balancing as in claim 13, wherein dividing the data storage drives in the array into a plurality of partnership groups of data storage drives comprises dividing the data storage drives in the array into partnership groups such that each partnership group contains a number of data storage drives that is not greater than a predetermined maximum partnership group size.

15. A data storage system that provides load balancing while providing mapped RAID (Redundant Array of Independent Disks) data protection, comprising:

at least one storage processor including processing circuitry and a memory;

an array of data storage drives communicably coupled to the storage processor; and

wherein the memory has program code stored thereon for performing load balancing, wherein the program code, when executed by the processing circuitry, causes the processing circuitry to;

divide the data storage drives in the array into a plurality of partnership groups of data storage drives, wherein each data storage drive in the array is contained in only one of the partnership groups;

generate a RAID extent table, wherein the RAID extent table contains a plurality of RAID extent entries, wherein each RAID extent entry contained in the RAID extent table indicates a plurality of drive extents, wherein each drive extent comprises a unique contiguous region of non-volatile data storage located on one of the data storage drives and each one of the data storage drives has multiple drive extents located thereon, wherein the RAID extent table contains a plurality of RAID extent groups, wherein each one of the RAID extent groups contains a plurality of RAID extent entries and corresponds to one of the partnership groups, and wherein the RAID extent entries in each RAID extent group only indicate drive extents that are located in data storage drives contained in the corresponding one of the partnership groups;

divide each RAID extent group into a plurality of rotation groups, wherein each rotation group contains the same number of RAID extent entries, and wherein each RAID extent entry is contained in only one rotation group;

generate at least one corresponding logical unit (LUN) for each one of the RAID extent groups, wherein each LUN is made up of a plurality of slices, wherein host data directed to each slice is stored in drive extents indicated by RAID extent entries in a rotation group to which the slice is mapped according to a mapping between the slices in the LUN and the rotation groups in the corresponding RAID extent group; and

for each RAID extent group, perform a rebalancing operation that includes i) identifying, within the RAID extent group, a heavily loaded rotation group having a high I/O load and a lightly loaded rotation group having a low I/O load, and ii) modifying the mapping between the slices in the corresponding LUN and the rotation groups in the RAID extent group such that at least one slice of the corresponding LUN that is mapped to the heavily loaded rotation group is remapped to the lightly loaded rotation group.

16. The data storage system of claim 15, further comprising:

wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to;

divide each RAID extent group into a plurality of rotation groups at least in part by dividing each RAID extent group into an integral number of rotation groups,

store the host data directed to each slice completely in drive extents indicated by the RAID extent entries contained in the rotation group to which the slice is mapped according to the mapping between the slices in the LUN that contains the slice and the rotation groups contained in the RAID extent group corresponding to the LUN that contains the slice; and

wherein for each LUN the mapping between the slices in the LUN and the rotation groups contained in the corresponding RAID extent group maps multiple slices to each rotation group in the corresponding RAID extent group.

17. The data storage system of claim 16, wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to divide each RAID extent group into a plurality of rotation groups further at least in part by dividing each RAID extent group into a plurality of rotation groups such that no data storage drive that contains a drive extent that is indicated by any one of the RAID extent entries contained in a rotation group contains another drive extent that is indicated by any other RAID extent entry in the same rotation group.

18. The data storage system of claim 17, wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to divide each RAID extent group into a plurality of rotation groups further at least in part by dividing each RAID extent group into a plurality of rotation groups such that each one of the rotation groups contained within the same RAID extent group contains the same number of RAID extent entries.

19. The data storage system of claim 17, wherein the program code, when executed by the processing circuitry, further causes the processing circuitry to divide each RAID extent group into a plurality of rotation groups at least in part by dividing each RAID extent group into a plurality of rotation groups such that each rotation group contained within a RAID extent group contains a number of RAID extent entries that is equal to the total number of drives in the corresponding partnership group integer divided by the total number of drive extents indicated by each RAID extent entry.

20. A non-transitory computer readable medium for performing load balancing in a data storage system while providing mapped RAID (Redundant Array of Independent Disks) data protection, wherein the data storage system includes at least one storage processor and an array of data storage drives communicably coupled to the storage processor, the non-transitory computer readable medium comprising instructions stored thereon that when executed on processing circuitry in the storage processor perform the steps of:

dividing the data storage drives in the array into a plurality of partnership groups of data storage drives, wherein each data storage drive in the array is contained in only one of the partnership groups;

generating a RAID extent table, wherein the RAID extent table contains a plurality of RAID extent entries, wherein each RAID extent entry contained in the RAID extent table indicates a plurality of drive extents, wherein each drive extent comprises a unique contiguous region of non-volatile data storage located on one of the data storage drives and each one of the data storage drives has multiple drive extents located thereon, wherein the RAID extent table contains a plurality of RAID extent groups, wherein each one of the RAID extent groups contains a plurality of RAID extent entries and corresponds to one of the partnership groups, and wherein the RAID extent entries in each RAID extent group only indicate drive extents that are located in data storage drives contained in the corresponding one of the partnership groups;

dividing each RAID extent group into a plurality of rotation groups, wherein each rotation group contains the same number of RAID extent entries, and wherein each RAID extent entry is contained in only one rotation group;

generating at least one corresponding logical unit (LUN) for each one of the RAID extent groups, wherein each LUN is made up of a plurality of slices, wherein host data directed to each slice is stored in drive extents indicated by RAID extent entries in a rotation group to which the slice is mapped according to a mapping between the slices in the LUN and the rotation groups in the corresponding RAID extent group; and

for each RAID extent group, performing a rebalancing operation that includes i) identifying, within the RAID extent group, a heavily loaded rotation group having a high I/O load and a lightly loaded rotation group having a low I/O load, and ii) modifying the mapping between the slices in the corresponding LUN and the rotation groups in the RAID extent group such that at least one slice of the corresponding LUN that is mapped to the heavily loaded rotation group is remapped to the lightly loaded rotation group.