Multi-chassis array with multi-level load balancing

• US 10,574,754 B1
• Filed: 01/31/2018
• Issued: 02/25/2020
• Est. Priority Date: 06/04/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Sajeepa, Prabhath, Talayco, Daniel, Yang, Qing, Lee, Robert
• Primary Examiner(s): Chery, Mardochee
• Application Number:
  US15/885,666
• Time in Patent Office: 755 Days
• Field of Search: 711103
• CPC Class Codes: G06F 11/1068 in sector programmable memories, e.g. flash disk G06F11/1072 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/2092 Techniques of failing over between control units
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/845 Systems in which the redundancy can be transformed in increased performance
• CPC Class Codes: G06F 2212/7206 Reconfiguration of flash memory system
• CPC Class Codes: G06F 2212/7207 management of metadata or control data
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 : Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0688 : Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 3/0689 : Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: G11C 29/52 : Protection of memory contents; Detection of errors in memory contents
• CPC Class Codes: H03M 13/154 : Error and erasure correction, e.g. by using the error and erasure locator or Forney polynomial
• CPC Class Codes: H04L 47/125 : by balancing the load, e.g. traffic engineering
• CPC Class Codes: H04L 49/10 : characterised by the switching fabric construction
• CPC Class Codes: H04L 49/111 : Switch interfaces, e.g. port details
• CPC Class Codes: H04L 49/15 : Interconnection of switching modules
• CPC Class Codes: H04L 49/356 : for storage area networks
• CPC Class Codes: H04L 67/1004 : Server selection for load balancing
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/51 : Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

A storage system is provided. The storage system includes a first storage cluster, the first storage cluster having a first plurality of storage nodes coupled together and a second storage cluster, the second storage cluster having a second plurality of storage nodes coupled together. The system includes an interconnect coupling the first storage cluster and the second storage cluster and a first pathway coupling the interconnect to each storage cluster. The system includes a second pathway, the second pathway coupling at least one fabric module within a chassis to each blade within the chassis.

Claims

1. A storage system, comprising:

a plurality of chassis, each of the plurality of chassis including a plurality of storage nodes;

an interconnect coupling the plurality of chassis together, the interconnect to perform a first load balancing operation to identify a chassis among the plurality of chassis to which data should be transmitted, wherein the chassis comprises;

a load balancer to perform a second load balancing operation to identify a storage node among the plurality of storage nodes of the chassis to which the data should be transmitted.

2. The storage system of claim 1, wherein each of the plurality of chassis includes a plurality of blades, in which one of the plurality of blades has a differing network bandwidth than another one of the plurality of blades.

3. The storage system of claim 1, wherein the first load balancing operation is based on a number of blades in each of the plurality of chassis.

4. The storage system of claim 1, wherein the first load balancing operation is based on an aggregate bandwidth of blades within each of the plurality of chassis.

5. The storage system of claim 1, wherein the second load balancing operation is based on a table accessed by a fabric module of the chassis, and wherein the fabric module is configurable to forward packets to a blade in a differing chassis.

6. The storage system of claim 1, wherein the interconnect is a pair of redundant switches.

7. The storage system of claim 1, wherein the storage system presents a single media access control address to external networks and wherein each of the plurality of chassis within the storage system is assigned a unique media access control address.

8. A method of load balancing for a storage system, comprising:

receiving an input/output request at a switch coupled to multiple chassis, each of the multiple chassis having at least one fabric module and multiple blades coupled to the at least one fabric module;

forwarding the I/O request from the switch to a fabric module of one of the multiple chassis based on a first load balancing mechanism; and

forwarding the I/O request from the fabric module of the one of the multiple chassis to one of the multiple blades based upon a second load balancing mechanism, the second load balancing mechanism differing from the first load balancing mechanism.

9. The method of claim 8, wherein the first load balancing mechanism is based on an amount of blades within each of the multiple chassis.

10. The method of claim 8, wherein the second load balancing mechanism is based on a table accessed by the at least one fabric module, and wherein the fabric module is configurable to forward data to a blade in a differing chassis.

11. The method of claim 8, wherein the multiple blades within one of the multiple chassis have differing network bandwidth rates.

12. The method of claim 8, wherein the first load balancing mechanism is based on aggregate bandwidth of blades within a chassis.

13. The method of claim 8, wherein the storage system presents a single media access control address to external networks and wherein each one of the multiple chassis within the storage system is assigned a unique media access control address.

14. The method of claim 8, wherein chassis with differing blade populations receive differing traffic loads.

15. A tangible, non-transitory, computer-readable media having instructions thereupon which, when executed by a processor, cause the processor to perform a method comprising:

receiving an input/output request at a switch coupled to multiple chassis, each of the multiple chassis having at least one fabric module and multiple blades coupled to the at least one fabric module;

forwarding the I/O request from the switch to a fabric module of one of the multiple chassis based on a first load balancing mechanism; and

forwarding the I/O request from the fabric module of the one of the multiple chassis to one of the multiple blades based upon a second load balancing mechanism, the second load balancing mechanism differing from the first load balancing mechanism.

16. The method of claim 15, wherein the first load balancing mechanism is based on an amount of blades within each of the multiple chassis.

17. The method of claim 15, wherein the second load balancing mechanism is based on a table accessed by the at least one fabric module, and wherein the fabric module is configurable to forward data to a blade in a differing chassis.

18. The method of claim 15, wherein the multiple blades within one of the multiple chassis have differing network bandwidth rates.

19. The method of claim 15, wherein the first load balancing mechanism is based on aggregate bandwidth of blades within a chassis.

20. The method of claim 15, wherein the storage system presents a single media access control address to external networks and wherein each one of the multiple chassis within the storage system is assigned a unique media access control address.