System and method for sharing SATA drives in active-active RAID controller system

• US 7,536,508 B2
• Filed: 11/07/2006
• Issued: 05/19/2009
• Est. Priority Date: 06/30/2006
• Status: Expired due to Fees

• Current Assignee: Dot Hill Systems Corporation (Seagate Technology Holdings Plc)
• Original Assignee: Dot Hill Systems Corporation (Seagate Technology Holdings Plc)
• Inventors: Davies, Ian Robert
• Primary Examiner(s): Elmore; Stephen C
• Application Number:
  US11/557,178
• Publication Number:
  US 20080005470A1
• Time in Patent Office: 924 Days
• Field of Search: 711/114, 711/156, 711/170, 714/6, 714/7
• US Class Current: 711/114
• CPC Class Codes: G06F 11/2092 Techniques of failing over between control units
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD

Abstract

An active-active RAID system includes first and second active-active RAID controllers which efficiently share access to SATA drives. SAS expanders connect the RAID controllers to the drives. The controllers establish an affiliation within the SAS expanders with respectively-owned first and second subsets of the SATA drives. The controllers directly transmit to the SAS expanders commands destined for affiliated drives, but forward to the other RAID controller, via an inter-controller communications link, commands destined for unaffiliated drives for transmission by the other RAID controller. The controllers handle drive ownership changes by clearing previously-established affiliations, updating ownership data stored on the drives, including forwarding the update commands as necessary, and re-establishing affiliations based on the new ownership. In response to a SAS configuration change, the controllers clear all affiliations, and employ a distributed lock mechanism to ensure exclusive access to perform the SAS discover process and read ownership data.

Claims

1. An active-active redundant array of inexpensive disks (RAID) system for efficiently sharing Serial Advanced Technology Attachment (SATA) drives, comprising:

first and second RAID controllers, each configured to couple to a plurality of SATA drives via one or more Serial Attached SCSI (SAS) expanders, wherein said first RAID controller is configured to establish an affiliation in said one or more SAS expanders between said first RAID controller and each of a first subset of said plurality of SATA drives, and said second RAID controller is configured to establish an affiliation in said one or more SAS expanders between said second RAID controller and each of a second subset of said plurality of SATA drives, wherein said first and second subsets of said plurality of SATA drives are mutually exclusive; and

a communications link, for coupling said first and second RAID controllers to facilitate communications therebetween;

wherein said first RAID controller is configured to transmit commands destined for said first subset of said plurality of SATA drives to said one or more SAS expanders, and is configured to forward commands destined for said second subset of said plurality of SATA drives on said communications link to said second RAID controller for said second RAID controller to responsively transmit to said one or more SAS expanders;

wherein said second RAID controller is configured to transmit commands destined for said second subset of said plurality of SATA drives to said one or more SAS expanders, and is configured to forward commands destined for said first subset of said plurality of SATA drives on said communications link to said first RAID controller for said first RAID controller to responsively transmit to said one or more SAS expanders.

2. The active-active RAID system of claim 1, wherein said first RAID controller is further configured to detect that said second RAID controller has failed and to responsively clear said established affiliations in said one or more SAS expanders between said second RAID controller and said second subset of said plurality of SATA drives.

3. The active-active RAID system of claim 2, wherein said first RAID controller is further configured to transmit said commands destined for said second subset of said plurality of SATA drives to said one or more SAS expanders, rather than forwarding said commands on said communications link to said second RAID controller, after said second RAID controller has failed.

4. The active-active RAID system of claim 2, wherein said first RAID controller is further configured to clear said established affiliations by resetting each SAS Physical Layer Device (PHY) connected to said second subset of said plurality of SATA drives.

5. The active-active RAID system of claim 1, wherein said first RAID controller is further configured to transmit said commands destined for said second subset of said plurality of SATA drives to said one or more SAS expanders, rather than forwarding said commands on said communications link to said second RAID controller, if said communications link has failed.

6. The active-active RAID system of claim 1, wherein said first RAID controller is further configured to transmit said commands destined for said second subset of said plurality of SATA drives to said one or more SAS expanders, rather than forwarding said commands on said communications link to said second RAID controller, if said first RAID controller is performing a SAS discover process to discover said plurality of SATA drives.

7. The active-active RAID system of claim 1, wherein said communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link.

8. The active-active RAID system of claim 1, further comprising:

a plurality of SAS multiplexers, connected to corresponding ones of said plurality of SATA drives, configured to enable first and second SAS expanders of said one or more SAS expanders to communicate with said plurality of SATA drives.

9. The active-active RAID system of claim 1, wherein said one or more SAS expanders comprises first and second SAS expanders, the active-active RAID system further comprising:

a SAS link connecting said first and second SAS expanders;

first SAS pathways, inclusive of said SAS link, connecting said first and second RAID controllers to said plurality of SATA drives;

second SAS pathways, exclusive of said SAS link, connecting said first and second RAID controllers to said plurality of SATA drives.

10. The active-active RAID system of claim 9, wherein said first and second RAID controllers are configured to establish an affiliation with each of a third subset of said plurality of SATA drives only in ones of said one or more SAS expanders along said first SAS pathways, and to refrain from establishing an affiliation with each of said third subset of said plurality of SATA drives in ones of said one or more SAS expanders along said second SAS pathways, wherein said first, second, and third subsets of said plurality of SATA drives are mutually exclusive.

11. The active-active RAID system of claim 10, wherein said first and second subset of said plurality of SATA drives are owned by said first and second RAID controllers, respectively, whereas said third subset of said plurality of SATA drives are owned by neither of said first and second RAID controllers.

12. A method for efficiently sharing Serial Advanced Technology Attachment (SATA) drives in an active-active redundant array of inexpensive disks (RAID) system having first and second RAID controllers coupled to a plurality of SATA drives via one or more SAS expanders and a communications link between the first and second RAID controllers, the method comprising:

establishing an affiliation in the one or more SAS expanders between the first RAID controller and each of a first subset of the plurality of SATA drives;

establishing an affiliation in the one or more SAS expanders between the second RAID controller and each of a second subset of the plurality of SATA drives, wherein the first and second subsets of the plurality of SATA drives are mutually exclusive;

transmitting, by the first RAID controller, commands destined for the first subset of said plurality of SATA drives to the one or more SAS expanders, and forwarding, by the first RAID controller, commands destined for the second subset of said plurality of SATA drives on the communications link to the second RAID controller for the second RAID controller to responsively transmit to the one or more SAS expanders; and

transmitting, by the second RAID controller, commands destined for the second subset of said plurality of SATA drives to the one or more SAS expanders, and forwarding, by the second RAID controller, commands destined for the first subset of said plurality of SATA drives on the communications link to the first RAID controller for the first RAID controller to responsively transmit to the one or more SAS expanders.

13. The method of claim 12, further comprising:

detecting that the second RAID controller has failed; and

clearing, by the first RAID controller, the established affiliations in the one or more SAS expanders between the second RAID controller and the second subset of the plurality of SATA drives, in response to said detecting.

14. The method of claim 13, further comprising:

transmitting, by the first RAID controller, the commands destined for the second subset of said plurality of SATA drives to the one or more SAS expanders, rather than forwarding the commands on the communications link to the second RAID controller, after the second RAID controller has failed.

15. The method of claim 13, wherein the first RAID controller said clearing the established affiliations comprises the first RAID controller resetting each SAS Physical Layer Device (PHY) connected to the second subset of the plurality of SATA drives.

16. The method of claim 12, further comprising:

detecting that the communications link has failed; and

transmitting, by the first RAID controller, the commands destined for the second subset of said plurality of SATA drives to the one or more SAS expanders, rather than forwarding the commands on the communications link to the second RAID controller, in response to said detecting.

17. The method of claim 12, further comprising:

transmitting, by the first RAID controller, the commands destined for the second subset of said plurality of SATA drives to the one or more SAS expanders, rather than forwarding the commands on the communications link to the second RAID controller, if the first RAID controller is performing a SAS discover process to discover the plurality of SATA drives.

18. The method of claim 12, wherein the communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link.

19. The method of claim 12, wherein the one or more SAS expanders comprises first and second SAS expanders, and the active-active RAID system includes a SAS link connecting the first and second SAS expanders, first SAS pathways, inclusive of the SAS link, connecting the first and second RAID controllers to the plurality of SATA drives, and second SAS pathways, exclusive of the SAS link, connecting the first and second RAID controllers to the plurality of SATA drives, the method further comprising:

establishing, by the first and second RAID controllers, an affiliation with each of a third subset of the plurality of SATA drives only in ones of the one or more SAS expanders along the first SAS pathways; and

refraining, by the first and second RAID controllers, from establishing an affiliation with each of the third subset of the plurality of SATA drives in ones of the one or more SAS expanders along the second SAS pathways, wherein the first, second, and third subsets of the plurality of SATA drives are mutually exclusive.

20. The method of claim 19, wherein the first and second subset of the plurality of SATA drives are owned by the first and second RAID controllers, respectively, whereas the third subset of the plurality of SATA drives are owned by neither of the first and second RAID controllers.

21. An active-active redundant array of inexpensive disks (RAID) system for changing an ownership state stored on a shared Serial Advanced Technology Attachment (SATA) drive, comprising:

first and second RAID controllers;

a communications link, connecting said first and second RAID controllers; and

at least one Serial Attached SCSI (SAS) expander, configured to connect said first and second RAID controllers to the SATA drive;

wherein said first RAID controller is configured to clear at least one affiliation previously established in said at least one SAS expander between said first RAID controller and the SATA drive;

wherein said first RAID controller is configured to forward a command to write predetermined data to the SATA drive from said first RAID controller to said second RAID controller via said communications link, after clearing said affiliation, wherein said predetermined data indicates the ownership state of the SATA drive;

wherein said second RAID controller is configured to transmit said forwarded command to said at least one SAS expander for subsequent transmission by said at least one SAS expander to the SATA drive.

22. The active-active RAID system of claim 21, wherein the first RAID controller is further configured to forward to said second RAID controller via said communications link all commands destined for the SATA drive subsequent to said command to write said predetermined data to the SATA drive.

23. The active-active RAID system of claim 21, wherein the changing of the ownership state is from ownership by said first RAID controller to ownership by said second RAID controller, wherein said predetermined data indicates the SATA drive is owned by said second RAID controller rather than said first RAID controller.

24. The active-active RAID system of claim 23, wherein said first RAID controller is configured to clear said affiliation and forward said command in response to detecting initiation of a failback operation from said first RAID controller to said second RAID controller.

25. The active-active RAID system of claim 21, wherein the changing of the ownership state is from a free ownership state to ownership by said first RAID controller, wherein said predetermined data indicates the SATA drive is owned by said first RAID controller.

26. The active-active RAID system of claim 21, wherein said first RAID controller clears said at least one affiliation by transmitting a SAS clear affiliation command to said at least one SAS expander, rather than transmitting a SAS Physical Layer Device (PHY) reset command.

27. The active-active RAID system of claim 21, wherein said communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link coupling said first and second RAID controllers.

28. A method for changing an ownership state stored on a Serial Advanced Technology Attachment (SATA) drive within an active-active redundant array of inexpensive disks (RAID) system having first and second RAID controllers and at least one Serial Attached SCSI (SAS) expander connecting the first and second RAID controllers to the SATA drive, the method comprising:

clearing, by the first RAID controller, at least one affiliation previously established in the at least one SAS expander between the first RAID controller and the SATA drive;

forwarding, from the first RAID controller to the second RAID controller via a communications link, a command to write predetermined data to the SATA drive, after said clearing, wherein said predetermined data indicates the ownership state of the SATA drive; and

transmitting, by the second RAID controller, the command to the at least one SAS expander for subsequent transmission by the at least one SAS expander to the SATA drive, in response to said forwarding.

29. The method of claim 28, further comprising:

forwarding, from the first RAID controller to the second RAID controller via the communications link, all commands destined for the SATA drive subsequent to the command to write the predetermined data to the SATA drive.

30. The method of claim 28, wherein the changing of the ownership state is from ownership by the first RAID controller to ownership by the second RAID controller, wherein said predetermined data indicates the SATA drive is owned by the second RAID controller rather than the first RAID controller.

31. The method of claim 30, further comprising:

detecting initiation of a failback operation from the first RAID controller to the second RAID controller, prior to said clearing;

wherein said clearing and said forwarding are performed in response to said detecting.

32. The method of claim 28, wherein the changing of the ownership state is from a free ownership state to ownership by the first RAID controller, wherein said predetermined data indicates the SATA drive is owned by the first RAID controller.

33. The method of claim 28, wherein said clearing said at least one affiliation comprises transmitting a SAS clear affiliation command to said at least one SAS expander, rather than transmitting a SAS Physical Layer Device (PHY) reset command.

34. The method of claim 28, wherein said communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link coupling the first and second RAID controllers.

35. An active-active redundant array of inexpensive disks (RAID) system for changing an ownership state stored on a shared Serial Advanced Technology Attachment (SATA) drive, comprising:

first and second RAID controllers; and

first and second Serial Attached SCSI (SAS) expanders, configured to connect respective said first and second RAID controllers to the SATA drive;

wherein the first RAID controller is configured to clear an affiliation previously established in the second SAS expander between the first RAID controller and the SATA drive;

wherein the first RAID controller is configured to transmit to the first SAS expander a command to write predetermined data to the SATA drive for subsequent transmission by the first SAS expander to the SATA drive, after clearing the affiliation, wherein said predetermined data indicates a free ownership state of the SATA drive;

wherein the first RAID controller is configured to notify the second RAID controller that the ownership state of the SATA drive has changed, after transmitting the command.

36. The active-active RAID system of claim 35, wherein the second RAID controller is configured to read the predetermined data from the SATA drive in response to the notification.

37. The active-active RAID system of claim 35, wherein the first RAID controller notifies the second RAID controller via a communications link coupling the first and second RAID controllers.

38. The active-active RAID system of claim 37, wherein said communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link.

39. The active-active RAID system of claim 37, wherein the first RAID controller is configured to issue first subsequent commands to the SATA drive only (1) via a first SAS path to the SATA drive exclusive of the second SAS expander and (2) by forwarding the first subsequent commands to the second RAID controller via the communications link for subsequent transmission by the second RAID controller via a second SAS path to the SATA drive exclusive of the first SAS expander, whereby the first RAID controller avoids establishing an affiliation within the second SAS expander between the first RAID controller and the SATA drive, and wherein the second RAID controller is configured to issue second subsequent commands to the SATA drive only (1) via the second SAS path and (2) by forwarding the second subsequent commands to the first RAID controller via the communications link for subsequent transmission by the first RAID controller via the first SAS path to the SATA drive, whereby the second RAID controller avoids establishing an affiliation within the first SAS expander between the second RAID controller and the SATA drive.

40. The active-active RAID system of claim 35, wherein said first RAID controller is configured to clear said affiliation and transmit said command in response to receiving a request to delete a drive array of which the SATA drive was previously a constituent drive.

41. The active-active RAID system of claim 35, wherein said first RAID controller clears the affiliation by transmitting a SAS clear affiliation command to the second SAS expander, rather than transmitting a SAS Physical Layer Device (PHY) reset command.

42. A method for changing an ownership state stored on a Serial Advanced Technology Attachment (SATA) drive within an active-active redundant array of inexpensive disks (RAID) system having first and second RAID controllers and respective first and second Serial Attached SCSI (SAS) expanders connecting the respective first and second RAID controllers to the SATA drive, the method comprising:

clearing, by the first RAID controller, an affiliation previously established in the second SAS expander between the first RAID controller and the SATA drive;

transmitting to the first SAS expander, by the first RAID controller, a command to write predetermined data to the SATA drive for subsequent transmission by the first SAS expander to the SATA drive, after said clearing, wherein said predetermined data indicates a free ownership state of the SATA drive; and

notifying the second RAID controller, by the first RAID controller, that the ownership state of the SATA drive has changed, after said transmitting.

43. The method of claim 42, further comprising:

reading from the SATA drive, by the second RAID controller, the predetermined data, in response to said notifying.

44. The method of claim 42, wherein said notifying comprises the first RAID controller notifying the second RAID controller via a communications link coupling the first and second RAID controllers.

45. The method of claim 44, wherein said communications link comprises a Peripheral Component Interconnect (PCI) PCI-Express link.

46. The method of claim 44, further comprising:

issuing first subsequent commands to the SATA drive, by the first RAID controller, only (1) via a first SAS path to the SATA drive exclusive of the second SAS expander and (2) by forwarding the first subsequent commands to the second RAID controller via the communications link for subsequent transmission by the second RAID controller via a second SAS path to the SATA drive exclusive of the first SAS expander, whereby the first RAID controller avoids establishing an affiliation within the second SAS expander between the first RAID controller and the SATA drive; and

issuing second subsequent commands to the SATA drive, by the second RAID controller, only (1) via the second SAS path and (2) by forwarding the second subsequent commands to the first RAID controller via the communications link for subsequent transmission by the first RAID controller via the first SAS path to the SATA drive, whereby the second RAID controller avoids establishing an affiliation within the first SAS expander between the second RAID controller and the SATA drive.

47. The method of claim 42, further comprising:

receiving a request to delete a drive array of which the SATA drive was previously a constituent drive, prior to said clearing;

wherein said clearing and said transmitting are performed in response to said receiving the request.

48. The method of claim 42, wherein said clearing the affiliation comprises transmitting a SAS clear affiliation command to the second SAS expander, rather than transmitting a SAS Physical Layer Device (PHY) reset command.

49. An active-active redundant array of inexpensive disks (RAID) system for sharing Serial Advanced Technology Attachment (SATA) drives, comprising:

first and second RAID controllers; and

at least one Serial Attached SCSI (SAS) expander, configured to connect said first and second RAID controllers to said SATA drives;

wherein said first RAID controller is configured to obtain a lock that excludes said second RAID controller from accessing said SATA drives in response to receiving from said at least one SAS expander a SAS configuration change event broadcast primitive;

wherein said first RAID controller is configured to send a command to said second RAID controller to clear all affiliations established in said at least one SAS expander between said second RAID controller and said SATA drives after obtaining said lock; and

wherein said first RAID controller is configured to read ownership information stored on said SATA drives while said second RAID controller is excluded from accessing said SATA drives after sending said command.

50. The active-active RAID system of claim 49, wherein said first RAID controller is configured to establish within said at least one SAS expander an affiliation between said first RAID controller and each of said SATA drives owned by said first RAID controller.

51. The active-active RAID system of claim 49, further comprising:

a communications link connecting said first and second RAID controllers;

wherein said first RAID controller is configured to forward via said communications link to said second RAID controller commands destined for each of said SATA drives owned by said second RAID controller, for subsequent transmission by said second RAID controller to said SATA drives owned by said second RAID controller.

52. The active-active RAID system of claim 51, wherein said first RAID controller is configured to refrain from establishing within said at least one SAS expander an affiliation between said first RAID controller and each of said SATA drives owned by said second RAID controller.

53. The active-active RAID system of claim 49, wherein said command to clear all affiliations further commands said second RAID controller to temporarily cease transmitting commands to said SATA drives.

54. The active-active RAID system of claim 49, further comprising:

first and second respective SAS pathways from said first and second RAID controllers to each of said SATA drives, said first and second respective SAS pathways comprising said at least one SAS expander;

wherein said first RAID controller is configured to establish an affiliation within said at least one SAS expander along said first SAS pathway to each of said SATA drives not currently owned by either of said first and second RAID controllers; and

wherein said second RAID controller is configured to establish an affiliation within said at least one SAS expander along said second SAS pathway to each of said SATA drives not currently owned by either of said first and second RAID controllers.

55. A method for sharing Serial Advanced Technology Attachment (SATA) drives by first and second redundant array of inexpensive disks (RAID) controllers within an active-active RAID system having at least one Serial Attached SCSI (SAS) expander connecting the first and second RAID controllers to the SATA drives, the method comprising:

obtaining, by the first RAID controller, a lock that excludes the second RAID controller from accessing the SATA drives, in response to receiving from the at least one SAS expander a SAS configuration change event broadcast primitive;

sending, by the first RAID controller, a command to the second RAID controller to clear all affiliations established in the at least one SAS expander between the second RAID controller and the SATA drives, after said obtaining the lock; and

reading, by the first RAID controller, ownership information stored on the SATA drives, while the second RAID controller is excluded from accessing the SATA drives, after said sending the command.

56. The method of claim 55, further comprising:

establishing within said at least one SAS expander, by the first RAID controller, an affiliation between the first RAID controller and each of the SATA drives owned by the first RAID controller, based on said reading the ownership information.

57. The method of claim 55, further comprising:

forwarding, by the first RAID controller, to the second RAID controller commands destined for each of the SATA drives owned by the second RAID controller, based on said reading the ownership information, for subsequent transmission by the second RAID controller to the SATA drives owned by the second RAID controller, wherein said forwarding is performed via a communications link connecting the first and second RAID controllers.

58. The method of claim 57, further comprising:

refraining from establishing within said at least one SAS expander, by the first RAID controller, an affiliation between the first RAID controller and each of the SATA drives owned by the second RAID controller, based on said reading the ownership information.

59. The method of claim 55, wherein said command to clear all affiliations further commands the second RAID controller to temporarily cease transmitting commands to the SATA drives.

60. The method of claim 55, wherein first and second respective SAS pathways connect the first and second RAID controllers to each of the SATA drives, the first and second respective SAS pathways comprising the at least one SAS expander, the method further comprising:

establishing, by the first RAID controller, an affiliation within said at least one SAS expander along the first SAS pathways to each of the SATA drives not currently owned by either of the first and second RAID controllers, based on said reading the ownership information; and

establishing, by the second RAID controller, an affiliation within said at least one SAS expander along the second SAS pathways to each of the SATA drives not currently owned by either of the first and second RAID controllers, based on said reading the ownership information.