Automatic disk power-cycle

• US 8,839,026 B2
• Filed: 10/03/2011
• Issued: 09/16/2014
• Est. Priority Date: 10/03/2011
• Status: Active Grant

• Current Assignee: Infinidat Ltd.
• Original Assignee: Infinidat Ltd.
• Inventors: Kopylovitz, Haim
• Primary Examiner(s): BUTLER, SARAI E
• Application Number:
  US13/251,520
• Publication Number:
  US 20130086416A1
• Time in Patent Office: 1,079 Days
• Field of Search: 714/6.2, 714/6.1, 714/6.22, 714/2
• US Class Current: 714/6.2
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0793 Remedial or corrective actions recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40

Abstract

According to the presently disclosed subject matter there is provided, inter alia, a system and method which enable to identify, in a storage-system, malfunctioning disks, and in response, to activate a power-cycle process only for the specific failing disks, in order to bring these disks into proper operational mode. During the power-cycle process of a failing disk, other disks, which are not failing, remain operative and available.

Claims

1. A method of automatic disk power-cycle, for use in a storage system comprising one or more disk enclosures comprising a plurality of disks located in respective disk slots in said enclosures, said enclosures being adapted to be controlled by enclosure control commands, the method comprising:

receiving disk fault information indicative of errors occurring in a certain disk of the plurality of disks;

estimating, based on the disk fault information, whether or not the certain disk undergoes a media failure;

if it is estimated that the certain disk does not undergo the media failure, activating a power cycling for the certain disk, by sending to a disk enclosure, containing a disk slot corresponding to the certain disk, a first request to turn off the disk slot, and sending to the disk enclosure, a second request to turn on the disk slot; and

in response to a write command for writing modified data, directed to the certain disk while being power-cycled, writing the modified data to an alternative disk in the storage system, in accordance with a log-write technique.

2. The method according to claim 1 further comprising:

analyzing said disk fault information and estimating whether the certain disk is likely to be repaired by the power cycling, and activating the power cycling, if it is estimated that the power cycling is an effective measure for repairing the certain disk.

3. The method according to claim 1 further comprising delaying the sending of said second request to turn on the disk slot, for a predefined period of time from the time of sending said first request to turn off the disk slot.

4. The method according to claim 1 further comprising:

after sending to the disk slot said first request to turn off the disk slot, validating that the disk slot and the certain disk are turned off.

5. The method according to claim 1 further comprising:

after sending to the disk slot said second request to turn the certain disk on, validating that the disk slot and the certain disk are turned on.

6. The method according to claim 1 further comprising:

after turning the disk slot and the certain disk on, checking whether the certain disk is operating properly.

7. The method according to claim 6 further comprising:

in case where the certain disk is not operating properly, repeating activation of the power cycling for the certain disk.

8. The method according to claim 1 further comprising:

in response to a control command designated to a disk in said storage system, determining whether the disk is in a power-cycle mode; and

in case it is, delaying execution of the control command for a predefined period of time.

9. The method according to claim 1, wherein said storage system is configured with Redundant Array of Independent Disks (RAID) with parity, wherein the method further comprising:

responsive to a read command designated to a disk in said storage system and for receiving requested data,

determining whether the disk is in power-cycle mode; and

in case it is, use parity or mirroring for obtaining the requested data, thereby providing the requested data without delay.

10. The method according to claim 1, wherein said storage system is configured in accordance with a log-write technique, the method further comprising:

responsive to a write command, allocating one or more disks for writing data in said write command;

determining whether any one of said one or more disks is in power-cycle mode;

in case it is, allocating one or more alternative disks; and

writing data to one or more allocated disks.

11. The method according to claim 10, wherein said storage system is configured with Redundant Array of Independent Disks (RAID) with parity, further comprising:

calculating parity based on the received data and writing parity to a respective disk.

12. The method according to claim 1 wherein said storage system is configured to operate in accordance with a Serial Attached SCSI (SAS) and SAS Enclosure Service (SES).

13. The method according to claim 12 wherein said first request to turn off the disk slot is a first SES command that includes a device slot control element that contains a DEVICE OFF bit set to 1 in order to turn off said disk slot and wherein said second request to turn on the disk slot is a second SES command that includes a device slot control element that contains a DEVICE OFF bit set to 0 in order to turn on said disk slot.

14. The method of claim 1, wherein the disk fault information comprises locations of the errors within the certain disk;

wherein the method comprises estimating that the certain disk undergoes a media failure and avoiding activating power cycling, in cases where the locations of the errors are limited to a specific area on the certain disk.

15. The method of claim 1, wherein the disk fault information comprises at least one of:

locations of the errors within the certain disk and types of commands associated with the errors;

wherein the method comprises estimating that the certain disk does not undergo a media failure, in cases where the errors fulfil at least one condition of;

(i) the errors are associated with different types of commands; and

(ii) the locations of the errors are randomly spread over different locations on the certain disk.

16. A storage system comprising one or more disk enclosures controlled by one or more storage control devices, the one or more storage control devices constituting a storage control layer, said storage control layer operatively coupled to the one or more disk enclosures, said one or more disk enclosures comprising a plurality of disks located in respective disk slots in said enclosures;

said enclosures being adapted to be controlled by said control layer in accordance with enclosure control commands,

said storage control layer is configured to;

receive disk fault information indicative of errors occurring in a certain disk of the plurality of disks;

estimate, based on the disk fault information, whether or not the certain disk undergoes a media failure;

if it is estimated that the certain disk does not undergo a media failure, to activate power cycling for the certain disk, by sending to a disk enclosure containing a disk slot corresponding to the certain disk, a first request to turn off the disk slot and a second request to turn on the disk slot; and

in response to a write command for writing modified data, directed to the certain disk while being power-cycled, said control layer is configured to write the modified data to an alternative disk in the storage system, in accordance with a log-write technique.

17. The storage system according to claim 16, wherein said control layer is further configured to analyze the disk fault information being indicative of errors occurring in the certain disk, to estimate whether the certain disk is likely to be repaired by a disk power-cycle;

and to activate the power cycling, if it is estimated that the power cycling is an effective measure for repairing the certain disk.

18. The storage system according to claim 16, wherein said control layer is further configured to delay the sending of said second request to turn on the disk slot, for a predefined period of time, from the time of sending said first request to turn off the disk slot.

19. The storage system according to claim 16, wherein said control layer is further configured to validate that the disk slot and the certain disk are turned off, after the first request to turn off the disk slot is sent.

20. The storage system according to claim 16, wherein said control layer is further configured to validate that the disk slot and the certain disk are turned on, after the second request to turn the disk slot on is sent.

21. The storage system according to claim 16, wherein said control layer is further configured to validate that the certain disk is operating properly after the disk slot and the certain disk are turned on.

22. The storage system according to claim 16, wherein said control layer is further configured, responsive to a control command designated to a disk in said storage system, to determine whether the disk is in a power-cycle mode;

and in case it is, delay execution of the control command for a predefined period of time.

23. The storage system according to claim 16, wherein said storage system is configured with Redundant Array of Independent Disks (RAID) with parity and wherein said control layer is further configured, in response to a read command designated to a disk in said storage system for requested data, to determine whether the disk is in power-cycle mode;

and in case it is, use parity or mirroring for obtaining the requested data, thereby provide the requested data without delay.

24. The storage system according to claim 16 wherein said storage system is configured in accordance with a log-write technique and wherein said control layer is further configured, in response to a write command, to allocate one or more disks for writing data in said write command;

determine whether any one of said one or more disks is in power-cycle mode;

in case it is, allocate one or more alternative disks; and

write data to the one or more allocated disks.

25. The storage system according to claim 24, wherein said storage system is configured with Redundant Array of Independent Disks (RAID) with parity, the control layer is further configured to calculate parity based on the received data and writing parity to a respective disk.

26. The storage system according to claim 16, wherein said control layer is associated with a data-repository configured for storing data in respect of availability of said certain disk.

27. The storage system according to claim 16 wherein said storage system is configured to operate in accordance with a Serial Attached SCSI (SAS) and SAS Enclosure Service (SES).

28. The storage system according to claim 27 wherein said first request to turn off the disk slot is a first SES command that includes a device slot control element that contains a DEVICE OFF bit set to 1 in order to turn off said disk slot and wherein said second request to turn on the disk slot is a second SES command that includes a device slot control element contains a DEVICE OFF bit set to 0 in order to turn on said disk slot.

29. The storage system of claim 16, wherein the disk fault information comprises locations of the errors within the certain disk;

wherein the storage control layer is configured to estimate that the certain disk undergoes a media failure and to avoid activating power cycling, in cases where the locations of the errors are limited to a specific area on the certain disk.

30. The storage system of claim 16, wherein the disk fault information comprises at least one of:

locations of the errors within the certain disk and types of commands associated with the errors;

wherein the storage control layer is configured to estimate that the certain disk does not undergo a media failure, in cases where the errors fulfil at least one condition of;

(i) the errors are associated with different types of commands; and

(ii) the locations of the errors are randomly spread over different locations on the certain disk.

31. A disk performance monitor, operatively connectible to a storage system comprising one or more disk enclosures controlled by a plurality of storage control devices, the storage control devices constituting a storage control layer, said storage control layer operatively coupled to the one or more disk enclosures;

said one or more disk enclosures comprising a plurality of disks located in respective disk slots in said enclosures;

said enclosures being adapted to be controlled by said control layer in accordance with enclosure control commands,

said disk performance monitor is configured to;

receive disk fault information indicative of errors occurring in a certain disk of the plurality of disks;

estimate, based on the disk fault information, whether or not the certain disk undergoes a media failure;

if it is estimated that the certain disk does not undergo the media failure, to generate data indicative that said certain disk is temporarily unavailable;

to activate power cycling for the certain disk, by sending to a disk enclosure containing a disk slot corresponding to the certain disk, a first request to turn off the disk slot and a second request to turn on the disk slot; and

to generate data indicative that said certain disk is available; and

in response to a write command for writing modified data, directed to the certain disk while being power-cycled and therefore indicated as unavailable, to enable the control layer to write the modified data to an alternative disk that is indicated as available.

32. A program storage device readable by a machine, tangibly embodying a program of instructions that when executed by the machine causing the machine to perform method steps of a method of automatic disk power-cycle, for use in a storage system comprising one or more disk enclosures comprising a plurality of disks located in respective disk slots in said enclosures, said enclosures being adapted to be controlled in accordance with enclosure control commands, the method comprising:

receiving disk fault information indicative of errors occurring in a certain disk of the plurality of disks;

estimating, based on the disk fault information, whether or not the certain disk undergoes a media failure;

if it is estimated that the certain disk does not undergo a media failure, activating power cycling for the certain disk, by sending to a disk enclosure containing a disk slot corresponding to the certain disk, a first request to turn off the disk slot, and sending to the disk enclosure, a second request to turn on the disk slot; and

in response to a write command for writing modified data, directed to the certain disk while being power-cycled, writing the modified data to an alternative disk in the storage system.