Techniques for maintaining operation of data storage system during a failure

• US 20050223284A1
• Filed: 03/25/2004
• Published: 10/06/2005
• Est. Priority Date: 03/25/2004
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Strickland, Stephen, Burroughs, John V., Dorr, Timothy
• Granted Patent: US 7,293,198 B2
• Time in Patent Office: Days
• US Class Current: 714/23
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality

Abstract

A data storage system has a first storage processor, a second storage processor, and a communications subsystem. The communications subsystem has (i) an interfacing portion interconnected between the first storage processor and the second storage processor, (ii) a clock circuit coupled to the interfacing portion, and (iii) a controller coupled to the interfacing portion and the clock circuit. The controller is configured to enable operation of the interfacing portion to provide communications between the first and second storage processors, sense a failure within the clock circuit, and reset the interfacing portion in response to the sensed failure to enable one of the first and second storage processors to continue operation. Such resetting of the interfacing portion prevents the remaining storage processor from locking up, thus freeing that storage processor so that it is capable of continuing to operate even after the failure.

Claims

1. In a data storage system having (i) a first storage processor, (ii) a second storage processor and (iii) a communications subsystem coupled to the first and second storage processors, a method for operating the data storage system during a failure within the communications subsystem, the method comprising:

while the first and second storage processors perform data storage operations, enabling operation of the communications subsystem to provide communications between the first and second storage processors;

sensing a failure within a critical portion of the communications subsystem; and

resetting an interfacing portion of the communications subsystem in response to the sensed failure to enable one of the first and second storage processors to continue operation.

2. The method of claim 1 wherein the critical portion of the communications subsystem includes clock circuitry, and wherein sensing the failure includes:

generating an error signal in response to loss of a clock signal from the clock circuitry within a predetermined timeout period.

3. The method of claim 2 wherein the communications subsystem includes a first interface device coupled to the first storage processor, and a second interface device coupled to the second storage processor, the first and second interface devices being connected together through a communications bus;

and wherein resetting the interfacing portion includes;

outputting a reset signal to the first interface device to enable the second storage processor to continue operation.

4. The method of claim 1 wherein the interfacing portion of the communications subsystem includes a first interface coupled to the first storage processor and a second interface coupled to the second storage processor;

and wherein the method further comprises;

opening a switch disposed between the first and second interfaces in response to the sensed failure.

5. The method of claim 4 wherein the critical portion of the communications subsystem includes (i) a first power supply input configured to receive a first power supply signal from a first power supply of the first storage processor, and (ii) a second power supply input configured to receive a second power supply signal from a second power supply of the second storage processor;

and wherein opening the switch includes;

breaking electrical pathways between the first and second interfaces in response to loss of one of the first and second power supply signals.

6. A data storage system, comprising:

a first storage processor;

a second storage processor; and

a communications subsystem having (i) an interfacing portion interconnected between the first storage processor and the second storage processor, (ii) a clock circuit coupled to the interfacing portion, and (iii) a controller coupled to the interfacing portion and the clock circuit, the controller being configured to;

enable operation of the interfacing portion to provide communications between the first and second storage processors;

sense a failure within the clock circuit; and

reset the interfacing portion in response to the sensed failure to enable one of the first and second storage processors to continue operation.

7. The data storage system of claim 6 wherein the controller of the communications subsystem includes:

a watchdog stage which is configured to generate an error signal in response to loss of a clock signal from the clock circuit within a predetermined timeout period.

8. The data storage system of claim 7 wherein the interfacing portion of the communications subsystem includes a first interface device coupled to the first storage processor, a second interface device coupled to the second storage processor, and a communications bus connecting the first and second interface devices together;

and wherein the controller of the communications subsystem further includes;

an output stage coupled to the watchdog stage, the output stage being configured to provide a reset signal to the first interface device in response to the error signal, the reset signal enabling the second storage processor to continue operation.

9. The data storage system of claim 6 wherein the interfacing portion of the communications subsystem includes a Cache Mirroring Interface (CMI) bus.

10. The data storage system of claim 9 wherein the interfacing portion of the communications subsystem further includes:

a first interface device having a first PCI interface coupled to the first storage processor and a first CMI interface coupled to the CMI bus, and

a second interface device having a second PCI interface coupled to the second storage processor and a second CMI interface coupled to the CMI bus.

11. The data storage system of claim 6 wherein the interfacing portion of the communications subsystem includes:

a first interface coupled to the first storage processor;

a second interface coupled to the second storage processor; and

a switch coupled to the controller of the communications subsystem, the switch being disposed between the first and second interface.

12. The data storage system of claim 11 wherein the first storage processor receives power from a first power supply, wherein the second storage processor receives power from a second power supply, and wherein the controller of the communications subsystem is further configured to:

open the switch in response to loss of a power supply signal from one of the first and second power supplies.

13. A data storage system, comprising:

a first storage processor;

a second storage processor; and

a communications subsystem having (i) an interfacing portion interconnected between the first storage processor and the second storage processor, (ii) a clock circuit coupled to the interfacing portion, and (iii) a controller coupled to the interfacing portion and the clock circuit, the controller including;

means for enabling operation of the interfacing portion to provide communications between the first and second storage processors;

means for sensing a failure within the clock circuit; and

means for resetting the interfacing portion in response to the sensed failure to enable one of the first and second storage processors to continue operation.

14. A communications subsystem for a data storage system having a first storage processor and a second storage processor, the communications subsystem comprising:

an interfacing portion configured to interconnect the first storage processor with the second storage processor;

a clock circuit coupled to the interfacing portion; and

a controller coupled to the interfacing portion and the clock circuit, the controller being configured to;

enable operation of the interfacing portion to provide communications between the first and second storage processors;

sense a failure within the clock circuit; and

reset the interfacing portion in response to the sensed failure to enable one of the first and second storage processors to continue operation.

15. The communications subsystem of claim 14 wherein the controller includes:

a watchdog stage which is configured to generate an error signal in response to loss of a clock signal from the clock circuit within a predetermined timeout period.

16. The communications subsystem of claim 15 wherein the interfacing portion includes a first interface device configured to couple to the first storage processor, a second interface device configured to couple to the second storage processor, and a communications bus connecting the first and second interface devices together;

and wherein the controller includes;

an output stage coupled to the watchdog stage, the output stage being configured to provide a reset signal to the first interface device in response to the error signal, the reset signal enabling the second storage processor to continue operation.

17. The communications subsystem of claim 14 wherein the interfacing portion includes a Cache Mirroring Interface (CMI) bus.

18. The communications subsystem of claim 17 wherein the interfacing portion further includes:

a first interface device having a first PCI interface configured to couple to the first storage processor and a first CMI interface configured to couple to the CMI bus, and

a second interface device having a second PCI interface configured to couple to the second storage processor and a second CMI interface configured to couple to the CMI bus.

19. The communications subsystem of claim 14 wherein the interfacing portion of the communications subsystem includes:

a first interface coupled to the first storage processor;

a second interface coupled to the second storage processor; and

a switch coupled to the controller, the switch being disposed between the first and second interface.

20. The communications subsystem of claim 19 wherein the first storage processor receives power from a first power supply, wherein the second storage processor receives power from a second power supply, and wherein the controller is further configured to:

open the switch in response to loss of a power supply signal from one of the first and second power supplies.