Fault-tolerant computer with three independently clocked processors asynchronously executing identical code that are synchronized upon each voted access to two memory modules

• US 5,193,175 A
• Filed: 03/06/1991
• Issued: 03/09/1993
• Est. Priority Date: 12/09/1988
• Status: Expired due to Term

• Current Assignee: Tandem Computers Incorporated (HP Inc.)
• Original Assignee: Tandem Computers Incorporated (HP Inc.)
• Inventors: Allison, John D., Horst, Robert W., Jewett, Douglas E., Cutts, Richard W. Jr., Mehta, Nikhil A., DeBacker, Kenneth C., Norwood, Peter C.
• Primary Examiner(s): Lee, Thomas C.
• Assistant Examiner(s): ELLIS, RICHARD L
• Application Number:
  US07/666,495
• Time in Patent Office: 734 Days
• Field of Search: 364/200 MS File, 364/900 MS File, 371/9.1, 371/11.1, 371/67.1, 371/68.3, 371/36, 395/575, 395/550, 395/425, 395/800
• US Class Current: 714/11
• CPC Class Codes: G06F 11/0724 in a multiprocessor or a multi-core unit multiprocessors per se G06F15/80
• CPC Class Codes: G06F 11/0757 by exceeding a time limit, i.e. time-out, e.g. watchdogs
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1405 at machine instruction level
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1641 where the comparison is not performed by the redundant processing components
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/1679 at clock signal level
• CPC Class Codes: G06F 11/1683 at instruction level
• CPC Class Codes: G06F 11/1687 at event level, e.g. by interrupt or result of polling
• CPC Class Codes: G06F 11/1691 using a quantum
• CPC Class Codes: G06F 11/18 using passive fault-masking of the redundant circuits error detection by comparing the output of redundant processing systems with continued operation after detection of the error G06F11/165
• CPC Class Codes: G06F 11/181 Eliminating the failing redundant component
• CPC Class Codes: G06F 11/182 based on mutual exchange of the output between redundant processing components
• CPC Class Codes: G06F 11/184 where the redundant components implement processing functionality
• CPC Class Codes: G06F 11/185 and the voting is itself performed redundantly
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 11/2015 Redundant power supplies power supply failure G06F1/30
• CPC Class Codes: G06F 11/2017 where memory access, memory control or I/O control functionality is redundant redundant communication control functionality G06F11/2005; redundant storage control functionality G06F11/2089
• CPC Class Codes: G06F 2201/85 : Active fault masking without idle spares
• CPC Class Codes: G06F 9/3824 : Operand accessing
• CPC Class Codes: G06F 9/3851 : from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/3863 : using multiple copies of the architectural state, e.g. shadow registers
• CPC Class Codes: G06F 9/3885 : using a plurality of independent parallel functional units
• CPC Class Codes: G11C 29/74 : using duplex memories, i.e. using dual copies

Abstract

A computer system in a fault-tolerant configuration employs three identical CPUs executing the same instruction stream, with two identical, self-checking memory modules storing duplicates of the same data. Memory references by the three CPUs are made by three separate busses connected to three separate ports of each of the two memory modules. The three CPUs are loosely synchronized, as by detecting events such as memory references and stalling any CPU ahead of others until all execute the function simultaneously; interrupts can be synchronized by ensuring that all three CPUs implement the interrupt at the same point in their instruction stream. Memory references via the separate CPU-to-memory busses are voted at the three separate ports of each of the memory modules. I/O functions are implemented using two identical I/O busses, each of which is separately coupled to only one of the memory modules. A number of I/O processors are coupled to both I/O busses.

Claims

1. A fault-tolerant computer system, comprising:

first, second and third CPUs, each executing a same given instruction stream;

first and second memory modules each storing the same data in response to execution by said CPUs of said instruction stream;

busses coupling each of the first, second and third CPUs individually to each of said first and second memory modules for said first, second and third CPUs to access said first and second memory modules separately or in duplicate, said first, second and third CPUs each executing said same given instruction stream, wherein each said CPU execution is clocked independently such that at least one earlier CPU among said CPUs asynchronously accesses said first and second memory modules after said earlier CPU executes plural consecutive instructions asynchronously and prior to a last CPU among said CPUs asynchronously accessing said first and second memory modules, and said earlier and last CPUs accessing to said memory modules being detected by said memory modules;

synchronizing means for synchronizing said first, second and third CPUs, said synchronizing means enabling said first and second memory modules to detect said asynchronous access by said earlier and last CPUs to said first and second memory modules, said synchronizing means stalling any CPUs for which the access occurs earlier to wait until the last one of said CPUs executes said access, and said synchronizing means thereafter allowing the access to occur asynchronously;

a first input/output bus coupled to said first memory module and a second input/output bus coupled to said second memory modules; and

a first input/output processor coupled to both said first and second input/output busses, and a second input/output processor coupled to both said first and second input/output busses.

2. A system according to claim 1 wherein said first, second and third CPUs operate asynchronously on independent clocks during execution of the given instruction stream.

3. A system according to claim 1 wherein one of said first and second memory modules is designated as a primary memory and the other of said first and second memory modules is designated as back-up, and wherein write operations by the CPUs are performed in both of said first and second memory modules but during read operations said CPUs receive data from only the primary memory module.

4. A system according to claim 3 further comprising means for designating the back-up memory module as the primary memory and designating the primary memory as the back-up memory module, at any time.

5. A system according to claim 1 including at least one additional input/output processor connected to both said first and second input/output busses.

6. A system according to claim 1 wherein said first and second memory modules include means for voting said memory accesses.

7. A system according to claim 6 wherein data is voted only for memory writes in said means for voting, and addresses and commands are voted for both memory reads and memory writes.

8. A system according to claim 6 wherein each of said first and second memory modules includes a random access memory and said random access memory is accessed in parallel with operation of said means for voting.

9. A system according to claim 1 wherein upon failure of one of said first, second and third CPUs, said one failed CPU is placed off-line and the remaining two CPUs continue to execute said instruction stream.

10. A method of operating a computer system having first, second and third CPUs and first and second memory modules, the method comprising the steps of:

asynchronously clocking on individual clocks each of the first, second and third CPUs;

in parallel with the asynchronous clocking step, executing an instruction stream in each of the first, second and third CPUs, said instruction stream being the same in each of said first, second and third CPUs;

in parallel with or after the instruction stream executing step, storing identical data in the first and second memory modules;

performing asynchronous memory references by each of the first, second and third CPUs individually to each of said first and second memory modules after each of said CPUs has executed plural consecutive instructions asynchronously, wherein said step of performing of asynchronous memory references includes the step of voting said asynchronous memory references to both said first and second memory modules from said first, second and third CPUs such that a voted asynchronous memory reference is used to access asynchronously said first and second memory modules; and

performing store or recall operations in said first and second memory modules in reponse to said memory references when said memory references have been received from all of said first, second and third CPUs.

11. A method according to claim 10 for operating a computer system having first and second input/output busses and first and second input/output processors, the method further including the steps of:

before the instruction stream executing step, coupling the first input/output bus to the first memory module and the second input/output bus to the second memory module;

performing input/output functions to the first input/output bus coupled to said first memory module, and performing input/output functions to the second input/output bus coupled separately from said first memory module to said second memory modules;

coupling the first input/output processor to both the first and second input/output busses and the second input/output processor to both the first and second input/output busses; and

in parallel with or after the instruction stream executing step, processing said input/output functions in both the first input/output processor and in the second input/output processor, wherein both the first and second input/output processors are operable to execute the same function.

12. A method according to claim 11 further including the step, after the step of coupling said input/output processors, of processing said input/output functions in at least one additional input/output processor connected to both said first and second input/output busses.

13. A method according to claim 10 further including the step of:

during the asynchronous memory reference performing step, detecting an event of a memory reference by a leading CPU which executes said instruction stream at an instruction step associated with said detected memory reference in said instruction stream which is earlier in actual execution of said instruction stream than when other said CPUs execute said instruction stream, and;

wherein the asynchronous memory reference performing step further includes performing the memory reference by all three CPUs relative to said detection of the event of the asynchronous memory reference by the leading CPU, wherein such execution of said instruction stream by said leading CPU is stalled until other said CPUs each executes said instruction stream to catch up with said leading CPU such that all three CPUs are executable in synchronism.

14. A method according to claim 10 further including the step of synchronizing said first, second and third CPUs with one another by stalling a leading CPU which performs said voted asynchronous memory reference earlier in actual performance time than when other said CPUs perform said voted asynchronous memory reference, in parallel with said asynchronous memory reference performing step.

15. A method according to claim 14 further including the step of accessing each said first and second memory modules and substantially simultaneously performing said step of voting, in parallel with or after said instruction stream executing step.

16. A computer system comprising:

first, second and third CPUs executing a same given instruction stream, each of said CPUs having an address range, each of said CPUs having a separate memory access port;

first and second memory means having identical address spaces within said address range of said CPUs for storing duplicative data at identical address spaces therein, each one of said first and second memory means having first, second and third input/output ports coupled to said memory access ports of said first, second and third CPUs, respectively; and

voting means coupled to each one of said first, second and third ports to compare information appearing at the ports upon asynchronous access occurring after asynchronous execution of plural consecutive instructions by said CPUs and to allow such asynchronous accesses to be completed only in response to the same information appearing at at least two of the ports.

17. A computer system according to claim 16 including first and second input/output busses, and wherein said first and second memory means each has a separate peripheral I/O port, and said first and second input/output busses are separately coupled to said I/O ports of said first and second memory means.

18. A computer system according to claim 16 including means for synchronizing execution of the instruction stream in said CPUs by stalling execution of an asynchronous memory reference by a leading CPU among said CPUs, wherein said leading CPU executes said asynchronous memory reference in said instruction stream which is earlier in actual execution than when other said CPUs execute said asynchronous memory reference in said instruction stream, until all of said first, second and third CPUs are executing the exact same asynchronous memory reference.

19. A system according to claim 16 wherein said information includes address and data information for write operations.

20. A system according to claim 16 comprising a separate clock coupled to each one of said CPUs for operating said CPUs on the given instruction stream asynchronously.

21. A system according to claim 20 comprising a separate clock coupled to each one of said first and second memory modules for operating said CPUs and memory modules asynchronously.

22. A system according to claim 21, wherein said CPUs are synchronized with one another in response to an event of an asynchronous memory reference by stalling a leading CPU until at least one trailing CPU executes the given instruction stream to catch up with said leading CPU, wherein said leading CPU executes said given instruction stream at an instruction step in said given instruction stream which is earlier in actual execution time of said given instruction stream than when each said trailing CPU executes said given instruction stream.

23. A method of operating a computer system having first and second memory means, and having first, second and third CPUs, each having an address range, and including first, second and third memory access busses coupling the CPUs and the memory means, the method comprising the steps of:

executing a same given instruction stream in the first, second and third CPUs;

in parallel with said instruction stream executing step, generating asynchronous memory accesses occurring after asynchronous execution of plural consecutive instructions by each of said first, second and third CPUs at separate ones of the first, second and third memory access busses;

in parallel with or after said instruction stream executing step, performing asynchronous memory accesses to each one of said first and second memory means via said first, second and third memory busses;

in parallel with or after said instruction stream executing step, storing duplicative data in identical address spaces in the first and second memory means within said address range of said CPUs;

in parallel with or after said asynchronous memory access performing step, voting each one of said memory accesses in said first and second memory means when received from said first, second and third memory access busses, said voting including comparing information representing said memory accesses; and

completing said asynchronous access in response to at least two of said first, second and third memory access busses presenting the same data.

24. A method according to claim 23 further including the step of synchronizing said first, second and third CPUs to execute the same instruction stream substantially simultaneously, in parallel with said instruction stram executing step.

25. A method according to claim 24 wherein said step of synchronizing said CPUs includes stalling performance of asynchronous memory accesses until each of the first, second and third CPUs is performing the same asynchronous memory access at substantially the same time.

26. A method according to claim 25 wherein said step of synchronizing also includes the synchronous application of external interrupts to the first, second and third CPUs, such that said CPUs are interrupted at the same point in said same given instruction stream.