Multi-processor system bridge with controlled access

• US 6,587,961 B1
• Filed: 06/15/1998
• Issued: 07/01/2003
• Est. Priority Date: 06/15/1998
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Garnett, Paul J., Oyelakin, Femi A., Rowlinson, Stephen
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): Bonzo, Bryce P.
• Application Number:
  US09/097,485
• Time in Patent Office: 1,842 Days
• Field of Search: 714/9, 714/11, 714/4, 710/305, 710/306, 710/307
• US Class Current: 714/11
• CPC Class Codes: G06F 11/004 Error avoidance G06F11/07 and subgroups take precedence
• CPC Class Codes: G06F 11/1625 in communications, e.g. transmission, interfaces
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/2268 Logging of test results
• CPC Class Codes: G06F 13/4027 using bus bridges G06F13/4022 takes precedence

Abstract

A bridge for a multi-processor system includes bus interfaces for connection to an I/O bus of a first processing set, an I/O bus of a second processing set and a device bus. It also comprises a bridge control mechanism configured to be operable, in an operational mode to permit access by at least one of the first and second processing sets to bridge resources and to the device bus and, in an error mode, to prevent access by the processing sets to the device bus and to permit restricted access to at least one of the processing sets to at least predetermined bridge resources. By providing restricted access to selected parameters held in the bridge during an error mode, the bridge can act as a secure repository for information which can be used by the processing sets to investigate the error and hopefully to recover therefrom, while preventing I/O devices connected to device bus from being corrupted by a faulty processing set. Storage in the bridge provides for buffering data pending resolution of the error.

Claims

1. A bridge for a multi-processor system, the bridge comprising a first processor bus interface for connection to an I/O bus of a first processing set, a second processor bus interface for connection to an I/O bus of a second processing set, a device bus interface for connection to a device bus and a bridge control mechanism configured to be operable:

in an operational mode to permit access by at least one of the first and second processing sets to bridge resources and to the device bus; and

in an error mode to prevent access by the processing sets to the device bus and to permit restricted access by at least one of the processing sets to at least predetermined bridge resources.

2. The bridge of claim 1, wherein the bridge control mechanism is configured to be operable, in response to detection of an error state to cause the bridge to cease operation in the operational mode and instead to operate in the error mode.

3. The bridge of claim 2, comprising storage for buffering data pending resolution of the error.

4. The bridge of claim 1 comprising error state registers for saving operating parameters on entry to the error mode, read only access to the error state registers being permitted by the processing sets during the error mode.

5. The bridge of claim 1 comprising a posted write buffer for the storage of writes already posted by at least one processing set on entry to the error mode, read only access to the posted write buffer being permitted by the processing sets during the error mode.

6. The bridge of claim 1, wherein the bridge control mechanism is configured to be operable in an initial error mode:

to store in the posted write buffer any internal bridge write accesses initiated by the processing sets and to allow and to arbitrate any internal bridge read accesses initiated by the processing sets.

7. The bridge of claim 1, wherein the bridge control mechanism is configured to be operable in an initial error mode:

to store in a posted write buffer any device bus write accesses initiated by the processing sets and to abort any device bus read accesses initiated by the processing sets.

8. The bridge of claim 1, wherein the bridge control mechanism is configured to be operable in a primary error mode in which a processing set asserts itself as a primary processing set:

to allow and to arbitrate any internal bridge write accesses initiated by the primary processing set, to discard any internal bridge write accesses initiated by any other processing set, and to allow and to arbitrate any internal bridge read accesses initiated by the processing sets.

9. The bridge of claim 8, wherein the bridge control mechanism is configured to be operable in the primary error mode:

to discard any device bus write accesses initiated by the processing sets and to abort any device bus read accesses initiated by the processing sets.

10. The bridge of claim 1, wherein the bridge control mechanism is configured to be operable:

in a first, split, operational mode to arbitrate between the first and the second processing sets for access to each others I/O bus and to the device bus; and

in a second, combined, operational mode to monitor lockstep operation of the first and second processing sets.

11. The bridge of claim 10, wherein the bridge control mechanism is configured to be operable on power up of the bridge to operate in an initial error mode until a processor set asserts itself as a primary processing set, then to operate in the split operational mode to enable all processing sets to be set to a corresponding state before transferring to the combined operational mode.

12. The bridge of claim 1, comprising a memory sub-system and a controllable routing matrix connected between the first processor bus interface, the second processor bus interface, the device bus interface and the memory sub-system, the bridge control mechanism being configured to be operable to control the routing matrix selectively to interconnect the first processor bus interface, the second processor bus interface, the device bus interface and the memory sub-system according to a current mode of operation.

13. The bridge of claim 1, comprising at least one further processor bus interface for connection to an I/O bus of a further processing set.

14. A bridge for a multi-processor system, the bridge comprising means for connection to an I/O bus of a first processing set, to an I/O bus of a second processing set and to a device bus, and a means for controlling operation of the bridge:

in an operational mode to permit access by at least one of the first and second processing sets to bridge resources and to the device bus; and

in an error mode to prevent access by the processing sets to the device bus and to permit restricted access by at least one of the processing sets to at least predetermined bridge resources.

15. A computer system comprising a first processing set having an I/O bus, a second processing set having an I/O bus, a device bus and a bridge, the bridge being connected to the I/O bus of the first processing set, the I/O bus of the second processing set and the device bus and comprising a bridge control mechanism configured to be operable:

in an operational mode to permit access by at least one of the first and second processing sets to bridge resources and to the device bus; and

in an error mode to prevent access by the processing sets to the device bus and to permit restricted access by at least one of the processing sets to at least determined bridge resources.

16. The system of claim 15, wherein the bridge control mechanism is configured to be operable, in response to detection of an error state to cause the bridge to cease operation in the operational mode and instead to operate in the error mode.

17. The system of claim 16, wherein the bridge comprises storage for buffering data pending resolution of the error.

18. The system of claim 15, wherein the bridge comprises error state registers for saving operating parameters on entry to the error mode, read only access to the error state registers being permitted by the processing sets during the error mode.

19. The system of claim 15, wherein the bridge comprises a posted write buffer for the storage of writes already posted by at least one processing set on entry to the error mode, read only access to the posted write buffer being permitted by the processing sets during the error mode.

20. The system of claim 15, wherein the bridge control mechanism is configured to be operable in an initial error mode:

to store in the posted write buffer any internal bridge write accesses initiated by the processing sets and to allow, to arbitrate any internal bridge read accesses initiated by the processing sets, to store in a posted write buffer any device bus write accesses initiated by the processing sets and to abort any device bus read accesses initiated by the processing sets.

21. The system of claim 15, wherein the bridge control mechanism is configured to be operable in a primary error mode in which a processing set asserts itself as a primary processing set:

to allow and to arbitrate any internal bridge write accesses initiated by the primary processing set, to discard any internal bridge write accesses initiated by any other processing set, to allow and to arbitrate any internal bridge read accesses initiated by the processing sets, to discard any device bus write accesses initiated by the processing sets and to abort any device bus read accesses initiated by the processing sets.

22. The system of claim 15, wherein the bridge control mechanism is configured to be operable:

in a first, split, operational mode to arbitrate between the first and the second processing sets for access to each others I/O bus and to the device bus; and

in a second, combined, operational mode to monitor lockstep operation of the first and second processing sets.

23. The system of claim 22, wherein the bridge control mechanism is configured to be operable on power up of the bridge to operate in an initial error mode until a processor set asserts itself as a primary processing set, then to operate in the split operational mode to enable all processing sets to be set to a corresponding state before transferring to the combined operational mode.

24. The system of claim 15, wherein the bridge comprises a memory sub-system and a controllable routing matrix connected between the first processor bus interface, the second processor bus interface, the device bus interface and the memory sub-system, the bridge control mechanism being configured to be operable to control the routing matrix selectively to interconnect the first processor bus interface, the second processor bus interface, the device bus interface and the memory sub-system according to a current mode of operation.

25. The system of claim 15, wherein each processing set comprises at least one processor, memory and a processing set I/O bus controller.

26. The system of claim 15, further comprising at least one further processing set, the bridge comprising at least one further processor bus interface for connection to an I/O bus of the at least one further processing set.

27. A method of operating a multi-processor system comprising a first processing set having an I/O bus, a second processing set having an I/O bus, a device bus and a bridge, the bridge being connected to the I/O bus of the first processing set, the I/O bus of the second processing set and the device bus, the method comprising selectively operating the bridge:

in an operational mode to permit access by at least one of the first and second processing sets to bridge resources and to the device bus; and

in an error mode to prevent access by the processing sets to the device bus and to permit restricted access by at least one of the processing sets to at least predetermined bridge resources.

28. The method of claim 27, wherein, in response to detection of an error state, the bridge ceases operation in the operational mode and instead operates in the error mode.

29. The method of claim 28, comprising buffering data in the bridge pending resolution of the error.

30. The method of claim 27, comprising saving operating parameters in error state registers in the bridge on entry to the error mode and permitting read only access to the error state registers by the processing sets during the error mode.

31. The method of claim 27, comprising storing, in a posted write buffer in the bridge, writes already posted from at least one processing set on entry to the error mode, and permitting read only access to the posted write buffer by the processing sets during the error mode.

32. The method of claim 27, comprising operating in an initial error mode in which:

any internal bridge write accesses initiated by the processing sets are stored in a posted write buffer in the bridge and any internal bridge read accesses initiated by the processing sets are allowed and arbitrated by the bridge; and

any device bus write accesses initiated by the processing sets are posted in a posted write buffer in the bridge and any device bus read accesses initiated by the processing sets are aborted by the bridge.

33. The method of claim 32, comprising subsequently operating in a primary error mode in which a processing set asserts itself as a primary processing set, in which:

any internal bridge write accesses initiated by the primary processing set are allowed and arbitrated by the bridge, any internal bridge write accesses initiated by any other processing set are discarded by the bridge, and any internal bridge read accesses initiated by the processing sets are allowed and arbitrated by the bridge; and

any device bus write accesses initiated by the processing sets are discarded by the bridge and any device bus read accesses initiated by the processing sets are aborted by the bridge.

34. The method of claim 27, comprising operating in:

a split operational mode in which accesses by the first and the second processing sets to each others I/O bus and to the device bus are arbitrated by the bridge; and

a combined operational mode in which lockstep operation of the first and second processing sets is monitored by the bridge.

35. The method of claim 34, wherein on power up the bridge operates in an initial error mode until a processor set asserts itself as a primary processing set, and then the bridge operates in the split operational mode to enable all processing sets to be set to a corresponding state before transferring to the combined operational mode.