Channel interface and protocols for cache coherency in a scalable symmetric multiprocessor system

• US 6,516,442 B1
• Filed: 03/30/1999
• Issued: 02/04/2003
• Est. Priority Date: 12/07/1997
• Status: Expired due to Term

• Current Assignee: Intellectual Ventures II LLC (Intellectual Ventures LLC)
• Original Assignee: Conexant Systems Incorporated (Synaptics Incorporated)
• Inventors: Wang, Yuanlong, Biard, Brian R., Fu, Daniel, Cohen, Earl T., Amdahl, Carl G.
• Primary Examiner(s): Decady, Albert
• Assistant Examiner(s): BRITT, CYNTHIA H
• Application Number:
  US09/281,749
• Time in Patent Office: 1,407 Days
• Field of Search: 714/748, 714/18, 714/751, 714/20, 714/4, 714/776, 370/244, 370/235, 711/141, 711/146
• US Class Current: 714/776
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0822 Copy directories local copy tags for implementing a bus snooping protocol G06F12/0831
• CPC Class Codes: G06F 15/17375 One dimensional, e.g. linear array, ring

Abstract

A preferred embodiment of a symmetric multiprocessor system includes a switched fabric (switch matrix) for data transfers that provides multiple concurrent buses that enable greatly increased bandwidth between processors and shared memory. A high-speed point-to-point Channel couples command initiators and memory with the switch matrix and with I/O subsystems. Each end of a channel is connected to a Channel Interface Block (CIB). The CIB presents a logical interface to the Channel, providing a communication path to and from a CIB in another IC. CIB logic presents a similar interface between the CIB and the core-logic and between the CIB and the Channel transceivers. A channel transport protocol is is implemented in the CIB to reliably transfer data from one chip to another in the face of errors and limited buffering.

Claims

1. A shared-memory multi-processor system comprising:

a switch fabric configured to switch packets containing data;

a plurality of channels configured to transfer the packets;

a plurality of switch interfaces configured to exchange the packets with the switch fabric, exchange the packets over the channels, and perform error correction of the data in the packets exchanged over the channels;

a plurality of microprocessor interfaces configured to exchange the data with a plurality of microprocessors, exchange the packets with the switch interfaces over the channels, and perform error correction of the data in the packets exchanged over the channels; and

a memory interface configured to exchange the data with a memory device, exchange the packets with the switch interfaces over the channels, and perform error correction of the data in the packets exchanged over the channels.

2. The shared-memory multi-processor system of claim 1 wherein the interfaces are configured to add error correction codes to the packets being transferred over the channels to check the error correction codes in the packets being received over the channels and to transfer a retry request if one of the packets being received has an error.

3. The shared-memory multi-processor system of claim 1 wherein the interfaces are configured to add sequence codes to the packets being transferred over the channels, to check the sequence codes in the packets being received over the channels, and to transfer a retry request if there is a sequence error.

4. The shared-memory multi-processor system of claim 1 wherein the interfaces are configured to recalibrate one of the channels after a number of retry failures over the one channel.

5. The shared-memory multi-processor system of claim 1 wherein all of the interfaces are configured to perform a synchronous reset.

6. The shared-memory multi-processor system of claim 1 wherein the interfaces are configured to transfer a backoff request to reduce packet transmission over one of the channels.

7. The shared-memory multi-processor system of claim 1 wherein the microprocessor interfaces are configured to operate based clocks in the microprocessors.

8. The shared-memory multi-processor system of claim 1 further comprising a bus interface configured to exchange the data with a bus, exchange the packets with the switch interfaces over the channels, and perform error correction of the data in the packets exchanged over the channels.

9. The shared-memory multi-processor system of claim 1 further comprising a transaction controller configured to maintain an order of data transactions between the microprocessors and the memory device based on a cache coherency protocol.

10. The shared-memory multi-processor system of claim 1 further comprising a transaction controller configured to maintain a duplicate set of cache tags for the microprocessors and maintain an order of cache tag updates for the microprocessors.

11. The shared-memory multi-processor system of claim 1 further comprising the microprocessors and the memory device.

12. The shared-memory multi-processor system of claim 1 wherein the channels each comprise a point-to-point connection between a pair of the interfaces.

13. A shared-memory multi-processor system comprising:

a switch fabric means for switching packets containing data;

a channel means for transferring the packets;

a switch interface means for exchanging the packets with the switch fabric means, exchanging the packets over the channel means, and performing error correction of the data in the packets exchanged over the channel means;

a microprocessor interface means for exchanging the data with a plurality of microprocessors, exchanging the packets with the switch interface means over the channel means, and performing error correction of the data in the packets exchanged over the channel means; and

a memory interface means for exchanging the data with a memory device, exchanging the packets with the switch interface means over the channel means, and performing error correction of the data in the packets exchanged over the channel means.

14. The shared-memory multi-processor system of claim 13 wherein the interface means are for adding error correction codes to the packets being transferred over the channel means, checking the error correction codes in the packets being received over the channel means, and transferring a retry request if one of the packets being received has an error.

15. The shared-memory multi-processor system of claim 13 wherein the interface means are for adding sequence codes to the packets being transferred over the channel means, checking the sequence codes in the packets being received over the channel means, and transferring a retry request if there is a sequence error.

16. The shared-memory multi-processor system of claim 13 wherein the interface means are for recalibrating the channel means after a number of retry failures.

17. The shared-memory multi-processor system of claim 13 wherein the interface means are for performing a synchronous reset.

18. The shared-memory multi-processor system of claim 13 wherein the interface means are for transferring a backoff request to reduce packet transmission over the channel means.

19. The shared-memory multi-processor system of claim 13 wherein the microprocessor interface means are for operating based clocks in the microprocessors.

20. The shared-memory multi-processor system of claim 13 further comprising a bus interface means for exchanging the data with a bus, exchanging the packets with the switch interface means over the channel means, and performing error correction of the data in the packets exchanged over the channel means.

21. The shared-memory multi-processor system of claim 13 further comprising a transaction controller means for maintaining an order of data transactions between the microprocessors and the memory device based on a cache coherency protocol.

22. The shared-memory multi-processor system of claim 13 further comprising a transaction controller means for maintaining a duplicate set of cache tags for the microprocessors and maintaining an order of cache tag updates for the microprocessors.

23. The shared-memory multi-processor system of claim 13 further comprising the microprocessors and the memory device.

24. A method of operating a shared-memory multi-processor system, the method comprising:

exchanging data between a plurality of microprocessors and a plurality of microprocessor interfaces;

exchanging packets containing the data between the microprocessor interfaces and a plurality of switch interfaces over channels;

exchanging the packets between the switch interfaces through a switch fabric;

exchanging the packets between the switch interfaces and a memory interface over the channels;

exchanging the data between the memory interface and a memory device; and

in the interfaces, performing error correction of the data in the packets exchanged over the channels.

25. The method of claim 24 wherein performing error correction of the data in the packets exchanged over the channels comprises:

adding error correction codes to the packets being transferred over the channels;

checking the error correction codes in the packets being received over the channels; and

transferring a retry request if one of the packets being received has an error.

26. The method of claim 24 wherein performing error correction of the data in the packets exchanged over the channels comprises:

adding sequence codes to the packets being transferred over the channels;

checking the sequence codes in the packets being received over the channels; and

transferring a retry request if there is a sequence error.

27. The method of claim 24 comprising recalibrating one of the channels after a number of retry failures over the one channel.

28. The method of claim 24 comprising performing a synchronous reset of one of the channels.

29. The method of claim 24 comprising transferring a backoff request to reduce packet transmission over one of the channels.

30. The method of claim 24 comprising operating the microprocessor interfaces based on clocks in the microprocessors.

31. The method of claim 24 further comprising:

exchanging the packets between the switch interfaces and a bus interface over the channels; and

exchanging the data between the bus interface and a bus.

32. The method of claim 24 further comprising maintaining an order of data transactions between the microprocessors and the memory device based on a cache coherency protocol.

33. The method of claim 24 further comprising:

maintaining a duplicate set of cache tags for the microprocessors; and

maintaining an order of cache tag updates for the microprocessors.

34. The method of claim 24 wherein the channels each comprise a point-to-point connection between a pair of the interfaces.

35. A system for transporting information comprising:

a switched fabric including a single system-serialization point;

a transaction initiator having a cache memory, wherein said transaction initiator is coupled to said switched fabric via a point-to-point connection;

a first channel interface, situated between said transaction initiator and said point-to-point connection;

a second channel interface, situated between said point-to-point connection and said switched fabric, wherein said first and second channel interfaces transport packets across said point-to-point connection according to a channel protocol to perform error correction of data in said packets; and

a transaction controller to monitor said packets passing through said system-serialization point and to maintain the coherency of said cache memory.

36. The system of claim 35 wherein said transaction initiator includes core logic that operates according to a first clock, said packets include an error correction code, said first channel interface checks said error correction code within one cycle of said first clock after receiving a first packet from said second channel interface to determine whether an error has occurred, and if so, notifies said second channel interface to re-send said first packet.

37. The system of claim 36 wherein said channel protocol specifies the number of times said first channel interface will notify said second channel interface to re-send said first packet before re-calibrating said point-to-point connection.

38. The system of claim 36 wherein said first channel interface sends said first packet to said core logic before completing said check of the error correction code, and if an error is found, notifies said core logic that said first packet is invalid.

39. The system of claim 35 wherein said first and second channel interfaces exchange flow control information.

40. The system of claim 39 wherein said flow control information comprises a command to cease sending packets containing data.

41. The system of claim 35 wherein said transaction initiator includes core logic that provides a transaction-level layer of said channel protocol, and wherein said first and second channel interfaces include channel interface logic that provides a link-level layer of said channel protocol and a transceiver that provides a physical layer of said channel protocol.

42. A system for transporting information comprising:

a switched fabric including a single system-serialization point;

a first processor having a cache memory, wherein said processor is coupled to said switched fabric via a first point-to-point connection;

a first channel interface, situated between said processor and said first point-to-point connection, and a second channel interface, situated between said first point-to-point connection and said switched fabric, wherein said first and second channel interfaces transport packets across said first point-to-point connection according to a channel protocol to perform error correction of data in said packets;

a system memory coupled to said switched fabric via a second point-to-point connection;

a third channel interface, situated between said system memory and said second point-to-point connection, and a fourth channel interface, situated between said second point-to-point connection and said switched fabric, wherein said third and fourth channel interfaces transport packets across said second point-to-point connection according to said channel protocol to perform error correction of data in said packets; and

a transaction controller to monitor said packets passing through said system-serialization point and to maintain the coherency of said cache memory.

43. The system of claim 42 wherein said first, second, third, and fourth channel interfaces present a common interface.

44. The system of claim 42 further comprising:

an input/output (I/O) device coupled to said switched fabric via a third point-to-point connection; and

a fifth channel interface, situated between said I/O device and said third point-to-point connection, and a sixth channel interface, situated between said third point-to-point connection and said switched fabric, wherein said fifth and sixth channel interfaces transport packets across said third point-to-point connection according to said channel protocol to perform error correction of data in said packets.