Method and apparatus for transaction tag assignment and maintenance in a distributed symmetric multiprocessor system

• US 20030046356A1
• Filed: 06/05/2002
• Published: 03/06/2003
• Est. Priority Date: 11/08/1999
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Mui, David, Alvarez, Manuel Joseph II, Deshpande, Sanjay Raghunath, Klapproth, Kenneth Douglas
• Granted Patent: US 7,529,799 B2
• Time in Patent Office: Days
• US Class Current: 709/208
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration

Abstract

A distributed system structure for a large-way, symmetric multiprocessor system using a bus-based cache-coherence protocol is provided. The distributed system structure contains an address switch, multiple memory subsystems, and multiple master devices, either processors, I/O agents, or coherent memory adapters, organized into a set of nodes supported by a node controller. The node controller receives transactions from a master device, communicates with a master device as another master device or as a slave device, and queues transactions received from a master device. Since the achievement of coherency is distributed in time and space, the node controller helps to maintain cache coherency. A transaction tag format for a standard bus protocol is expanded to ensure unique transaction tags are maintained throughout the system. A sideband signal is used for intervention and Reruns to preserve transaction tags at the node controller in certain circumstances.

Claims

1. A method of maintaining cache coherency in a multiprocessor system comprising a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, wherein a master device bus connects one or more master devices within a node to a port of the node controller, the method comprising the steps of:

broadcasting a transaction to the plurality of master devices;

receiving a response for the broadcast transaction from a master device in the plurality of master devices, wherein the response indicates a subsequent transaction to be generated by the master device in response to the broadcast transaction; and

notifying a node controller for the master device of the subsequent transaction to be generated by the master device.

2. The method of claim 1, wherein the node controller is notified by using a sideband signal.

3. The method of claim 1 further comprising:

in response to receiving the notification at the node controller, preserving a system-level tag for the broadcast transaction.

4. The method of claim 1 further comprising:

registering the broadcast transaction in an entry in a transaction registry in the node controller.

5. The method of claim 1 wherein the notification to the node controller of the subsequent transaction is received with a global AResp result.

6. An apparatus for maintaining cache coherency in a multiprocessor system comprising a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, wherein a master device bus connects one or more master devices within a node to a port of the node controller, the apparatus comprising:

broadcasting means for broadcasting a transaction to the plurality of master devices;

first receiving means for receiving a response for the broadcast transaction from a master device in the plurality of master devices, wherein the response indicates a subsequent transaction to be generated by the master device in response to the broadcast transaction; and

notifying means for notifying a node controller for the master device of the subsequent transaction to be generated by the master device.

7. The apparatus of claim 6, wherein the notifying means is a sideband signal.

8. The apparatus of claim 6 further comprising:

preserving means for preserving, in response to receiving the notification at the node controller, a system-level tag for the broadcast transaction.

9. The apparatus of claim 6 further comprising:

registering means for registering the broadcast transaction in an entry in a transaction registry in the node controller.

10. The apparatus of claim 6 wherein the notification to the node controller of the subsequent transaction is received with a global AResp result.

11. A computer program product in a computer-readable medium for using in a multiprocessor system for maintaining cache coherency, the multiprocessor system comprising a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, wherein a master device bus connects one or more master devices within a node to a port of the node controller, the computer program product comprising:

instructions for broadcasting a transaction to the plurality of master devices;

instructions for receiving a response for the broadcast transaction from a master device in the plurality of master devices, wherein the response indicates a subsequent transaction to be generated by the master device in response to the broadcast transaction; and

instructions for notifying a node controller for the master device of the subsequent transaction to be generated by the master device.

12. The computer program product of claim 11, wherein the instructions for notifying a node controller generate a sideband signal.

13. The computer program product of claim 12 further comprising:

instructions for preserving, in response to receiving the notification at the node controller, a system-level tag for the broadcast transaction.

14. The computer program product of claim 11 further comprising:

instructions for registering the broadcast transaction in an entry in a transaction registry in the node controller.

15. The computer program product of claim 11 wherein the notification to the node controller of the subsequent transaction is received with a global AResp result.

16. A method of maintaining cache coherency in a multiprocessor system, the method comprising the steps of:

receiving a first transaction from a master device, wherein the first transaction comprises a port-bus tag;

translating the port-bus tag for the first transaction to a system-level tag; and

registering the first transaction in an entry in a first transaction registry.

17. The method of claim 16 further comprising:

storing the port-bus tag for the first transaction in the entry in the first transaction registry.

18. The method of claim 16 further comprising:

broadcasting a second transaction, wherein the second transaction comprises a system-level tag for the first transaction.

19. The method of claim 18 further comprising:

registering the second transaction in an entry in a second transaction registry.

20. The method of claim 19 further comprising:

translating a system-level tag for the second transaction to a snoop tag;

storing the system-level tag for the second transaction in the entry in the second transaction registry; and

forwarding the second transaction with the snoop tag to master devices.

21. The method of claim 20 further comprising:

receiving intervention data;

matching a system tag for the intervention data with the port-bus tag for the first transaction in the entry in the first transaction registry;

forwarding the intervention data with the port-bus tag for the first transaction.

22. The method of claim 20 further comprising:

receiving a Rerun command;

matching a system tag for the Rerun command with the port-bus tag for the first transaction in the entry in the first transaction registry;

reissuing the first transaction with its port-bus tag with an address modifier bit set.

23. The method of claim 16 wherein the multiprocessor system comprises a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, and wherein a master device bus connects one or more master devices within a node to a port of the node controller.

24. The method of claim 23 wherein the port-bus tag further comprises:

a source identifier that uniquely identifies the master device within a node that issued the first transaction; and

a transaction identifier that uniquely identifies the first transaction in a set of transactions issued by the master device.

25. An apparatus for maintaining cache coherency in a multiprocessor system, the apparatus comprising:

first receiving means for receiving a first transaction from a master device, wherein the first transaction comprises a port-bus tag;

first translating means for translating the port-bus tag for the first transaction to a system-level tag; and

first registering means for registering the first transaction in an entry in a first transaction registry.

26. The apparatus of claim 25 further comprising:

first storing means for storing the port-bus tag for the first transaction in the entry in the first transaction registry.

27. The apparatus of claim 25 further comprising:

broadcasting means for broadcasting a second transaction, wherein the second transaction comprises a system-level tag for the first transaction.

28. The apparatus of claim 27 further comprising:

first registering means for registering the second transaction in an entry in a second transaction registry.

29. The apparatus of claim 28 further comprising:

first translating means for translating a system-level tag for the second transaction to a snoop tag;

first storing means for storing the system-level tag for the second transaction in the entry in the second transaction registry; and

forwarding means for forwarding the second transaction with the snoop tag to master devices.

30. The apparatus of claim 29 further comprising:

first receiving means for receiving intervention data;

first matching means for matching a system tag for the intervention data with the port-bus tag for the first transaction in the entry in the first transaction registry;

forwarding means for forwarding the intervention data with the port-bus tag for the first transaction.

31. The apparatus of claim 29 further comprising:

second receiving means for receiving a Rerun command;

second matching means for matching a system tag for the Rerun command with the port-bus tag for the first transaction in the entry in the first transaction registry;

reissuing means for reissuing the first transaction with its port-bus tag with an address modifier bit set.

32. The apparatus of claim 25 wherein the multiprocessor system comprises a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, and wherein a master device bus connects one or more master devices within a node to a port of the node controller.

33. The apparatus of claim 31 wherein the port-bus tag further comprises:

a source identifier that uniquely identifies the master device within a node that issued the first transaction; and

a transaction identifier that uniquely identifies the first transaction in a set of transactions issued by the master device.

34. A computer program product in a computer-readable medium for use in a multiprocessor system for maintaining cache coherency in the multiprocessor system, the computer program product comprising:

instructions for receiving a first transaction from a master device, wherein the first transaction comprises a port-bus tag;

instructions for translating the port-bus tag for the first transaction to a system-level tag; and

instructions for registering the first transaction in an entry in a first transaction registry.

35. The computer program product of claim 34 further comprising:

instructions for storing the port-bus tag for the first transaction in the entry in the first transaction registry.

36. The computer program product of claim 34 further comprising:

instructions for broadcasting a second transaction, wherein the second transaction comprises a system-level tag for the first transaction.

37. The computer program product of claim 36 further comprising:

instructions for registering the second transaction in an entry in a second transaction registry.

38. The computer program product of claim 37 further comprising:

instructions for translating a system-level tag for the second transaction to a snoop tag;

instructions for storing the system-level tag for the second transaction in the entry in the second transaction registry; and

instructions for forwarding the second transaction with the snoop tag to master devices.

39. The computer program product of claim 38 further comprising:

instructions for receiving intervention data;

instructions for matching a system tag for the intervention data with the port-bus tag for the first transaction in the entry in the first transaction registry;

instructions for forwarding the intervention data with the port-bus tag for the first transaction.

40. The computer program product of claim 38 further comprising:

instructions for receiving a Rerun command;

instructions for matching a system tag for the Rerun command with the port-bus tag for the first transaction in the entry in the first transaction registry;

instructions for reissuing the first transaction with its port-bus tag with an address modifier bit set.

41. The computer program product of claim 34 wherein the multiprocessor system comprises a plurality of master devices and a plurality of node controllers, wherein a node controller organizes a subset of one or more of the plurality of master devices into a node, and a plurality of bidirectional master device buses, and wherein a master device bus connects one or more master devices within a node to a port of the node controller.

42. The computer program product of claim 40 wherein the port-bus tag further comprises:

a source identifier that uniquely identifies the master device within a node that issued the first transaction; and

a transaction identifier that uniquely identifies the first transaction in a set of transactions issued by the master device.