CACHE COHERENCE UNIT FOR INTERCONNECTING MULTIPROCESSOR NODES HAVING PIPELINED SNOOPY PROTOCOL

• US 20010013089A1
• Filed: 03/12/1998
• Published: 08/09/2001
• Est. Priority Date: 03/12/1998
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: WEBER, WOLF-DIETRICH
• Granted Patent: US 6,631,448 B2
• Time in Patent Office: Days
• US Class Current: 711/146
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0828 with concurrent directory accessing, i.e. handling multiple concurrent coherency transactions
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means

Abstract

The present invention consists of a cache coherence protocol within a cache coherence unit for use in a data processing system. The data processing system is comprised of multiple nodes, each node having a plurality of processors with associated caches, a memory, and input/output. The processors within the node are coupled to a memory bus operating according to a “snoopy” protocol. This invention includes a cache coherence protocol for a sparse directory in combination with the multiprocessor nodes. In addition, the invention has the following features: the current state and information from the incoming bus request are used to make an immediate decision on actions and next state; the decision mechanism for outgoing coherence is pipelined to follow the bus; and the incoming coherence pipeline acts independently of the outgoing coherence pipeline.

Claims

1. A data processing system comprising:

a plurality of multiprocessor nodes;

a plurality of coherence units coupled to said multiprocessor nodes; and

a node interconnect coupled to said coherence units.

2. A cache coherence unit for use in a data processing system having multiple nodes each having a plurality of processors coupled to a memory bus operating according to a snoopy protocol, each processor having an associated cache memory for caching information, said cache coherence unit having an address phase, a snoop phase, and a response phase, and comprising:

a bus interface element coupled to said memory bus;

a coherence control element coupled to said bus interface element and coupled to said cache memory; and

a directory coupled to said coherence control element for storing information of locations in said cache memory.

3. The cache coherence unit as in

claim 2

, wherein said coherence control element reads state information from said directory after said address phase and during said snoop phase, and updates said state information in said directory before a next snoop phase.

4. The coherence unit of

claim 2

further comprising a protocol decision pipeline coupled to said coherence control element for processing requests for cache memory from a remote node.

5. The coherence unit of

claim 2

where said directory is a sparse directory.

6. The coherence unit of

claim 2

wherein said directory has dual ports.

7. The cache coherence unit of

claim 2

, wherein said directory stores state information characterizing a current transient state and a next transient state.

8. The cache coherence unit of

claim 7

, wherein said coherence control element updates said information characterizing a current transient state.

9. The cache coherence unit of

claim 7

, wherein said coherence control element updates said information characterizing a current transient state and then updates said information characterizing a next transient state.

10. A method for maintaining cache coherence in a cache coherence unit in a data processing system having multiple nodes each having a plurality of processors coupled to a memory bus operating according to a snoopy protocol having an address phase, a snoop phase, and a response phase, each processor having an associated cache memory, comprising the steps of:

coupling a bus interface element to said memory bus;

coupling a coherence control element to said bus interface element and to said cache memory; and

storing information of locations in said cache memory locations in a directory coupled to said coherence control element.

11. The method of

claim 10

, wherein said coupling step further comprises:

reading state information from said directory after said address phase and before said snoop phase; and

updating said state information before a next snoop phase.

12. The cache coherence unit of

claim 10

, wherein said coherence control element further comprises:

means for reading state information from said directory after said address phase and before said snoop phase; and

means for updating said state information before a next snoop phase.

13. The cache coherence unit of

claim 11

, wherein said coherence control element further comprises:

means for updating information characterizing a current transient state;

means for reading information characterizing a next transient state; and

means for updating said information characterizing a next transient state after said means for updating information characterizing a current transient state is completed.

14. A data processing system comprising:

a first node having a first memory coupled to a first memory bus, a plurality of processors coupled to said first memory bus, each processor having a respective cache and said memory bus operated according to a snoopy bus protocol for maintaining coherence between said caches, said snoopy bus protocol having an address phase, a snoop phase, and a response phase;

a second node having a second memory coupled to a second memory bus, a plurality of processors coupled to said second memory bus, each processor having a respective cache and said memory bus operated according to said snoopy bus protocol;

a first internode communication unit coupled to said first memory bus and having a first directory for indicating states of cached blocks of said first and second memories and a coherence control element coupled to said first bus and coupled to said first directory for reading state information from said first directory after said address phase and before said snoop phase and updating said state information before a next snoop phase; and

a second internode communication unit coupled to said second memory bus and coupled to said first internode communication unit, said first memory being accessible to said second node and said second memory being accessible to said first node, and said second coherence unit having a second directory for indicating states of cached blocks of said second memory and a coherence control element coupled to said second bus and coupled to said second directory for reading state information from said second directory after said address phase and before said snoop phase and updating said state information before a next snoop phase.

15. The data processing system of

claim 14

further comprising a plurality of nodes.

16. The data processing system of

claim 14

wherein said first and second directories are sparse directories.

17. The data processing system of

claim 14

, wherein said first and second internode communication units further comprise:

a mesh interface unit;

a network interface unit coupled to said mesh interface unit; and

a transfer agent coupled to said network interface unit, said memory buses, and said directories.

18. The data processing system of

claim 17

, wherein said network interface unit further comprises:

a data transfer unit coupled to an interconnect unit and to said transfer agent for transferring data and control requests from said interconnect unit to said transfer agent; and

a coherence action unit coupled to said interconnect unit and said directory for reading state information from said directory.

19. A method of maintaining cache coherency within a data processing system having a plurality of nodes and an interconnect, each node having a plurality of multiprocessors with associated caches and memory, a memory bus controlled by a snoopy bus protocol, a directory for storing state information of cached memory lines, and a mesh interface controlled by a mesh protocol, comprising the steps of:

receiving a bus request from the said memory bus for the control of said cached memory line;

reading said state information of said cached line;

updating said state information of said cache line;

transforming said bus request into a network request; and

forwarding said network request to said mesh interface.

20. The method of

claim 19

, further comprising the steps of:

receiving a network request from said mesh interface;

arbitrating with said memory bus for scheduling said request onto the said memory bus;

passing said network request to a transfer agent; and

transferring said network requests from said transfer agent to said memory bus.

21. A cache coherence unit within a data processing system having a plurality of nodes, each node having a plurality of processors coupled to a memory bus operating according to a snoopy bus protocol, each processor having an associated cache, and coupled to an interconnect, said cache coherence unit comprising:

a control element for arbitrating with said memory bus;

a coherence control unit coupled to said control element for maintaining coherence of cached memory locations of bus requests received from said memory bus;

a directory coupled to said coherence control unit for storing state information of said cached memory locations;

a first interface unit coupled to said coherence control unit for transforming said bus requests into mesh-out requests;

a mesh interface coupled to said first interface unit for transferring said mesh-out requests to said interconnect;

a second interface unit coupled to said mesh interface for receiving mesh-in requests from said interconnect; and

a transfer agent coupled to said second interface unit and coupled to said memory bus for transferring said mesh-in requests from said second interface unit to said memory bus.

22. The cache coherence unit of

claim 21

, wherein said coherence control unit further comprises:

a first agent coupled to said first interface unit for transforming said bus into mesh-out requests; and

a second agent coupled to said first interface for transferring said mesh-out requests to said interconnect.

23. The cache coherence unit of

claim 21

, wherein said coherence control unit further comprises:

a third agent coupled to said first interface for transferring data from said memory bus to said interconnect.

24. The cache coherence unit of

claim 21

, wherein said second interface unit further comprises:

a first input agent for mapping incoming network request from said remote nodes to bus transactions; and

a second input agent for handling incoming mesh replies and dealing with all mesh-to-bus data transfers.

25. The cache coherence unit of

claim 21

, wherein said second interface unit further comprises:

a data transfer unit coupled to said mesh interface and said data storage unit; and

a coherence action unit coupled to said mesh interface unit and said directory.

26. A cache coherence unit within a data processing system having a plurality of nodes and an interconnect, each node having a plurality of multiprocessors with associated caches and memory, a memory bus controlled by a snoopy bus protocol, a directory for storing state information of cached memory lines, and a mesh interface controlled by a mesh protocol, comprising:

a control unit coupled to said memory bus for receiving bus requests;

means for reading said state information of said cached line;

means for updating said state information of said cache line; and

a mesh interface for forwarding said bus requests to said interconnect.

27. The cache coherence unit of

claim 26

, further comprising:

means for receiving network requests from said interconnect;

a bus arbitration unit coupled to said memory bus for scheduling said network requests onto the said memory bus;

means for passing said network request to a transfer agent; and

means for transferring said network requests from said transfer agent to said memory bus.

28. A method for maintaining cache coherence in a multi-node data. processing system, comprising the steps of:

receiving a request for a cache line from a local node;

determining the status of said cache line; and

sending a copy of said cache line to said local node.

29. The method of

claim 28

, further comprising the steps of:

invalidating the status of remote copies of said cache line; and

notifying said local node that transfer of said cache line is complete.

30. The method of

claim 28

, further comprising the steps of:

requesting a copy of said cache line from a home node; and

receiving a copy of said cache line from said home node.

31. A system for maintaining cache coherence in a multi-node data processing system, comprising:

means for receiving a request for a cache line from a local node;

means for determining the status of said cache line; and

means for sending a copy of said cache line to said local node.

32. The system of

claim 31

, further comprising:

means for invalidating the status of remote copies of said cache line; and

means for notifying said local node that transfer of said cache line is complete.

33. The system of

claim 31

, further comprising:

means for requesting a copy of said cache line from a home node; and

means for receiving a copy of said cache line from said home node.