METHOD AND SYSTEM FOR ORDERING I/O ACCESS IN A MULTI-NODE ENVIRONMENT

• US 20150254207A1
• Filed: 03/07/2014
• Published: 09/10/2015
• Est. Priority Date: 03/07/2014
• Status: Active Grant

• Current Assignee: Marvell Asia Pte Limited (Marvell Technology Group Limited)
• Original Assignee: Cavium, LLC (Marvell Technology Group Limited)
• Inventors: Kessler, Richard E.
• Granted Patent: US 10,592,459 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0822 Copy directories local copy tags for implementing a bus snooping protocol G06F12/0831
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 13/10 Program control for peripheral devices G06F13/14 - G06F13/42 take precedence
• CPC Class Codes: G06F 13/20 for access to input/output bus
• CPC Class Codes: G06F 13/423 with synchronous protocol

Abstract

According to at least one example embodiment, a multi-chip system includes multiple chip devices configured to communicate to each other and share resources, such as I/O devices. According to at least one example embodiment, a method of synchronizing access to an input/output (I/O) device in the multi-chip system comprises initiating, by a first agent of the multi-chip system, a first operation for accessing the I/O device, the first operation is queued, prior to execution by the I/O device, in a queue. Once the first operation is queued, an indication of such queuing is provided. Upon detecting, by a second agent of the multi-chip system, the indication of queuing the first operation in the queue, initiating a second operation to access the I/O device, the second operation is queued subsequent to the first operation in the queue.

Claims

1. A method of synchronizing access to an input/output (I/O) device in a multi-chip system, the method comprising:

initiating, by a first agent of the multi-chip system, a first operation for accessing the I/O device, the first operation is queued, prior to execution by the I/O device, in a queue;

providing an indication of queuing the first operation in the queue;

upon detecting, by a second agent of the multi-chip system, the indication of queuing the first operation in the queue, initiating a second operation to access the I/O device, the second operation is queued subsequent to the first operation in the queue.

2. The method as recited in claim 1 further comprising executing a synchronization operation by the first agent of the multi-chip system, the synchronization operation enforcing the indication to be provided only after the first operation is already queued in the queue.

3. The method as recited in claim 2, wherein executing a synchronization operation includes:

executing a first synchronization operation if the first agent and the second agent reside in the same chip device of the multi-chip system; and

executing a second synchronization operation if the first agent and the second agent reside in different chip devices of the multi-chip system.

4. The method as recited in claim 1, wherein the queue and the I/O device reside in a same chip device of the multi-chip system.

5. The method as recited in claim 4, wherein the I/O device and the first agent reside in the same chip device of the multi-chip system, and wherein initiating the first operation includes sending the first operation directly to the queue for ordering access to the I/O device.

6. The method as recited in claim 4, wherein the I/O device and the queue reside in a first chip device, the first agent resides in a second chip device of the multi-chip system, and wherein initiating the first operation includes:

sending the first operation to an other queue residing in the second chip device; and

forwarding the first operation from the other queue in the second chip device to the queue residing in the first chip device through an inter-chip interconnect interface.

7. The method as recited in claim 6 further comprising:

receiving, by the first agent, an acknowledgement of queuing the first operation from the first chip device, wherein providing the indication of queuing the first operation is in response to receiving the acknowledgement.

8. The method as recited in claim 4, wherein the second agent and the I/O device reside in the same chip device of the multi-chip system, and wherein initiating the second operation includes sending the second operation directly to the queue.

9. The method as recited in claim 4, wherein the I/O device and the queue reside in a first chip device, the first agent resides in a second chip device of the multi-chip system, and wherein initiating the second operation includes:

sending the second operation to an other queue in the second chip device; and

forwarding the second operation from the other queue in the second chip device to the queue residing in the first chip device through an inter-chip interconnect interface.

10. The method as recited in claim 1, wherein the I/O device resides on a first chip device of the multi-chip system, and both the first agent and the second agent reside on a second chip of the multi-chip system, the method further comprising providing an indication of synchronizing access to the I/O device locally within the second chip device.

11. The method as recited in claim 10, wherein the queue resides on the second chip device.

12. The method as recited in claim 1, wherein providing an indication of queuing the first operation includes modifying a flag value by the first agent.

13. The method as recited in claim 1, wherein the first or second operation is a store operation for writing data to the I/O device.

14. The method as recited in claim 1, wherein the first agent or the second agent is a core processor of the multi-chip system.

15. A multi-chip system comprising:

multiple chip devices, at least one chip device of the multiple chip devices including an input/output (I/O) device, and each chip device of the multiple chip devices including;

an inter-chip interconnect interface configured to couple the chip device to one or more other chip devices of the multi-chip system,

multiple agents configured to initiate operations for accessing the I/O device;

a queue configured to queue operation for accessing the I/O device generated by one or more agents of the multiple agents,

wherein a first agent of the multi-chip system is further configured to, upon initiating a first operation for accessing the I/O device, provide an indication of queuing the first operation in a first queue of the multi-chip system, and

a second agent of the multi-chip system is configured to initiate a second operation to access the I/O device, in response to detecting the indication of queuing the first operation in the first queue, the second operation is queued subsequent to the first operation in the first queue.

16. The multi-chip system as recited in claim 15, wherein the first agent of the multi-chip system is further configured to execute a synchronization operation, the synchronization operation enforcing the indication to be provided only after the first operation is already queued in the queue.

17. The multi-chip system as recited in claim 16, wherein in executing a synchronization operation the first agent of the multi-chip system is configured to:

execute a first synchronization operation if the first agent and the second agent reside in the same chip device of the multi-chip system; and

execute a second synchronization operation if the first agent and the second agent reside in different chip devices of the multi-chip system.

18. The multi-chip system as recited in claim 15, wherein the queue and the I/O device reside in a same chip device of the multi-chip system.

19. The multi-chip system as recited in claim 18, wherein the I/O device and the first agent reside in the same chip device of the multi-chip system, and wherein in initiating the first operation the first agent is further configured to send the first operation directly to the queue for ordering access to the I/O device.

20. The multi-chip system as recited in claim 18, wherein the I/O device and the queue reside in a first chip device, the first agent resides in a second chip device of the multi-chip system, and wherein in initiating the first operation, the first agent is further configured to:

send the first operation to an other queue residing in the second chip device, the first operation is forwarded from the other queue in the second chip device to the queue residing in the first chip device through an inter-chip interconnect interface.

21. The multi-chip system as recited in claim 20, wherein the first agent is further configured to receive an acknowledgement of queuing the first operation from the first chip device, and wherein providing the indication of queuing the first operation is in response to receiving the acknowledgement.

22. The multi-chip system as recited in claim 18, wherein the second agent and the I/O device reside in the same chip device of the multi-chip system, and wherein in initiating the second operation, the second agent is further configured to send the second operation directly to the queue.

23. The multi-chip system as recited in claim 18, wherein the I/O device and the queue reside in a first chip device, the first agent resides in a second chip device of the multi-chip system, and wherein in initiating the second operation, the second agent is further configured to:

send the second operation to an other queue in the second chip device, the second operation is forwarded from the other queue in the second chip device to the queue residing in the first chip device through an inter-chip interconnect interface.

24. The multi-chip system as recited in claim 15, wherein the I/O device resides on a first chip device of the multi-chip system, and both the first agent and the second agent reside on a second chip of the multi-chip system, the second chip device is configured provide an indication of synchronizing access to the I/O device locally within the second chip device.

25. The multi-chip system as recited in claim 24, wherein the queue resides on the second chip device.

26. The multi-chip system as recited in claim 15, wherein in providing an indication of queuing the first operation, the first agent is further configured to modify a flag value.

27. The multi-chip system as recited in claim 15, wherein the first or second operation is a store operation for writing data to the I/O device.

28. The multi-chip system as recited in claim 15, wherein the first agent or the second agent is a core processor of the multi-chip system.