Method and apparatus for communicating over a resource interconnect

• US 7,664,897 B2
• Filed: 12/01/2005
• Issued: 02/16/2010
• Est. Priority Date: 02/08/2005
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Eatherton, William, Fingerhut, John A., Steiss, Donald, Cohen, Earl T., Williams, John Jr.
• Primary Examiner(s): Rinehart; Mark
• Assistant Examiner(s): Cerullo; Jeremy S
• Application Number:
  US11/293,285
• Publication Number:
  US 20060179204A1
• Time in Patent Office: 1,538 Days
• Field of Search: 710/305, 710/105, 710/317, 370/351, 370/352
• US Class Current: 710/105
• CPC Class Codes: H04L 45/60 Router architectures
• CPC Class Codes: H04L 47/50 Queue scheduling
• CPC Class Codes: H04L 47/56 implementing delay-aware scheduling

Abstract

A resource interconnect architecture and associated descriptor protocol provides more efficient communication between different resources in a data processing system. One embodiment uses a backdoor interconnect that allows some resources to communicate without using a central resource interconnect. Another embodiment uses nested descriptors that allow operations by different resources to be chained together without having to communicate back to an originating descriptor resource. In another embodiment, the descriptors are generated in hardware or in software. Other embodiments assign priority or privilege values to the descriptors that optimize processing and error handling performance.

Claims

1. A data processing system, comprising:

a first interconnect coupled between multiple different resources and configured to be shared between the multiple different resources for sending and receiving descriptors, wherein the descriptors are used by the different resources for communicating across the first interconnect, the descriptors including a destination identifier configured to identify one of the resources receiving the descriptors sent on the first interconnect and a source identifier configured to identify one of the resources sending the descriptors on the first interconnect;

a second interconnect separate from the first interconnect and separately coupled from the first interconnect between the multiple different resources, wherein the second interconnect is configured to be shared between the multiple different resources for sending and receiving the descriptors independently of the first interconnect; and

a processor configured to send a request in a first one of the descriptors to a first resource from among the multiple different resources over the first interconnect, wherein the first resource is configured to receive the request via the first interconnect and communicate over the second interconnect with a second resource from among the multiple different resources using a second one of the descriptors, wherein the second resource is configured to respond back to the processor using a third one of the descriptors over one of the first or second interconnect pursuant to the request.

2. The data processing system according to claim 1 wherein the the first and second interconnect each comprise an interconnection mesh that selectively connecting the different resources together according to the destination identifier or source identifier in the descriptors.

3. The data processing system according to claim 1 wherein the first interconnect and the second interconnect are part of the same resource interconnect circuit.

4. The data processing system according to claim 1 wherein the first interconnect is part of a central resource interconnect and the second interconnect is part of a backdoor interconnect that is configured to operate independently of the central resource interconnect.

5. The data processing system according to claim 4 wherein the backdoor interconnect is configured to operate in parallel with the central resource interconnect so that two different messages can be sent and two different messages can be received by the same processor or resource at the same time.

6. The data processing system according to claim 1 wherein at least one of the first and second interconnect are configured to carry messages on sub-interconnect boundaries.

7. The data processing system according to claim 1 wherein the first resource is configured to use the descriptors for communicating over both the first and second interconnect.

8. The data processing system according to claim 7 wherein the processor includes descriptor hardware logic configured to generate the descriptors in hardware and descriptor software logic configured to generate the descriptors in software.

9. The data processing system according to claim 8 wherein the processor is configured to generate the descriptors in hardware when there is a cache miss and then send the hardware generated descriptors to a memory resource.

10. The data processing system according to claim 8 wherein the processor includes a descriptor interface configured to direct hardware identified descriptor responses to the descriptor hardware logic and direct received software identified descriptor responses to the descriptor software logic.

11. The data processing system according to claim 7 wherein at least one of the first resource or second resource is configured to conduct an operation for at least one of the group including failure processing and data processing according to descriptor privilege identifiers.

12. The data processing system according to claim 11 wherein at least one of the first resource or second resource is configured to return a descriptor response back to the processor indicating success or failure of the operation, the processor also configured to then conduct an operation associated with the descriptor response.

13. The data processing system according to claim 7 wherein the descriptors vary in size according to an associated descriptor command.

14. The data processing system according to claim 1 wherein at least one of the processor, first resource, second resource, first interconnect, or second interconnect is configured to prioritize request processing according to priority identifiers in the request.

15. The data processing system according to claim 1 wherein at least one of the processor, first resource, or second resource includes a descriptor request-receive interconnect for receiving descriptor requests and a separate descriptor response-send interconnect for sending responses to the descriptor requests.

16. The data processing system according to claim 1 wherein at least one of the processor, first resource, or second resource includes a descriptor request-send interconnect for sending descriptor requests and a separate descriptor response-receive interconnect for receiving responses to the descriptor requests.

17. The data processing system according to claim 1 including:

a nested descriptor that includes a first descriptor that causes the first resource to execute at least a portion of the nested descriptor and then forward results and/or status to the second resource,

the nested descriptor including a second descriptor separate from the first descriptor that is executed by the second resource separately from the first resource and configured to cause the second resource to continue execution of the nested descriptor and then forward subsequent results and/or status in a response descriptor back to the processor.

18. A method for operating a data processing device, comprising:

generating descriptor requests including source identifiers, destination identifiers and commands, wherein at least one of the descriptor requests is a nested descriptor comprising;

a first descriptor located within the nested descriptor that has a first destination identifier identifying a primary resource and an associated first command configured to be executed by the primary resource; and

a second separate descriptor located within the same nested descriptor that has a second destination identifier identifying a secondary resource and an associated second command configured to be executed by the secondary resource;

using the first destination identifier to send the nested descriptor over a resource interconnect to the primary resource, the primary resource executing the first command in the nested descriptor;

using the second destination identifier to send the nested descriptor from the primary resource to the secondary resource, wherein the secondary resource executes the second command in the nested descriptor; and

using a source identifier in the nested descriptor to send a response descriptor from the secondary resource over the resource interconnect to a third one of the resources.

19. The method according to claim 18 including sending the nested descriptor from the primary resource to the secondary resource over a backdoor interconnect that is connected to the primary and secondary resource independently of the resource interconnect.

20. The method according to claim 18 including:

generating sub-unit identifiers in the descriptor requests for identifying processor sub-units associated with the descriptor requests; and

using the sub-unit identifiers to track outstanding processor descriptor requests generated by the processor sub-units.

21. The method according to claim 20 including:

using the sub-unit identifiers to identify descriptor requests generated either by hardware or software;

sending the descriptor requests to the resources;

receiving descriptor responses to the descriptor requests that include the hardware/software identifiers from the descriptor requests; and

processing the descriptor responses either in hardware or software according to the received hardware/software identifiers.

22. The method according to claim 18 including:

receiving descriptor requests that include privilege identifiers;

detecting privilege violations or execution failures executing the received descriptor requests; and

generating failure notifications according to the privilege identifiers.

23. The method according to claim 22 including:

receiving a first descriptor request having a high privilege identifier;

receiving a second descriptor request having a low privilege identifier;

generating a fatal error notice when execution of the first descriptor request fails; and

generating a non-fatal error notice when execution of the second descriptor request fails.

24. The method according to claim 18 including:

receiving descriptor requests that include associated priority identifiers; and

processing the descriptor requests according to the associated priority identifiers.

25. The method according to claim 18 including:

sending a first descriptor starting from a first sub-interconnect boundary on a interconnect; and

sending a second descriptor starting from a second sub-interconnect boundary on the same interconnect.

26. The method according to claim 18 wherein resources connected to the resource interconnect perform one or more of the following:

sending descriptor requests on a first set of interconnects;

sending descriptor responses on a second set of interconnects;

receiving descriptor requests on a third set of interconnects; and

receiving descriptor responses on a fourth set of interconnects.

27. An apparatus, comprising:

a first interconnect coupled between multiple different resources and configured to be shared between the multiple different resources for sending and receiving messages;

a second interconnect separately coupled between the same multiple different resources, wherein the second interconnect is configured to be shared between the multiple different resources for sending and receiving messages independently of the messages sent and received over the first interconnect; and

a processor configured to send a first message to a first one of the resources over the first interconnect, wherein;

the first resource is configured to both receive the first message over the first interconnect and send a second message to a second one of the resources over the second interconnect at the same time, and

the second resource is configured to process the second message received over the second interconnect and send a third message back to the processor over the first interconnect responsive to the second message received from the first resource over the second interconnect.

28. The apparatus of claim 27 wherein the first and second interconnect are both cross-bar switches.

29. The apparatus of claim 27, wherein the first and second message are both part of a same nested descriptor comprising:

a first descriptor located within the nested descriptor that has a first destination identifier identifying the first resource and an associated first command configured to be executed by the first resource;

a second descriptor located within the same nested descriptor that has a second destination identifier identifying the second resource and an associated second command configured to be executed by the second resource; and

a source identifier located within the same nested descriptor identifying the processor as a destination for the third message.