Descriptor-based load balancing

• US 20050080952A1
• Filed: 10/14/2003
• Published: 04/14/2005
• Est. Priority Date: 10/14/2003
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Dion, Jeremy, Oner, Koray
• Granted Patent: US 6,981,074 B2
• Time in Patent Office: Days
• US Class Current: 710/52
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 13/385 for adaptation of a particular data processing system to different peripheral devices
• CPC Class Codes: G06F 2213/0038 System on Chip

Abstract

A multiprocessor switching device substantially implemented on a single CMOS integrated circuit is described in connection with a descriptor-based packet processing mechanism for use in efficiently assigning and processing packets to a plurality of processors. A plurality of descriptors associated with each packet transfer are written back to memory in order, divided into subset groups and assigned to processors, where each processor searches the assigned subset for EOP and associated SOP descriptors to process.

Claims

1. An integrated circuit multiprocessor apparatus for processing a plurality of data packets, comprising:

a plurality of processors;

an interface circuit for receiving and transmitting a plurality of data packets;

a memory comprising a plurality of descriptors, each of said plurality of descriptors comprising an ownership indication, a start of packet indication, an end of packet indication, a buffer length value and a buffer address for specifying a location of a buffer in memory for storing at least a portion of a data packet;

a packet manager circuit coupled between the interface circuit and the memory to transfer data packets between the interface circuit and memory, wherein the packet manager circuit is configured to transfer a first data packet under control of at least a first descriptor and to transfer a second data packet under control of at least a second descriptor, wherein said packet manager is configured to write back to memory all descriptors associated with a data packet upon completion of the transfer of said data packet and said descriptors are written back in the order in which they are released.

2. The apparatus recited in claim 1, wherein the each data packet is associated with a descriptor ring comprising a plurality of descriptors, each of the plurality of descriptors identifying a memory buffer in the memory for storing at least a portion of a data packet.

3. The apparatus recited in claim 1, wherein the packet manager circuit comprises a packet manager input circuit coupled between the interface circuit and the memory to transfer data packets from the interface circuit to memory, wherein the packet manager input is configured to transfer a first data packet to memory only after the at least a first descriptor is read that has its ownership indication set.

4. The apparatus recited in claim 1, wherein said packet manager is configured to reset the ownership indication in each descriptor after completing transfer of data under said descriptor.

5. The apparatus recited in claim 1, wherein the at least a first descriptor comprises:

a start descriptor having its start of packet indication set, where said start descriptor identifies a memory buffer where the start of the first data packet is stored; and

a last descriptor having its end of packet indication set, where said last descriptor identifies a memory buffer where the end of the first data packet is stored.

6. The apparatus recited in claim 5, wherein the at least a first descriptor comprises at least one middle descriptor having both its start of packet indication and end of packet indication reset.

7. The apparatus recited in claim 1, wherein the at least a first descriptor has its start of packet indication and end of packet indication set.

8. The apparatus recited in claim 1, further comprising a load balancing processor for distributing the plurality of data packets to the plurality of processors for processing by assigning a first consecutive group of the plurality of descriptors to a first processor and a second consecutive group of the plurality of descriptors to a second processor, wherein the first processor snoops on a last descriptor in the first consecutive group of descriptors for said first processor to detect when the last descriptor has its ownership indication reset by the packet manager circuit.

9. The apparatus recited in claim 8, wherein the first processor processes at least a first claimed data packet by scanning back through the first consecutive group of descriptors, starting with the last descriptor having its ownership indication reset, to identify any descriptor having an end of packet indication set, and then scanning back further through the first consecutive group of descriptors to identify any packet start descriptor having a start of packet indication set, said at least first claimed data packet comprising the packet start descriptor and the last descriptor.

10. The apparatus recited in claim 9, wherein the first processor processes a claimed data packet by further scanning back through the plurality of descriptors to identify a packet start descriptor if said packet start descriptor is not included in the first consecutive group of descriptors assigned to the first processor.

11. A system for controlling distribution of a plurality of data packets to a plurality of processors, said system comprising:

a plurality of consecutive descriptor entries;

a DMA controller for reading the plurality of consecutive descriptor entries;

storing data packets in a plurality of buffers, each of the plurality of consecutive descriptor entries specifying a buffer for storing at least a portion of a data packet; and

releasing ownership of the plurality of consecutive descriptor entries in order upon completing storage of a data packet;

a plurality of processors, each processor configured to process packets; and

an assignment processor for assigning a first group of descriptors from the plurality of consecutive descriptor entries to a first processor and for assigning a second group of descriptors from the plurality of consecutive descriptor entries to a second processor;

where each processor in the plurality of processors is configured to spin on the final descriptor in its assigned group of descriptors and to process any packet comprising any EOP descriptors contained with its assigned group of descriptors along with any other descriptors associated with each EOP descriptor.

12. The system recited in claim 11, wherein each of said plurality of consecutive descriptor entries comprises an ownership indication, a start of packet indication, an end of packet indication, a buffer length value and a buffer address for specifying a location of a buffer in memory for storing at least a portion of a data packet.

13. The system recited in claim 12, wherein an EOP descriptor comprises a descriptor entry having an end of packet indication set.

14. The system recited in claim 11, wherein the descriptors associated with each EOP descriptor are determined by scanning back through the assigned group of descriptors to identify an SOP descriptor.

15. The system recited in claim 11, wherein the descriptors associated with each EOP descriptor are determined by scanning back through the plurality of consecutive descriptor entries, starting with its assigned group of descriptors, to identify an SOP descriptor if said an SOP descriptor is not included in the assigned group of descriptors for said processor.

16. The system recited in claim 12, wherein each processor in the plurality of processors determines the length of an assigned packet by summing the buffer length values for each descriptor associated with the assigned packet so as to support an unlimited packet size.

17. A method comprising:

creating and releasing a plurality of descriptors to a packet manager, said plurality of descriptors comprising a first descriptor group and a second descriptor group;

assigning a first subset of the plurality of descriptors to a first processor and a second subset of the plurality of descriptors to a second processor;

receiving a first packet in an interface circuit and transferring the first packet to at least a first memory buffer under control of the first descriptor group;

receiving a second packet in the interface circuit and transferring the second packet to at least a second memory buffer under control of the second descriptor group;

releasing the first descriptor group to software by writing the first descriptor group to memory;

releasing the second descriptor group to software by writing the second descriptor group to memory; and

at each processor, identifying any EOP descriptor in the assigned subset of the plurality of descriptors and processing any descriptor associated with said EOP descriptor.

18. The method recited in claim 17, wherein each processor identifies any EOP descriptor in the subset of the plurality of descriptors assigned to said processor by snooping on a last descriptor from the assigned subset of the plurality of descriptors and scanning back through said assigned subset of the plurality of descriptors to identify any EOP descriptor and any descriptor associated with said EOP descriptor.

19. The method recited in claim 18, wherein any descriptor associated with said EOP descriptor comprises said identified EOP descriptor and SOP descriptor.

20. The method of claim 17, wherein said first descriptor group comprises at least one descriptor, where said at least one descriptor comprises an ownership bit, a start of packet bit, an end of packet bit, a buffer length value and a buffer address for specifying a location of a buffer in memory for storing at least a portion of a data packet.

21. The method of claim 17, wherein a processor identifies a first descriptor as an EOP descriptor by checking to see if an EOP bit is set in the first descriptor.

22. The method of claim 17, wherein a processor identifies a first descriptor as an SOP descriptor by checking to see if an SOP bit is set in the first descriptor.

23. The method of claim 17, wherein a processor determines that a first descriptor contains an entire packet by checking to see if both and SOP bit and an EOP bit are set in the first descriptor.