Method and apparatus for a multi-gigabit ethernet architecture

• US 6,873,630 B1
• Filed: 05/19/1999
• Issued: 03/29/2005
• Est. Priority Date: 05/19/1999
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Muller, Shimon, Hendel, Ariel
• Primary Examiner(s): VANDERPUYE, KENNETH N
• Application Number:
  US09/314,782
• Time in Patent Office: 2,141 Days
• Field of Search: 370/352, 370/355, 370/356, 370/358, 370/474, 370/536, 370/542, 370/401
• US Class Current: 370/356
• CPC Class Codes: H04L 69/14 Multichannel or multilink protocols
• CPC Class Codes: H04L 69/323 in the physical layer [OSI layer 1]
• CPC Class Codes: H04L 9/40 Network security protocols

Abstract

An Ethernet architecture is provided for connecting a computer system or other network entity to a dedicated Ethernet network medium. The network interface enables the transmission and receipt of data by striping individual Ethernet frames across a plurality of logical channels and may thus operate at substantially the sum of the individual channel rates. Each channel may be conveyed by a separate conductor (e.g., in a bundle) or the channels may be carried simultaneously on a shared medium (e.g., an electrical or optical conductor that employs a form of multiplexing). On a sending station, a distributor within the sender'"'"'s network interface receives Ethernet frames (e.g., from a MAC) and distributes frame bytes in a round-robin fashion on the plurality of channels. Each “mini-frame” is separately framed and encoded for transmission across its channel. On a receiving station, the receiver'"'"'s network interface includes a collector for collecting the multiple mini-frames (e.g., after decoding) and reconstructing the frame'"'"'s byte stream (e.g., for transfer to the receiver'"'"'s MAC). The first and last bytes of each frame and mini-frame are marked for ease of recognition. Multiple unique idle symbols may be employed for transmission during inter-packet gaps to facilitate the collector'"'"'s synchronization of the multiple channels and/or enhance error detection. A maximum channel skew is specified, and each received channel may be buffered with an elasticity that is proportional to the maximum skew so that significant propagation delay may be encountered between channels without disrupting communications.

Claims

1. A method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, comprising:

receiving a communication from a process operating on a first network entity, wherein the communication is directed to a second network entity;

receiving said communication at a distribution module of a network interface device from a medium access control module across a first interface, wherein said distribution module is configured to distribute portions of said communication among a plurality of communication channels;

distributing elements of said communication into multiple portions, each said portion corresponding to one of a plurality of channels established to convey a communication between said first network entity and said second network entity;

encoding a first portion of said communication with a first starting delimiter;

encoding a second portion of said communication with a second starting delimiter, wherein said second starting delimiter is different from said first starting delimiter;

sending said first portion of said communication on a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

sending said second portion of said communication on a second channel established on a second communication medium coupled to said first network entity and said second network entity;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second.

2. The method of claim 1, wherein said communication is an Ethernet frame and wherein each of said multiple portions of said communication comprises one or more bytes.

36. The method of claim 1, wherein said first interface is configured to convey said communication at a data rate exceeding one gigabit per second.

37. The method of claim 1, in which said sending a first portion of said communication comprises forwarding an apportionment of said communication elements to a first physical coding module across a second interface;

and

wherein said first physical coding module is configured to perform said encoding of said first portion of said communication.

38. The method of claim 37, wherein said first physical coding module:

encodes a first element of said apportionment with a first start code if said first element is the first element of said communication and otherwise encodes said first element of said apportionment with a second start code; and

encodes a last element of said apportionment with a first end code if said last element is the last element of said communication and otherwise encodes said last element of said apportionment with a second end code.

40. The method of claim 1, further comprising:

transmitting a first idle signal on said first channel and said second channel prior to said receiving; and

transmitting a second idle signal on said first channel and said second channel after said sending said second portion of said communication;

wherein said second idle signal is different from said first idle signal.

41. The method of claim 1, further comprising:

encoding said first portion of said communication with a first ending delimiter; and

encoding said second portion of said communication with a second ending delimiter;

wherein said second ending delimiter is different from said first ending delimiter.

119. The method of claim 1, wherein said first portion of said communication includes the initial byte of said communication.

120. The method of claim 1, wherein said second portion of said communication does not include the initial byte of said communication.

121. The method of claim 1, wherein said encoding a second portion of said communication comprises:

encoding said second portion of said communication with a first end code if said second portion of said communication includes the final byte of said communication.

122. The method of claim 121, wherein said encoding a second portion of said communication further comprises:

encoding said second portion of said communication with a second end code, different from said first end code, if said second portion of said communication does not include the final byte of said communication.

3. A method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, comprising:

receiving a communication at a distribution module of a network interface device from a medium access control module across a first interface, wherein said distribution module is configured to distribute portions of said communication among a plurality of communication channels;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

sending a second portion of said communication on a second channel established on a second communication medium coupled to said first network entity and said second network entity.

4. The method of claim 3, wherein said first interface is configured to convey said communication at a data rate exceeding one gigabit per second.

5. The method of claim 3, in which said sending a first portion of said communication comprises forwarding an apportionment of said communication elements to a first physical coding module across a second interface;

and

wherein said first physical coding module is configured to encode said apportionment of communication elements into a series of codes for transmission over said first communication medium.

6. The method of claim 5, wherein said first physical coding module:

encodes a first element of said apportionment with a first start code if said first element is the first element of said communication and otherwise encodes said first element of said apportionment with a second start code; and

encodes a last element of said apportionment with a first end code if said last element is the last element of said communication and otherwise encodes said last element of said apportionment with a second end code.

7. The method of claim 5, wherein said second interface is configured to convey said first apportionment at a data rate exceeding one gigabit per second.

8. The method of claim 3, in which said distributing comprises allotting elements of said communication among a plurality of channels established to convey a communication between said first network entity and said second network entity.

9. The method of claim 8, wherein each of said channels is configured to traverse a separate physical communication link.

10. The method of claim 8, wherein each of said channels is configured to traverse a common physical communication link, said common physical communication link comprising said first communication medium and said second communication medium.

11. The method of claim 3, wherein:

one of said first portion of said communication and said second portion of said communication includes a first start symbol configured to indicate a start of said communication and the other of said first portion and said second portion includes a second start symbol configured to indicate a start of a portion of said communication; and

one of said first portion of said communication and said second portion of said communication includes a first end symbol configured to indicate an end of said communication and the other of said first portion and said second portion includes a second end symbol configured to indicate an end of a portion of said communication.

12. The method of claim 3, further comprising:

transmitting a first idle signal on said first channel and said second channel prior to said receiving; and

transmitting a different idle signal on said first channel and said second channel after said sending a second portion of said communication.

13. The method of claim 3, further comprising:

encoding the first element of said first portion of said communication with a first starting delimiter; and

encoding the first element of said second portion of said communication with a second starting delimiter.

14. The method of claim 13, further comprising:

encoding the last element of said first portion of said communication with a first ending delimiter; and

encoding the last element of said second portion of said communication with a second ending delimiter.

43. The method of claim 3, wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second.

44. The method of claim 3, wherein said communication is an Ethernet frame and wherein each of said multiple portions of said communication comprises one or more bytes.

113. The method of claim 3, wherein said sending a first portion of said communication comprises:

encoding said first portion with a first start code if said first portion includes the initial element of said communication.

114. The method of claim 113, wherein said sending a first portion of said communication further comprises:

encoding said first portion without a start code if said first portion does not include said initial element of said communication.

115. The method of claim 113, wherein said sending a first portion of said communication further comprises:

encoding said first portion with a second start code, different from said first start code, if said first portion does not include said initial element of said communication.

116. The method of claim 3, wherein said sending a second portion of said communication comprises:

encoding said second portion with a first end code if said second portion includes the final element of said communication.

117. The method of claim 116, wherein said sending a second portion of said communication further comprises:

encoding said second portion without an end code if said second portion does not include said final element of said communication.

118. The method of claim 116, wherein said sending a second portion of said communication further comprises:

encoding said second portion with a second end code, different from said first end code, if said second portion does not include said final element of said communication.

15. A method of receiving a communication at a second network entity from a first network entity, wherein the first network entity and the second network entity are coupled to a dedicated communication medium, comprising:

receiving at a second network entity a first idle code on each of multiple channels established between a first network and said second network entity;

receiving at said second network entity a first portion of a communication from said first network entity on a first channel of said multiple channels;

receiving at said second network entity a second portion of said communication on a second channel of said multiple channels;

collecting an element of said first portion and an element of said second portion;

receiving at said second network entity a second idle code, different from said first idle code, on each of said multiple channels; and

forwarding said communication toward a process operating on said second network entity.

16. The method of claim 15, wherein said communication is an Ethernet frame.

17. The method of claim 16, wherein said first portion of a communication comprises:

a first start signal configured to indicate a beginning of said communication; and

a first set of elements of said communication.

18. The method of claim 17, wherein said second portion of a communication comprises:

a second start signal configured to indicate a beginning of a portion of said communication, said second start signal differing from said first start signal; and

a second set of elements of said communication.

19. The method of claim 15, wherein said first communication channel and said second communication channel traverse a common communication medium.

20. The method of claim 15, herein said first communication channel and said second communication channel traverse separate physical mediums.

21. The method of claim 15, in which said collecting comprises:

receiving at a collection module an element of said first communication portion and an element of said second communication portion; and

combining said element of said first communication portion and said element of said second communication portion.

46. The method of claim 21, wherein said forwarding comprises sending said combined elements to a medium access control module across a first interface toward a process operating on said second network entity.

47. The method of claim 46, wherein said first interface is configured to convey said combined elements at a data rate greater than one gigabit per second.

22. A method of receiving a communication at a second network entity from a first network entity, wherein the first network entity and the second network entity are coupled to a dedicated communication medium, comprising:

receiving at a second network entity a first portion of a communication from a first network entity on a first channel established between said first network entity and said second network entity;

receiving at said second network entity a second portion of said communication on a second channel established between said first network entity and said second network entity;

receiving at a collection module an element of said first communication portion and an element of said second communication portion;

combining said element of said first communication portion and said element of said second communication portion; and

sending said combined elements to a medium access control module across a first interface toward a process operating on said second network entity.

23. The method of claim 22, wherein said first interface is configured to convey said combined elements at a data rate greater than one gigabit per second.

24. The method of claim 22, further comprising:

receiving a first idle code on each of said first channel and said second channel prior to said receiving a first portion of a communication; and

receiving a second idle code on each of said first channel and said second channel after said receiving a second portion of said communication.

49. The method of claim 22, wherein said communication is an Ethernet frame.

50. The method of claim 22, wherein said first portion of a communication comprises:

a first start signal configured to indicate a beginning of said communication; and

a first set of elements of said communication.

51. The method of claim 50, wherein said second portion of a communication comprises:

a second start signal configured to indicate a beginning of a portion of said communication, said second start signal differing from said first start signal; and

a second set of elements of said communication.

25. A method of receiving a communication from a first network entity at a second network entity across a plurality of channels, comprising:

receiving synchronization information across each of a plurality of channels coupling a first network entity to a second network entity;

receiving at said second network entity a set of bytes across each of said channels;

detecting a first byte and a last byte in each of said sets of bytes;

decoding each of said sets of bytes; and

re-assembling said sets of bytes into a stream of bytes of a communication directed from said first network entity to said second network entity.

26. The method of claim 25, in which:

said receiving synchronization information comprises receiving a first idle code on each of said channels; and

wherein said method further comprises receiving a second idle code on each of said channels after said receiving a set of bytes across each of said channels.

53. The method of claim 25, wherein:

the communication is a packet; and

said receiving a set of bytes comprises receiving across each said channel a mini-frame comprising a portion of the packet.

54. The method of claim 53, wherein said detecting comprises:

on a first of said channels, identifying a start of packet delimiter; and

on the other channels of said channels, identifying a start of mini-frame delimiter.

55. The method of claim 53, wherein said detecting comprises:

on a first of said channels, identifying an end of packet delimiter; and

on the other channels of said channels, identifying an end of mini-frame delimiter.

56. The method of claim 53, wherein:

said re-assembling comprises merging said mini-frames to re-form the packet; and

the method further comprises forwarding the packet toward a medium access control module.

57. The method of claim 25, wherein each said set of bytes is received at a data rate exceeding one gigabit per second.

58. The method of claim 25, wherein said decoding comprises:

at a physical coding module coupled to each of said channels, decoding a set of bytes from codes received over said channel.

109. The method of claim 25, wherein said decoding comprises:

decoding a first start code in a one of said sets of bytes, said one set of bytes containing an initial byte of the communication;

wherein no other set of bytes includes a start code.

110. The method of claim 25, wherein said decoding comprises:

decoding a first start code in one of said sets of bytes, said one set of bytes containing an initial byte of the communication; and

decoding a second start code in a different set of bytes, said second start code differing from said first start code.

111. The method of claim 25, wherein said decoding comprises:

decoding a first end code in one of said sets of bytes, said one set of bytes containing a final byte of the communication;

wherein no other set of bytes includes an end code.

112. The method of claim 25, wherein said decoding comprises:

decoding a first end code in one of said sets of bytes, said one set of bytes containing a final byte of the communication; and

decoding a second end code in a different set of bytes, said second end code differing from said first end code.

27. A method of operating a computer to communicate with a network entity, comprising:

operating a medium access control module configured to communicate a first frame from a computer system to a network entity and receive a second frame at said computer system from said network entity;

operating a distribution module to apportion contents of said first frame among a plurality of communication channels coupling said computer system to said network entity through one or more communication links; and

operating a collection module to combine contents of said second frame received through said plurality of communication channels;

wherein said distribution module and said collection module interface with each of said communication channels at a rate exceeding one gigabit per second; and

wherein said medium access control module interfaces with said distribution module and said collection module at a rate substantially equal to the sum of said rates at which said communication channels interface with said distribution module and said collection module.

28. The method of claim 27, further comprising:

operating a physical medium module configured to encode said first frame contents for transmission over said communication channels and decode said second frame contents received over said communication channels.

29. The method of claim 27, wherein said first frame is a communication frame configured for transmission over a network compatible with an Ethernet communication protocol.

39. The method of claim 27, wherein said second interface is configured to convey said apportionment at a data rate exceeding one gigabit per second.

60. The method of claim 27, wherein said distribution module apportions said contents of said first frame by:

receiving a portion of said first frame from said medium access control module; and

distributing said portion of said first frame among said plurality of communication channels in round robin order.

61. The method of claim 27, wherein said collection module combines said contents of said second frame by:

merging, in round robin order, segments of said second frame received from said plurality of communication channels; and

forwarding said merged segments to said medium access control module.

30. A network interface device for coupling a computer system to a network, comprising:

a medium access control module configured to communicate with an application executing on a computer system;

multiple physical coding modules, wherein each said physical coding module is configured to encode packet bytes for transmission on a network medium and decode encoded bytes received from said network medium, and wherein said network medium is configured to carry said bytes between said computer system and a network entity;

a distributor configured to accept a first packet from said medium access control module and divide said first packet into a first plurality of packet bytes for transmission across said network medium; and

a collector configured to accept a second plurality of packet bytes from said multiple physical coding modules and combine said second plurality of packet bytes into a second packet for transfer to said medium access control module.

31. The network interface device of claim 30, further comprising a first set of interfaces coupling said multiple physical coding modules to said distributor and said collector, wherein each of said first set of interfaces is configured to operate at a rate exceeding one gigabit per second.

32. The network interface device of claim 31, further comprising a second interface coupling said distributor and said collector to said medium access control module, wherein said second interface is configured to operate at a rate approximately equal to the sum of said operation rates of said first set of interfaces.

33. The network interface device of claim 32, wherein said second interface is configured to operate at a data rate of approximately ten gigabits per second.

105. The network interface device of claim 30, wherein:

said multiple physical coding modules include a first physical coding module;

said first plurality of packet bytes includes a first subset of bytes of said first packet; and

said first physical coding module is configured to;

encode said first subset of bytes with a first start code if said first subset of bytes includes the initial byte of said first packet; and

otherwise, encode said first subset of bytes without a start code.

106. The network interface device of claim 30, wherein:

said multiple physical coding modules include a first physical coding module;

said first plurality of packet bytes includes a first subset of bytes of said first packet; and

said first physical coding module is configured to;

encode said first subset of bytes with a first start code if said first subset of bytes includes the initial byte of said first packet; and

otherwise, encode said first subset of bytes with a second start code different from said first start code.

107. The network interface device of claim 30, wherein:

said multiple physical coding modules include a first physical coding module;

said first plurality of packet bytes includes a last subset of bytes of said first packet; and

said first physical coding module is configured to;

encode said last subset of bytes with a first end code if said last subset of bytes includes the final byte of said first packet; and

otherwise, encode said last subset of bytes without an end code.

108. The network interface device of claim 30, wherein:

said multiple physical coding modules include a first physical coding module;

said first plurality of packet bytes includes a last subset of bytes of said first packet; and

said first physical coding module is configured to;

encode said last subset of bytes with a first end code if said last subset of bytes includes the final byte of said first packet; and

otherwise, encode said last subset of bytes with a second end code different from said first end code.

34. A device for implementing an Ethernet protocol to communicate Ethernet frames between a first network entity and a second network entity, comprising:

a distributor configured to distribute bytes of a first Ethernet frame over a plurality of channels in a first order;

a collector configured to receive bytes of a second Ethernet frame over said channels in a second order;

a first interface coupling said distributor and said collector to a medium access control module at a data rate exceeding one gigabit per second, wherein data are transferred across said first interface in multi-byte units in synchronization with both edges of a clock signal; and

a second interface coupling said distributor and said collector to a first physical coding module at a data rate exceeding one gigabit per second in synchronization with both edges of a second clock signal.

35. The device of claim 34, wherein said first order and said second order are round robin.

90. The device of claim 34, wherein said distributor distributes a substantially equivalent number of bytes of the first Ethernet frame over each channel in said plurality of channels.

91. The device of claim 34, wherein said collector receives a substantially equivalent number of bytes of the second Ethernet frame over each channel in said plurality of channels.

92. The device of claim 34, further comprising a separate physical coding module for each channel in said plurality of channels, including said first physical coding module corresponding to a first channel.

93. The device of claim 92, wherein each said physical coding module is configured to encode the bytes of the first Ethernet frame that are distributed over the corresponding channel in said plurality of channels.

94. The device of claim 34, wherein:

a first idle is distributed over each channel in said plurality of channels before distributing the bytes of the first Ethernet frame; and

a second idle, different from said first idle, is distributed over each channel in said plurality of channels after distributing the bytes of the first Ethernet frame.

95. The device of claim 34, wherein a first set of bytes of the first Ethernet frame that are distributed over a first channel in said plurality of channels are preceded by a start code if said first set of bytes includes the initial byte of the first Ethernet frame.

96. The device of claim 34, wherein bytes of the first Ethernet frame that are distributed over each channel in said plurality of channels are preceded by a start code.

97. The device of claim 96, wherein said start code is different for a first channel in said plurality of channels than for one or more other channels in said plurality of channels.

98. The device of claim 34, wherein a last set of bytes of the first Ethernet frame that are distributed over a last channel in said plurality of channels are followed by an end code if said last set of bytes includes the final byte of the first Ethernet frame.

99. The device of claim 34, wherein bytes of the first Ethernet frame that are distributed over each channel in said plurality of channels are followed by an end code.

100. The device of claim 99, wherein said end code is different for a first channel in said plurality of channels than for the other channels in said plurality of channels.

101. The device of claim 34, wherein each channel in said plurality of channel traverses a separate physical communication link between the first network entity and the second network entity.

102. The device of claim 34, wherein each channel in said plurality of channel traverses a common physical communication link between the first network entity and the second network entity.

103. The device of claim 34, wherein said collector is further configured to:

combine the bytes to produce the second Ethernet frame; and

forward the second Ethernet frame to the medium access control module.

104. The device of claim 34, wherein said clock signal and said second clock signal are the same clock signal.

42. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of transmitting a communication from a first network entity to a second network entity, the method comprising:

receiving a communication at a distribution module of a network interface device from a medium access control module across a first interface, wherein said distribution module is configured to distribute portions of said communication among a plurality of communication channels;

distributing elements of said communication into multiple portions, each said portion corresponding to one of a plurality of channels established to convey a communication between said first network entity and said second network entity;

encoding a first portion of said communication with a first starting delimiter;

encoding a second portion of said communication with a second starting delimiter, wherein said second starting delimiter is different from said first starting delimiter;

sending said first portion of said communication on a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

sending said second portion of said communication on a second channel established on a second communication medium coupled to said first network entity and said second network entity;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second.

45. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of transmitting a communication from a first network entity to a second network entity, the method comprising:

receiving a communication at a distribution module of a network interface device from a medium access control module across a first interface, wherein said distribution module is configured to distribute portions of said communication among a plurality of communication channels;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

sending a second portion of said communication on a second channel established on a second communication medium coupled to said first network entity and said second network entity.

48. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of receiving a communication at a second network entity from a first network entity, the method comprising:

receiving at a second network entity a first idle code on each of multiple channels established between a first network and said second network entity;

receiving at said second network entity a first portion of a communication from said first network entity on a first channel of said multiple channels;

receiving at said second network entity a second portion of said communication on a second channel of said multiple channels;

collecting an element of said first portion and an element of said second portion;

receiving at said second network entity a second idle code, different from said first idle code, on each of said multiple channels; and

forwarding said communication toward a process operating on said second network entity.

52. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of receiving a communication at a second network entity from a first network entity, the method comprising:

receiving at a second network entity a first portion of a communication from a first network entity on a first channel established between said first network entity and said second network entity;

receiving at said second network entity a second portion of said communication on a second channel established between said first network entity and said second network entity;

receiving at a collection module an element of said first communication portion and an element of said second communication portion;

combining said element of said first communication portion and said element of said second communication portion; and

sending said combined elements to a medium access control module across a first interface toward a process operating on said second network entity.

59. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of receiving a communication from a first network entity at a second network entity across a plurality of channels, the method comprising:

receiving synchronization information across each of a plurality of channels coupling a first network entity to a second network entity;

receiving at said second network entity a set of bytes across each of said channels;

detecting a first byte and a last byte in each of said sets of bytes;

decoding each of said sets of bytes; and

re-assembling said sets of bytes into a stream of bytes of a communication directed from said first network entity to said second network entity.

62. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of operating a computer to communicate with a network entity, the method comprising:

operating a medium access control module configured to communicate a first frame from a computer system to a network entity and receive a second frame at said computer system from said network entity;

operating a distribution module to apportion contents of said first frame among a plurality of communication channels coupling said computer system to said network entity through one or more communication links; and

operating a collection module to combine contents of said second frame received through said plurality of communication channels;

wherein said distribution module and said collection module interface with each of said communication channels at a rate exceeding one gigabit per second; and

wherein said medium access control module interfaces with said distribution module and said collection module at a rate substantially equal to the sum of said rates at which said communication channels interface with said distribution module and said collection module.

63. A method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, comprising:

transmitting a first idle signal on a first channel and a second channel established on a first communication medium coupled to a first network entity and a second network entity;

receiving a communication from a process operating on said first network entity, wherein the communication is directed to said second network entity;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on said first channel;

sending a second portion of said communication on said second channel; and

transmitting a second idle signal on said first channel and said second channel after said sending said second portion of said communication, wherein said second idle signal is different from said first idle signal;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second;

wherein one of said first portion of said communication and said second portion of said communication includes a first start symbol configured to indicate a start of said communication and the other of said first portion and said second portion includes a second start symbol configured to indicate a start of a portion of said communication; and

wherein one of said first portion of said communication and said second portion of said communication includes a first end symbol configured to indicate an end of said communication and the other of said first portion and said second portion includes a second end symbol configured to indicate an end of a portion of said communication.

64. The method of claim 63, wherein:

said receiving comprises receiving a communication at a distribution module of a network interface device from a medium access control module across a first interface; and

said distribution module is configured to distribute portions of said communication among a plurality of communication channels, including said first channel and said second channel.

65. The method of claim 64, wherein said first interface is configured to convey said communication at a data rate exceeding one gigabit per second.

66. The method of claim 64, in which said sending a first portion of said communication comprises forwarding an apportionment of said communication elements to a first physical coding module across a second interface;

and

wherein said first physical coding module is configured to encode said apportionment of communication elements into a series of codes for transmission over said first communication medium.

67. The method of claim 66, wherein said first physical coding module:

encodes a first element of said apportionment with a first start code if said first element is the first element of said communication and otherwise encodes said first element of said apportionment with a second start code; and

encodes a last element of said apportionment with a first end code if said last element is the last element of said communication and otherwise encodes said last element of said apportionment with a second end code.

68. The method of claim 66, wherein said second interface is configured to convey said first apportionment at a data rate exceeding one gigabit per second.

69. The method of claim 63, in which said distributing comprises:

allotting elements of said communication among a plurality of channels established to convey a communication between said first network entity and said second network entity, including said first channel and said second channel.

70. The method of claim 63, further comprising:

encoding the first element of said first portion of said communication with a first starting delimiter; and

encoding the first element of said second portion of said communication with a second starting delimiter;

wherein said second starting delimiter is different from said first starting delimiter.

71. The method of claim 70, further comprising:

encoding the last element of said first portion of said communication with a first ending delimiter; and

encoding the last element of said second portion of said communication with a second ending delimiter;

wherein said second ending delimiter is different from said first ending delimiter.

72. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, the method comprising:

transmitting a first idle signal on a first channel and a second channel established on a first communication medium coupled to a first network entity and a second network entity;

receiving a communication from a process operating on said first network entity, wherein the communication is directed to said second network entity;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on said first channel;

sending a second portion of said communication on said second channel; and

transmitting a second idle signal on said first channel and said second channel after said sending said second portion of said communication, wherein said second idle signal is different from said first idle signal;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second;

wherein one of said first portion of said communication and said second portion of said communication includes a first start symbol configured to indicate a start of said communication and the other of said first portion and said second portion includes a second start symbol configured to indicate a start of a portion of said communication; and

wherein one of said first portion of said communication and said second portion of said communication includes a first end symbol configured to indicate an end of said communication and the other of said first portion and said second portion includes a second end symbol configured to indicate an end of a portion of said communication.

73. A method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, comprising:

transmitting a first idle signal on;

a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

a second channel established on a second communication medium coupled to said first network entity and said second network entity;

receiving a communication from a process operating on said first network entity, wherein the communication is directed to said second network entity;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on said first channel;

sending a second portion of said communication on said second channel; and

transmitting a second idle signal on said first channel and said second channel after said sending a second portion of said communication, wherein said second idle signal is different from said first idle signal;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second.

74. The method of claim 73, wherein:

said receiving comprises receiving a communication at a distribution module of a network interface device from a medium access control module across a first interface; and

said distribution module is configured to distribute portions of said communication among a plurality of communication channels, including said first channel and said second channel.

75. The method of claim 74, wherein said first interface is configured to convey said communication at a data rate exceeding one gigabit per second.

76. The method of claim 74, in which said sending a first portion of said communication comprises forwarding an apportionment of said communication elements to a first physical coding module across a second interface;

and

wherein said first physical coding module is configured to encode said apportionment of communication elements into a series of codes for transmission over said first communication medium.

77. The method of claim 76, wherein said first physical coding module:

encodes a first element of said apportionment with a first start code if said first element is the first element of said communication and otherwise encodes said first element of said apportionment with a second start code; and

encodes a last element of said apportionment with a first end code if said last element is the last element of said communication and otherwise encodes said last element of said apportionment with a second end code.

78. The method of claim 76, wherein said second interface is configured to convey said first apportionment at a data rate exceeding one gigabit per second.

79. The method of claim 73, in which said distributing comprises:

allotting elements of said communication among a plurality of channels established to convey a communication between said first network entity and said second network entity, including said first channel and said second channel.

80. The method of claim 73, wherein:

one of said first portion of said communication and said second portion of said communication includes a first start symbol configured to indicate a start of said communication and the other of said first portion and said second portion includes a second start symbol configured to indicate a start of a portion of said communication; and

one of said first portion of said communication and said second portion of said communication includes a first end symbol configured to indicate an end of said communication and the other of said first portion and said second portion includes a second end symbol configured to indicate an end of a portion of said communication.

81. The method of claim 73, further comprising:

encoding the first element of said first portion of said communication with a first starting delimiter; and

encoding the first element of said second portion of said communication with a second starting delimiter;

wherein said second starting delimiter is different from said first starting delimiter.

82. The method of claim 81, further comprising:

encoding the last element of said first portion of said communication with a first ending delimiter; and

encoding the last element of said second portion of said communication with a second ending delimiter;

wherein said second ending delimiter is different from said first ending delimiter.

83. The method of claim 73, wherein said sending a first portion of said communication comprises:

encoding said first portion of said communication with a first start code if said first portion of said communication includes the initial byte of said communication.

84. The method of claim 83, wherein said sending a first portion of said communication further comprises:

encoding said first portion of said communication with a second start code, different from said first start code, if said first portion of said communication does not include the initial byte of said communication.

85. The method of claim 83, wherein said sending a first portion of said communication further comprises:

encoding said first portion of said communication without a start code if said first portion of said communication does not include the initial byte of said communication.

86. The method of claim 73, wherein said sending a second portion of said communication comprises:

encoding said second portion of said communication with a first end code if said second portion of said communication includes the final byte of said communication.

87. The method of claim 86, wherein said sending a second portion of said communication further comprises:

encoding said second portion of said communication with a second end code, different from said first end code, if said second portion of said communication does not include the final byte of said communication.

88. The method of claim 86, wherein said sending a second portion of said communication further comprises:

encoding said second portion of said communication without an end code if said second portion of said communication does not include the final byte of said communication.

89. A computer readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, the method comprising:

transmitting a first idle signal on;

a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

a second channel established on a second communication medium coupled to said first network entity and said second network entity;

receiving a communication from a process operating on said first network entity, wherein the communication is directed to said second network entity;

distributing elements of said communication into multiple portions;

sending a first portion of said communication on said first channel;

sending a second portion of said communication on said second channel; and

transmitting a second idle signal on said first channel and said second channel after said sending a second portion of said communication, wherein said second idle signal is different from said first idle signal;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second.

123. A method of transmitting a communication from a first network entity to a second network entity, wherein the first network entity and the second network entity are coupled to a communication medium, comprising:

receiving a communication from a process operating on a first network entity, wherein the communication is directed to a second network entity;

receiving said communication at a distribution module of a network interface device from a medium access control module across a first interface at a data rate exceeding one gigabit per second;

distributing elements of said communication into multiple portions, each said portion corresponding to one of a plurality of channels established to convey a communication between said first network entity and said second network entity;

transferring a first portion of said communication to a first physical coding module across a second interface at a data rate exceeding one gigabit per second;

transferring a second portion of said communication to a second physical coding module across the second interface;

sending said first portion of said communication on a first channel established on a first communication medium coupled to said first network entity and said second network entity; and

sending said second portion of said communication on a second channel established on a second communication medium coupled to said first network entity and said second network entity;

wherein said communication is transmitted to said second entity at a data rate in excess of one gigabit per second; and

wherein;

one of said first portion of said communication and said second portion of said communication includes a first start symbol configured to indicate a start of said communication and the other of said first portion and said second portion includes a second start symbol configured to indicate a start of a portion of said communication; and

one of said first portion of said communication and said second portion of said communication includes a first end symbol configured to indicate an end of said communication and the other of said first portion and said second portion includes a second end symbol configured to indicate an end of a portion of said communication.

124. The method of claim 123,

wherein said first physical coding module is configured to encode said first portion of said communication with one or more of said first start symbol, said second start symbol, said first end symbol and said second end symbol.

125. The method of claim 123, further comprising:

transmitting a first idle signal on said first channel and said second channel prior to receiving said communication at said distribution module; and

transmitting a second idle signal on said first channel and said second channel after said sending said second portion of said communication;

wherein said second idle signal is different from said first idle signal.