Communicating data through a network so as to ensure quality of service

• US 20020181395A1
• Filed: 04/19/2002
• Published: 12/05/2002
• Est. Priority Date: 04/27/2001
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Dorsett, Michael A., Foster, Michael S.
• Granted Patent: US 7,164,656 B2
• Time in Patent Office: Days
• US Class Current: 370/229
• CPC Class Codes: H04L 49/101 using crossbar or matrix
• CPC Class Codes: H04L 49/357 Fibre channel switches
• CPC Class Codes: H04L 49/552 by ensuring the integrity of packets received through redundant connections
• CPC Class Codes: H04L 63/0236 Filtering by address, protocol, port number or service, e.g. IP-address or URL
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1014 based on the content of a request
• CPC Class Codes: H04L 67/1029 using data related to the state of servers by a load balancer
• CPC Class Codes: H04L 67/1031 Controlling of the operation of servers by a load balancer, e.g. adding or removing servers that serve requests
• CPC Class Codes: H04L 67/61 taking into account QoS or priority requirements
• CPC Class Codes: H04L 69/08 Protocols for interworking; Protocol conversion
• CPC Class Codes: H04L 69/22 Parsing or analysis of headers

Abstract

Techniques for communicating data through a network so as to satisfy various Quality Of Service (“QOS”) levels are described. The QOS levels may be selected based on a determined type of the data or the data communication, such as by a network manager after the data communication is registered, and QOS communication parameters are selected for the data communication to ensure that the QOS levels are achieved. Other techniques may also be used to ensure that a selected QOS level is achieved, including selecting an appropriate path through the network, controlling other data communications that use some or all of the selected path, and/or enabling preemption of such other data communications. QOS levels that are actually achieved may also be monitored, and the QOS communication parameters and/or path for a data communication may be dynamically modified based on differences between actual achieved QOS levels and desired QOS levels.

Claims

1. A method for communicating data through a network from a source node to one or more destination nodes at appropriate Quality Of Service (“

QOS”

) levels, the network having multiple routing devices and a network manager for configuring those routing devices to forward data, each routing device able to forward data to at least one or more other routing devices based on a current configuration, the method comprising;

under control of the source node,

receiving data for communication to a destination node device;

registering the data communication with the network manager by sending a registration request and receiving a transmittal virtual identifier and multiple transmission QOS communication parameters for use with the communication; and

sending to a first routing device the received data accompanied by the transmittal virtual identifier and the transmission QOS communication parameters;

under control of the network manager and in response to the registration request from the source node,

determining a type of the data to be communicated;

determining one or more QOS levels that are appropriate for communications of the determined type of data;

determining a path through the network via which communications of data of the determined type can be sent from the source node to the destination node device at the determined QOS levels, the path associated with a virtual identifier and comprising multiple intermediate routing devices including the first routing device;

determining multiple QOS communication parameters that when used by the intermediate routing devices with communications of data of the determined type should send those data communications at the determined QOS levels; and

sending the associated virtual identifier and the determined QOS communication parameters to the source node for use as the transmittal virtual identifier and the transmission QOS communication parameters for the data communication; and

under control of each of the intermediate routing devices along the path

receiving data accompanied by the transmittal virtual identifier and the transmission QOS communication parameters;

determining to forward the received data to a next device that is a next intermediate routing device along the path or the destination node device based on the accompanying transmittal virtual identifier;

determining a manner of forwarding the received data based on the accompanying transmission QOS communication parameters; and

forwarding the received data accompanied by the transmittal virtual identifier and the transmission QOS communication parameters to the determined next device in the determined manner,

so that the data communication will be sent to the destination node device at the appropriate QOS levels based on a type of the data being communicated.

2. The method of claim 1 including, under the control of the source node and after the registering of the communication of the data, repeatedly communicating additional data of the determined type to the destination node device by sending to the first routing device that additional data accompanied by the transmittal virtual identifier and the transmission QOS communication parameters.

3. The method of claim 1 including, under the control of a network monitor, monitoring network communications that include the forwarding of the received data by some or all of the intermediate routing devices in order to determine whether the data is sent at the determined QOS levels.

4. The method of claim 3 wherein the monitoring includes determining network load information for the network and notifying the network manager of the determined network load information.

5. The method of claim 3 wherein it is determined based on the monitoring that the data is not being sent at the determined QOS levels, and including, under the control of the network manager:

determining updated QOS communication parameters that are distinct from the transmission QOS communication parameters and that when used by the intermediate routing devices under current network load conditions should cause communications of data of the determined type to be sent at the determined QOS levels; and

modifying the transmission QOS communication parameters to be used with future data communication of the determined type by sending the updated QOS communication parameters to the source node as replacements.

6. The method of claim 3 wherein it is determined based on the monitoring that the data is not being sent at the determined QOS levels, and including, under the control of the network manager:

determining an updated path through the network from the source node to the destination node via which communications of data of the determined type can be sent at the determined QOS levels, the updated path distinct from the determined path; and

modifying the path to be used with future data communications of the determined type to be the updated path.

7. The method of claim 6 wherein the modifying of the path to be used with future data communications of the determined type includes associating the transmittal virtual identifier with the updated path.

8. The method of claim 3 wherein it is determined based on the monitoring that the data is not being sent at the determined QOS levels due to interference by other data communications that are being sent through the network via at least portions of the path, and including, under the control of the network manager, limiting the sending of those data communication via portions of the path to reduce the interference.

9. The method of claim 1 including, under the control of the network manager, configuring the intermediate routing devices to forward data accompanied by the determined QOS communication parameters in accordance with the determined QOS levels.

10. The method of claim 1 including, under the control of the network manager, configuring all but a last of the intermediate routing devices to forward data accompanied by the associated virtual identifier to the next intermediate routing device along the path, and configuring the last intermediate routing device along the path to forward data accompanied by the associated virtual identifier to the destination node device.

11. The method of claim 1 wherein the transmission QOS communication parameters include at least a transmission priority value and a transmission Class Of Service identifier.

12. The method of claim 11 wherein the determining of the manner of forwarding received data based on the accompanying transmission QOS communication parameters includes determining how quickly to forward the received data based at least in part on the accompanying transmission priority value and the accompanying transmission Class Of Service identifier.

13. The method of claim 1 wherein the determined QOS levels include at least one of a minimum level of throughput, a maximum level of network latency, and a maximum level of delivery jitter.

14. The method of claim 1 wherein the sent registration request includes a requested QOS level for the data communication and wherein the determining of the appropriate QOS levels includes determining whether to select the requested QOS level as one of the determined appropriate QOS levels.

15. The method of claim 1 including, under the control of the source node, negotiating with the network manager to select a mutually acceptable QOS level as one of the determined appropriate QOS levels.

16. The method of claim 1 wherein the network manager receives registration requests for data communications from each of multiple source nodes of the network, and wherein the network manager determines appropriate QOS levels and corresponding QOS communication parameters for each registered data communication.

17. The method of claim 1 wherein the network manager receives multiple registration requests for data communications from the source node, and wherein the network manager determines distinct appropriate QOS levels and corresponding QOS communication parameters for each registered data communication based on a type of data to be communicated.

18. The method of claim 1 including, under the control of the source node, storing the received multiple transmission QOS communication parameters in a provisioning table for the source node for use with future communications of data of the determined type.

19. The method of claim 1 including, under the control of the network manager, selecting one or more destination node devices for the data communication based on the determined type of the data, and wherein the determined path is selected so as to lead to the selected destination node devices.

20. A method for communicating data with an appropriate Quality Of Service (“

QOS”

) through a network having multiple routing devices and a network manager, the method comprising;

under control of a source,

registering a data communication of a first type with the network manager so that one or more QOS communication parameters for use in forwarding the data communication along a path from the source to a destination through one or more of the routing devices are identified; and

initiating a data communication of the first type to the destination by supplying data and an indication of the identified QOS communication parameters to a first of the routing devices along the path for forwarding in a manner based on the identified QOS communication parameters; and

under control of the network manager,

receiving an indication of the registering of a data communication of the first type;

determining at least one QOS level to be satisfied when transmitting a data communication of the first type along the path to the destination; and

identifying at least one QOS communication parameter that when used for transmitting a data communication of the first type along the path to the destination is expected to cause the transmitting to satisfy the determined QOS levels.

21. The method of claim 20 including, under the control of the network manager, selecting one of multiple paths through the network from the source to the destination as the path for a data communication of the first type, the selecting based on the one path being able to transmit a data communication of the first type to the destination in a manner so as to satisfy the determined QOS levels.

22. The method of claim 21 including, after the initiating of the data communication, determining that the data communication was not transmitted so as to satisfy the determined QOS levels and selecting a distinct one of the multiple paths through the network from the source node to the destination node for use with future data communications of the first type.

23. The method of claim 20 wherein the determined QOS levels include a specified amount of network bandwidth.

24. The method of claim 23 wherein the identified QOS communication parameters include an indication of a data communication Class Of Service for which at least the specified amount of network bandwidth will be allocated.

25. The method of claim 23 including, under control of the network manager, limiting other data communications that can be sent through the network via at least portions of the path so as to preserve at least the specified amount of network bandwidth for a data communication of the first type.

26. The method of claim 25 wherein the limiting of the other data communications is performed dynamically during the transmitting of a data communication of the first type.

27. The method of claim 20 wherein the determined QOS levels include a specified amount of network latency.

28. The method of claim 27 wherein the identified QOS communication parameters include an indication of a priority that will allow a data communication of the first type to be selected by the routing devices along the path for forwarding in a sufficiently prompt manner that the transmitting of that data communication will be performed with at most the specified amount of network latency.

29. The method of claim 27 including, under control of the network manager, limiting other data communications that can be sent through the network via at least portions of the path so as to prevent the routing devices along the path from delaying the forwarding of a data communication of the first type sufficiently that the specified amount of network latency is exceeded.

30. The method of claim 20 wherein the determined QOS levels include a specified amount of delivery jitter.

31. The method of claim 30 wherein the identified QOS communication parameters include an indication of a priority that will allow a data communication of the first type to be selected by the routing devices along the path for forwarding in a sufficiently predictable manner that the transmitting of that data communication will be performed with at most the specified amount of delivery jitter.

32. The method of claim 30 including, under control of the network manager, limiting other data communications that can be sent through the network via at least portions of the path so as to control a length of time that the routing devices along the path will delay the forwarding of a data communication of the first type in such a manner that the specified amount of delivery jitter is not exceeded.

33. The method of claim 20 wherein the determined QOS levels include a specified amount of network availability.

34. The method of claim 33 including, under control of the network manager, configuring the path from the source to the destination in such a manner that one or more of the routing devices along the path include multiple equivalent ports with which a data communication of the first type can be forwarded.

35. The method of claim 34 wherein, for at least one of the routing devices that includes multiple equivalent ports, the multiple equivalent ports lead to distinct next routing devices so that a problem with one of the next routing devices does not prevent the forwarding of a data communication of the first type to the destination via another of the next routing devices.

36. The method of claim 20 wherein the registering of the data communication with the network manager includes providing an indication of the first type.

37. The method of claim 20 wherein the registering of the data communication with the network manager includes providing information related to the data communication, and including, under the control of the network manager, analyzing the provided information to determine that the data communication is of the first type.

38. The method of claim 37 wherein the provided information includes an indication of data to be communicated.

39. The method of claim 20 wherein the first type of data communication involves communicating data of an first type.

40. The method of claim 20 wherein the first type of data communication involves communicating data in an indicated manner.

41. The method of claim 20 wherein the first type of data communication involves communicating data to an indicated destination.

42. The method of claim 20 wherein the first type of data communication involves communicating data that is associated with a specified destination identifier.

43. The method of claim 20 wherein the registering of the first type of data communication is performed only for an initial data communication of the first type.

44. The method of claim 20 including, under the control of the source and after the registering, repeatedly initiating additional data communications of the first type to the destination by supplying additional data to the first routing device for forwarding in a manner based on the identified QOS communication parameters.

45. The method of claim 20 including monitoring network communications in order to determine whether the initiated data communication is transmitted so as to satisfy the determined QOS levels.

46. The method of claim 20 wherein it is determined that the initiated data communication is not transmitted so as to satisfy the determined QOS levels, and including determining one or more distinct QOS communication parameters for use with future data communications of the first type.

47. The method of claim 20 wherein it is determined that the initiated data communication is not being transmitted so as to satisfy the determined QOS levels, and including limiting other data communications that are sent through the network via at least portions of the path.

48. The method of claim 20 including, under the control of the network manager, configuring the routing devices along the path to forward a data communication of the first type in accordance with the determined QOS levels.

49. The method of claim 20 including, under the control of the network manager, configuring the routing devices along the path to forward a data communication accompanied by an indication of the identified QOS communication parameters in accordance with the determined QOS levels.

50. The method of claim 20 wherein the registering of the data communication includes requesting a QOS level for the data communication, and wherein the determining of the QOS levels includes determining whether to select the requested QOS level as one of the determined QOS levels.

51. The method of claim 50 wherein the source is a source application executing on a source node, wherein the requested QOS level is specified with a manner of indicating QOS levels used by applications, wherein the determined QOS levels are specified with a distinct manner of indicating QOS levels used by the network, and wherein the determining of whether to select the requested QOS level as one of the determined QOS levels includes translating the requested QOS level into one or more equivalent QOS levels specified with the distinct manner used by the network.

52. The method of claim 20 including, under the control of the source, negotiating with the network manager to select a mutually acceptable QOS level as one of the determined QOS levels.

53. The method of claim 20 including, under the control of the network manager, selecting a virtual identifier that is associated with the path for use in routing data communications of the first type to the destination, and wherein the supplying of the data to the first routing device includes supplying an associated indication of the selected virtual identifier.

54. The method of claim 20 including, under the control of the network manager, selecting one of multiple possible destinations as the destination.

55. The method of claim 20 including, under the control of the network manager, responding to the registering by sending to the source an indication of the identified QOS communication parameters.

56. The method of claim 20 wherein multiple distinct sources each register one or more distinct types of data communications, and wherein the network manager determines appropriate QOS communication parameters for each registered type of data communication.

57. The method of claim 54 wherein the registering of the one or more distinct types of data communications occurs before the determining of the appropriate QOS communication parameters, and wherein the determining of the appropriate QOS communication parameters for each registered type of data communication is based at least in part on balancing available network resources among the distinct types of data communications.

59. The computer-readable medium of claim 58 including, under the control of the network manager, selecting the path through the network from the source to the destination based on the path being able to transmit data of the first type to the destination in a manner so as to satisfy the determined QOS levels.

60. The computer-readable medium of claim 58 including, after the initiating of the data communication, determining that the data communication was not transmitted so as to satisfy the determined QOS levels and updating a manner of transmitting future data of the first type so as to increase a likelihood that the future data will be transmitted so as to satisfy the determined QOS levels.

61. The computer-readable medium of claim 58 wherein the computer-readable medium is a memory of a computing device.

62. The computer-readable medium of claim 58 wherein the computer-readable medium is a data transmission medium transmitting a generated data signal containing the contents.

63. The computer-readable medium of claim 58 wherein the contents are instructions that when executed cause the computing device to perform the method.

58. A computer-readable medium whose contents cause a computing device to communicate data through a network with an appropriate Quality Of Service (“

QOS”

), by performing a method comprising;

under control of a source,

registering a communication of data of a first type with a network manager for the network so that one or more QOS communication parameters for use in forwarding the data along a path from the source to a destination through one or more of the routing devices can be identified; and

initiating a communication of data of the first type to the destination by supplying the data and an indication of the identified QOS communication parameters to a first of the routing devices along the path for forwarding in a manner based on the identified QOS communication parameters; and

under control of the network manager,

receiving an indication of the registering of a communication of data of the first type; and

identifying at least one QOS communication parameter that when used for transmitting data of the first type along the path to the destination should cause the transmitting to satisfy one or more QOS levels determined based on the first type.

64. A computing system for communicating data through a network with an appropriate Quality Of Service (“

QOS”

), comprising;

one or more source nodes each capable of registering a communication of data with a network manager so that one or more QOS communication parameters for use in forwarding the data along a path from the source to a destination through one or more of the routing devices can be identified, and of initiating the data communication to the destination by supplying the data and an indication of the identified QOS communication parameters to a first of the routing devices along the path for forwarding in a manner based on the identified QOS communication parameters; and

a network manager for the network that is capable of receiving an indication of registering of a communication of data, of determining based on a type of the data communication at least one QOS level to be satisfied during transmitting of the data communication, and of identifying at least one QOS communication parameter for use in transmitting the data communication so as to satisfy the determined QOS levels.

65. The computing system of claim 64 further comprising multiple routing devices each capable of receiving the data communication and an indication of the identified QOS communication parameters and of forwarding the data communication in accordance with the identified QOS communication parameters.

66. A computing system for communicating data through a network with an appropriate Quality Of Service (“

QOS”

), comprising;

at least one source means for registering a communication of data with a network manager so that one or more QOS communication parameters for use in forwarding the data along a path from the source to a destination through one or more of the routing devices can be identified and for initiating the data communication to the destination by supplying the data and an indication of the identified QOS communication parameters to a first of the routing devices along the path for forwarding in a manner based on the identified QOS communication parameters; and

a network manager means for receiving an indication of registering by a source node of a communication of data, for determining based on a type of the data communication at least one QOS level to be satisfied during transmitting of the data communication, and for identifying at least one QOS communication parameter for use in transmitting the data communication so as to satisfy the determined QOS levels.

68. The method of claim 67 including selecting a virtual identifier for use in communicating data of the registered type to the destination, and wherein the configuring of the routing devices to communicate received data of the registered type includes associating the selected virtual identifier with data communications of the registered type, wherein the communicating of the data to the first routing device includes providing an associated indication of the selected virtual identifier, and wherein the communicating of the received data to the next routing device includes providing an associated indication of the selected virtual identifier.

69. The method of claim 67 including, under control of the network manager, notifying the source of the determined QOS communication parameters, and wherein the communicating of the data to the first routing device includes providing an associated indication of the determined QOS communication parameters.

70. The method of claim 67 wherein the configuring of the routing devices to communicate received data of the registered type to a next routing device along the path in a manner consistent with the determined QOS communication parameters includes configuring those routing devices to filter received data of the registered type that have associated indications of QOS communication parameters other than the determined QOS communication parameters.

67. A method for communicating data through a network having multiple routing devices and a network manager, the method comprising:

under control of a source,

registering for communication of a type of data with the network manager so that data of that type will be communicated to a destination with an appropriate Quality Of Service (“

QOS”

); and

after the registering, communicating data of the registered type to the destination by providing the data to a first routing device along a path to the destination;

under control of the network manager,

receiving an indication of the registering;

determining one or more QOS communication parameters that are to be satisfied during communicating of data of the registered type; and

configuring at least some of one or more routing devices along the path to communicate received data of the registered type to a next routing device along the path in a manner consistent with the determined QOS communication parameters; and

under control of each of the configured routing devices,

receiving communicated data of the registered type; and

communicating the received data to the next routing device along the path as configured.

71. A method for a network manager to assist in communicating data through a network with appropriate Quality Of Service (“

QOS”

), the method comprising;

receiving an indication of a type of data communication to be transmitted through the network from a source to a destination;

determining one or more QOS levels for data communications of the indicated type;

determining a path through the network via which data communications of the indicated type can be sent from the source to the destination at the determined QOS levels, the path including at least one routing device;

determining a data communication transmission priority that is appropriate for use in transmitting data communications of the indicated type along the path to the destination at the determined QOS levels; and

associating the determined path and the determined data communication transmission priority with data communications of the indicated type from the source to the destination in such a manner that the determined data communication transmission priority will be used to transmit data communications of the indicated type along the path from the source to the destination.

72. The method of claim 71 including determining a Class Of Service for data communications of the indicated type such that data communications of the indicated type that are transmitted using a combination of the determined Class Of Service and the determined data communication transmission priority will be transmitted at the determined QOS level.

73. The method of claim 71 including, after determining that a data communication of the indicated type was not transmitted at the determined QOS levels, modifying the data communication transmission priority and/or the path associated with data communications of the indicated type so that future data communications of the indicated type are more likely to be transmitted at the determined QOS levels.

74. The method of claim 71 wherein the associating of the determined path and the determined data communication transmission priority with data communications of the indicated type from the source to the destination includes notifying the source to use the determined data communication transmission priority with data communications of the indicated type.

75. The method of claim 71 wherein the associating of the determined path and the determined data communication transmission priority with data communications of the indicated type from the source to the destination includes notifying the source to use a virtual identifier associated with the determined path with data communications of the indicated type.

76. The method of claim 71 wherein the associating of the determined path and the determined data communication transmission priority with data communications of the indicated type from the source to the destination includes configuring at least some of the routing devices along the path to forward a data communication of the indicated type along the determined path.

77. The method of claim 71 wherein the associating of the determined path and the determined data communication transmission priority with data communications of the indicated type from the source to the destination includes configuring at least some of the routing devices along the path to process a data communication of the indicated type using the determined data communication transmission priority.

78. A method for communicating data with an appropriate Quality Of Service (“

QOS”

) through a network having multiple routing devices and a network manager, the method comprising;

under control of a source,

registering a data communication with the network manager so that one or more QOS communication parameters for use in forwarding the data communication along a path from the source to a destination through one or more of the routing devices are identified; and

initiating the data communication to the destination by supplying data to a first of the routing devices along the path for forwarding in a manner based on the identified QOS communication parameters; and

under control of the network manager,

receiving an indication of the registering of the data communication;

determining at least one QOS level to be satisfied when transmitting the data communication along the path to the destination; and

identifying at least one QOS communication parameter for use in transmitting the data communication along the path to the destination so as to satisfy the determined QOS levels; and

associating the identified QOS communication parameters with the data communication in such a manner that the identified QOS communication parameters will be used to transmit the data communication along the path from the source to the destination.