Hierarchical multicast traffic security system in an internetwork

• US 6,606,706 B1
• Filed: 02/08/1999
• Issued: 08/12/2003
• Est. Priority Date: 02/08/1999
• Status: Expired due to Term

• Current Assignee: Avaya Incorporated
• Original Assignee: Nortel Networks Limited (Nortel Networks Corporation)
• Inventors: Li, Yunzhou
• Primary Examiner(s): Barron, Gilberto
• Assistant Examiner(s): Zand, Kambiz
• Application Number:
  US09/246,259
• Time in Patent Office: 1,646 Days
• Field of Search: 713/162-163, 713/153, 713/160, 713/201, 380/34, 709/240, 709/224, 709/238, 709/221, 370/412, 370/366, 370/392, 370/351, 370/239, 370/401
• US Class Current: 713/162
• CPC Class Codes: H04L 12/18 for broadcast or conference , e.g. multicast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/062 for key distribution, e.g. centrally by trusted party cryptographic mechanisms or cryptographic arrangements for key distribution involving a central third party H04L9/0819
• CPC Class Codes: H04L 63/104 Grouping of entities

Abstract

Multicast networks are partitioned into hierarchical security domains. Each security domain may comprise one or more lower security domains. Each security domain includes a security broker that distributes a group key and translates multicast data destined to the security domain, if necessary. A primary security broker at the second level of the hierarchical multicast system distributes the top security key to all peer members, including all peer security domain brokers to establish trust relationships. For each security domain boundary with security domain border routers, a multicast virtual link in configured that connects the security domain border routers and the security broker for the security domain to reduce the latency in forwarding multicast data. It can also make the backbone of the security domain contiguous so that multicast data can travel unchanged across the backbone. The multicast data is forwarded to the security domain through the security broker with security translation. A group key is distributed at each hierarchy level by exchange of Group Key Request and Group Key Reply messages. The rekey process is accomplished by multicasting Rekey Announcement messages, either regionally by a security broker, or globally by the group controller through the primary top regional security broker.

Claims

1. A method for transmitting a multicast data packet in an internetwork from a source node to a destination node, the method comprising:

encrypting at the source node the multicast data packet to create an encrypted multicast data packet having a first source address field specifying an address of the source node and a first destination address field specifying an multicast address recognized by the destination node;

transmitting the encrypted multicast data packet to a security broker in a security domain of which the source node and security broker are members;

decrypting the encrypted multicast data packet at the security broker;

encrypting at the security broker the multicast data packet to create an encrypted multicast packet having the second source address field specifying the address of the security broker and the second destination address field specifying the multicast address recognized by the destination node;

encapsulating at the security broker the encrypted multicast data packet to create an encapsulated encrypted multicast data packet having a third source address field specifying the address of the security broker and a third destination address field specifying an unicast address of a border router in the security domain;

transmitting the encapsulated encrypted multicast data packet to the border router;

receiving and decapsulating the encapsulated encrypted multicast data packet at the border router;

encapsulating at the border router the encrypted multicast data packet to create an encapsulated encrypted multicast data packet having the third source address field specifying the address of the border router and the third destination address field specifying an address recognized by a second security broker in a second security domain shared with the border router; and

transmitting the encapsulated encrypted multicast data packet to the second security broker.

2. A method for transmitting a multicast data packet in an internetwork from a source node to a destination node, the method comprising:

encrypting at the source node the multicast data packet to create an encrypted multicast data packet having a first source address field specifying an address of the source node and a first destination address field specifying an multicast address recognized by the destination node;

transmitting the encrypted multicast data packet to a security broker in a security domain of which the source node and security broker are members;

decrypting the encrypted multicast data packet at the security broker;

encrypting at the security broker the multicast data packet to create an encrypted multicast packet having the second source address field specifying the address of the security broker and the second destination address field specifying the multicast address recognized by the destination node;

encapsulating at the security broker the encrypted multicast data packet to create an encapsulated encrypted multicast data packet having a third source address field specifying the address of the security broker and a third destination address field specifying a multicast address recognized by a border router in the security domain;

transmitting the encapsulated encrypted multicast data packet to the border router;

receiving and decapsulating the encapsulated encrypted multicast data packet at the border router;

encapsulating at the border router the encrypted multicast data packet to create an encapsulated encrypted multicast data packet having the third source address field specifying the address of the border router and the third destination address field specifying an address recognized by a second security broker in a second security domain shared with the border router; and

transmitting the encapsulated encrypted multicast data packet to the second security broker.

3. A method for transmitting a multicast data packet, comprising:

a) encrypting the multicast data packet to create an encrypted multicast data packet having two source address fields each specifying an address of a source node and two destination address fields each specifying a multicast address recognized by a destination node;

b) transmitting the encrypted multicast data packet in a security domain shared by the source node;

c) decrypting the encrypted multicast data packet at a security broker in the security domain;

d) modifying one of the source address fields to specify an address of the security broker;

e) encapsulating the multicast data packet with a third source address field specifying the address of the security broker and a third destination address field specifying an address recognized by a border router in the security domain;

f) transmitting the encapsulated multicast data packet to the border router;

g) receiving the encapsulated multicast data packet at the border router;

h) modifying the third source address field to specify the address of the border router and the third destination address field to specify a unicast address recognized by a second security broker in a second security domain shared with the border router; and

i) transmitting the encapsulated multicast data packet to the second security broker.

4. The method of claim 3, wherein modifying the third source address field to specify the address of the border router and the third destination address field to specify an address recognized by a second security broker in a second security domain shared with the border router comprises modifying the third source address field to specify the address of the border router and the third destination address field to specify a multicast address recognized by a second security broker in a second security domain shared with the border router.

5. An article of manufacture comprising a computer usable medium having computer readable program code embodied therein to cause a multicast data packet to be transmitted, comprising:

computer readable program code to encrypt the multicast data packet to create an encrypted multicast data packet having two source address fields each specifying an address of a source node and two destination address fields each specifying a multicast address recognized by a destination node;

computer readable program code to transmit the encrypted multicast data packet in a security domain shared by the source node;

computer readable program code to decrypt the encrypted multicast data packet at a security broker in the security domain;

computer readable program code to modify one of the source address fields to specify an address of the security broker;

computer readable program code to encapsulate the multicast data packet with a third source address field specifying the address of the security broker and a third destination address field specifying a multicast address recognized by a border router in the security domain; and

computer readable program code to transmit the encapsulated multicast data packet to the border router.