Secure virtual LANs

• US 6,035,405 A
• Filed: 12/22/1997
• Issued: 03/07/2000
• Est. Priority Date: 12/22/1997
• Status: Expired due to Term

• Current Assignee: Avaya Incorporated
• Original Assignee: Nortel Networks Corporation
• Inventors: Himbeault, Lee, Wollensak, Martin, Gage, William W. A.
• Primary Examiner(s): Butler, Dennis M.
• Assistant Examiner(s): Mai, Rijue
• Application Number:
  US08/996,159
• Time in Patent Office: 806 Days
• Field of Search: 713/201, 713/200, 713/202, 709/227, 709/218, 709/219, 709/238, 709/203, 709/220, 709/223, 709/245, 709/249, 709/246, 709/250, 709/1, 709/237, 709/229, 709/224, 709/242, 709/228, 370/552, 370/16, 370/60, 370/94.01, 370/404, 380/23, 380/25, 380/48, 380/28, 345/356
• US Class Current: 726/15
• CPC Class Codes: H04L 12/4608 LAN interconnection over ATM networks
• CPC Class Codes: H04L 12/4641 Virtual LANs, VLANs, e.g. virtual private networks [VPN] LAN interconnection over a bridge based backbone H04L12/462; encapsulation techniques H04L12/4633; routing of packets H04L45/00; packet switches H04L49/00; virtual private networks for security H04L63/0272
• CPC Class Codes: H04L 2012/5617 Virtual LANs; Emulation of LANs
• CPC Class Codes: H04L 2012/5687 Security aspects
• CPC Class Codes: H04L 63/0272 Virtual private networks
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04Q 11/0478 Provisions for broadband connections

Abstract

The present invention discloses a method for securely adding a new end station to a local area network (LAN) segmented into a number of virtual local area networks (VLANs). The invention is applicable to various types of LANs such as Ethernet and token ring. The LAN comprises an authentication server (AS) which interacts with each new end station before connection to a VLAN is allowed. The method involves the AS administering a test to the new end station, which may involve prompting the new end station for a password or asking it to encrypt a given number using a secret algorithm known only to the new end station and to the AS. The AS examines the results of this test and determines whether the new end station is permitted to join the VLAN. For added security, the new end station can verify authenticity of the AS by administering a test of its own, which may consist of prompting the AS for a password of its own or asking it to encrypt a new number, the new end station subsequently determining whether the AS is indeed genuine before beginning to transmit any further information. In this way, an end station cannot join a VLAN without authentication by the AS and a legitimate end station can verify whether the test it is asked to pass comes from a legitimate source, thereby avoiding network security breaches.

Claims

1. A local area network, comprising a plurality of end stations and a authentication server, the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising at least one member end station, wherein the authentication server keeps track of which end stations are members of which VLAN, keeps track of which end stations are authorized to join which VLAN and administers an authentication test to new end stations joining a VLAN.

2. The local area network of claim 1 being a token ring LAN.

3. The local area network of claim 1 being an Ethernet LAN.

4. The local area network of claim 3 further comprising a plurality of Ethernet switches, each switch communicating with at least one end station through an Ethernet communication link.

5. A local area network, comprising:

a plurality of end stations;

a plurality of LAN emulation server (LESs);

a LAN emulation configuration server (LECS); and

an authentication server (AS);

the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising a respective LES and at least one member end station, each LES keeping track of which end stations are members in the respective VLAN, the LECS keeping track of which end stations are members of which VLAN;

wherein the authentication server keeps track of which end stations are authorized to join which VLAN and administers an authentication test to new end stations joining a VLAN.

6. The local area network of claim 5, wherein the LECS is merged with the AS.

7. The local area network of claim 5 being a token ring LAN.

8. The local area network of claim 5 being an Ethernet LAN.

9. The local area network of claim 8 further comprising a plurality of Ethernet switches, each switch communicating with at least one end station through an Ethernet communication link.

10. The local area network of claim 9, wherein the Ethernet switches communicate with each other through an ATM link.

11. A method for securely adding a new station to a local area network (LAN), the LAN comprising a plurality of end stations and an authentication server (AS), the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising at least one member end station, wherein the authentication server keeps tracks of which end stations are members of which VLAN, keeps track of which end stations are permitted to join which VLAN and performs authentication of end stations joining a VLAN, the method comprising:

the new end station sending to the AS a message identifying both the new end station and a desired VLAN;

the new end station taking an authentication test administered by the AS; and

upon successful authentication of the new end station, the AS sending to the new end station a message indication that the new end station has been permitted to join the desired VLAN.

12. The method of claim 11, wherein the new end station is identified by a 48-bit media access control address.

13. The method of claim 11, wherein the new end station is identified by a 32- bit Internet Protocol address.

14. The method of claim 11, wherein the new end station is identified by a physical port on an Ethernet switch.

15. The method of claim 11, wherein the authentication test consists of:

the AS generating a first encrypted number using a plain number and an algorithm known to the AS and to the new end station;

the AS sending to the new end station a message comprising the plain number;

the new end station generating a second encrypted number using the plain number and the algorithm;

the new end station sending to the AS a message comprising the plain number and the second encrypted number; and

the AS comparing the first encrypted number to the second encrypted number;

wherein authentication of the new end station is said to have been successful if the first and second encrypted numbers are identical.

16. The method of claim 15, wherein the plain number is a random number.

17. The method of claim 15, wherein the algorithm is a key-based encryption algorithm.

18. A method for securely adding a new end station to a local area network (LAN), the LAN comprising a plurality of end stations, a plurality of LAN emulation servers (LESs), a LAN emulation configuration server (LECS) and an authentication server (AS), each switch communicating with at least one end station, the new end station being connected to a switch, the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising a respective LES and at least one member end station, each LES keeping track of which end stations are currently members in the respective VLAN, the LECS keeping track of which end stations are permitted to be members of which VLAN, wherein the authentication server performs authentication of end stations joining a VLAN, the method comprising:

the new end station sending to its switch a message identifying both the new end station and a desired VLAN;

the switch sending to LECS a message requesting identity of the LES corresponding to the desired VLAN;

the LECS sending to the AS a message requesting authentication of the new end station;

the AS generating a first encrypted number using a plain number and an algorithm known to the AS and to the new end station;

the AS sending to the LECS a message comprising the plain number and the first encrypted number;

the LECS sending to the switch a message comprising the plain number;

the switch sending to the new end station a message comprising the plain number;

the new end station generating a second encrypted number using the plain number and the algorithm;

the new end station sending to the switch a message compising the plain bnumber and the second encrypted number;

the switch sending to the LECS a message comprising the plain number and the second encrypted number;

the LECS comparing the first encrypted number to the second encrypted number;

the LECS sending to the LES corresponding to the desired VLAN a message indicating that the new end station intends to join the desired VLAN;

the LECS sending to the switch a message comprising identity of the LES corresponding to the desired VLAN;

the switch sending to the LES corresponding to the desired VLAN a message requesting that the new end station join the desired VLAN; and

the LES corresponding to the desired VLAN sending to the switch a message indicating that the new end station has been allowed to join the desired VLAN.

19. The method of claim 18, wherein the new end station is identified by a 48-bit media access control address.

20. The method of claim 18, wherein the new end station is identified by a 32- bit Internet Protocol address.

21. The method of claim 18, wherein the new end station is identified by a physical port on an Ethernet switch.

22. The method of claim 18, wherein the plain number is a random number.

23. The method of claim 18, wherein the algorithm is a key-based encryption algorithm.

24. A method for securely adding a end station to a local area network (LAN), the LAN comprising a plurality of end stations and an authentication server (AS), the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising at least one member end station, wherein the authentication server keeps track of which end stations are members of which VLAN, keeps track of which end stations are permitted to join which VLAN and performs authentication of end stations joining a VLAN, the method comprising:

the new end station sending to the AS a message identifying both the new end station and a desired VLAN;

the AS and the new end station taking an authentication test;

upon successful authentication of the new end station, the AS sending to the new end station a message indicating that the new end station has been permitted to join the desired VLAN; and

upon successful authentication of the AS, the new end station joining the desired VLAN.

25. The method of claim 24, wherein the new end station is identified by a 48-bit media access control address.

26. The method of claim 24, wherein the new end station is identified by a 32-bit Internet Protocol address.

27. The method of claim 24, wherein the new end station is identified by a physical port on a Ethernet switch.

28. The method of claim 24, wherein both the AS and the new end station store respective first and second lists of passwords, and the authentication test consists of:

the new end station sending a message to the AS comprising a first password;

the AS comparing the first password to a second password contained in the first list of passwords and sending a message to the new end station comprising a third password; and

the new end station comparing the third password to a fourth password contained in the second list of passwords;

wherein authentication of the new end station is said to have been successful if the first and second passwords are identical;

wherein authentication of the AS is said to have been successful if the third and fourth passwords are identical.

29. The method of claim 24, wherein the authentication test consists of:

the AS generating a first encrypted number using a first plain number and an algorithm known to the AS and to the new end station;

the AS sending to the new end station a message comprising the plain number;

the new end station generating a second encrypted number using the plain number and the algorithm;

the new end station generating a third encrypted number using a second plain number and the algorithm;

the new end station sending to the AS a message comprising the first plain number, the second encrypted number and the second plain number;

the AS comparing the first encrypted number to the second encrypted number;

the AS generating a fourth encrypted number from the second plain number;

the AS sending to the new end station a message indicating that the new end station has been allowed to join the desired VLAN;

the AS sending to the new end station a message comprising the second plain number and the fourth encrypted number; and

the new end station comparing the third encrypted number to the fourth encrypted number;

wherein authentication of the new end station is said to have been successful if the first and second encrypted numbers are identical;

wherein authentication of the new end station is said to have been successful if the third and fourth encrypted numbers are identical.

30. The method of claim 29, wherein the first and second plain numbers are random numbers.

31. The method of claim 29, wherein the algorithm is a to key-based encryption algorithm.

32. A method for securely adding a new end station to a local area network (LAN), the LAN comprising a plurality of end stations, a plurality of LAN emulation servers (LESs), a LAN emulation configuration server (LECS) and an authentication server (AS), each switch communicating with at least one end station, the new end station being connected to a switch the LAN being segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising a respective LES and at least one member end station, each LES keeping track of which end stations are currently members in the respective VLAN, the LECS keeping track of which end stations are permitted to be members of which VLAN, wherein the authentication server performs authentication of end stations joining a VLAN, the method comprising:

the new end station sending to its switch a message identifying both the new end station and a desired VLAN;

the switch sending to the LECS a message requesting identity of the LES corresponding to the desired VLAN;

the LECS sending to the AS a message requesting authentication of the new end station;

the AS generating a first encrypted number using a first plain number and an algorithm known to the AS and to the new end station;

the AS sending to the LECS a message comprising the first plain number and the first encrypted number;

the LECS sending to the switch a message comprising the first plain number;

the switch sending to the new end station a message comprising the first plain number;

the new end station generating a second encrypted number using the first plain number and the algorithm;

the new end station generating a third encrypted number using the second plain number and the algorithm;

the new end station sending to the switch a message comprising the first plain number, the second encrypted number and the second plain number;

the switch sending to the LECS a message comprising the first plain number, the second encrypted number and the second plain number;

the LECS comparing the first encrypted number to the second encrypted number;

the LECS sending to the AS a message comprising the second plain number;

the AS generating a fourth encrypted number from the second plain number and the algorithm;

the AS sending to the LECS a message comprising the second plain number and the fourth encrypted number;

the LECS sending to the LES corresponding to the desired VLAN a message comprising the second plain number and the fourth encrypted number, and indicating that the new end station intends to join the desired VLAN;

the LECS sending to the switch a message comprising identity of the LES corresponding to the desired VLAN;

the switch sending to the LES corresponding to the desired VLAN a message requesting that the new end station join the desired VLAN;

the LES corresponding to the desired VLAN sending to the switch a message indicating that the new end station has been allowed to join the desired VLAN;

the LES corresponding to the desired VLAN sending to the switch a message comprising the second plain number and the fourth encrypted number;

the switch sending to the new end station a message comprising the second plain number and the fourth encrypted number; and

the new end station comparing the third encrypted number to the fourth encrypted number.

33. The method of claim 32, wherein the new end station is identified by a 48-bit media access control address.

34. The method of claim 32, wherein the new end station is identified by a 32-bit Internet Protocol address.

35. The method of claim 32, wherein the new end station is identified by a physical port on an Ethernet switch.

36. The method of claim 32, wherein the first and second plain numbers are random numbers.

37. The method of claim 32, wherein the algorithm is a key-based encryption algorithm.

38. An authentication server (AS) for use in a local area network (LAN) segmented into a plurality of virtual local area networks (VLANs), each VLAN comprising at least one member end station, comprising:

means for keeping track of which end stations are members of which VLAN;

means for keeping track of which end stations are permitted to join which VLAN; and

in response to a new end station sending to the AS a message identifying both the new end station and a desired VLAN, means for performing authentication of the new end station by administering a test to the new end station and, upon successful authentication of the new end station, sending to the new end station a message indicating that the new end station has been permitted to join the desired VLAN.

39. An authentication server (AS) as claimers in claim 38, wherein the AS administering the authentication test comprises:

the AS generating a first encrypted number using a plain number and an algorithm known to the AS and to the new end station;

the AS sending to the new end station a message comprising the plain number;

the AS receiving a message comprising the plain number and a second encrypted number; and

the AS comparing the first encrypted number to the second encrypted number, wherein authentication of the new end station is said to have been successful if the first and second encrypted numbers are identical.

40. An authentication server (AS) for use in a local area network (LAN) segmented into a plurality of virtual local area networks (VLANS), each VLAN comprising at least one member end station, comprising:

means for keeping track of which end stations are members of which VLAN;

means for keeping track of which end stations are permitted to join which VLAN; and

in response to a new end station sending to the AS a message identifying both the new end station and a desired VLAN, means for taking an authentication test together with the new end station and, upon successful authentication of the new end station, the AS sending to the new end station a message indicating that the new end station has been permitted to loin the desired VLAN.

41. An authentication server (AS) as claimed in claim 40, wherein the AS stores a first list of passwords and a second list of passwords and wherein the AS taking the authentication test comprises:

the AS receiving a message comprising a first password;

the AS comparing the first password to a second password contained in the first list of passwords and sending a message to the new end station comprising a third password for authentication of the AS by the new end station;

wherein authentication of the new end station is said to have been successful if the first and second passwords are identical.

42. An authentication server (AS) as claimed in claim 40, wherein the AS taking the authentication test comprises:

the AS generating a first encrypted number using a first plain number and an algorithm known to the AS and to the new end station;

the AS sending to the new end station a message comprising the first plain number;

the AS receiving a message comprising the first plain number, a second encrypted number and a second plain number;

the AS comparing the first encrypted number to the second encrypted number;

the AS generating a third encrypted number from the second plain number;

the AS sending to the new end station a message comprising the second plain number and the third encrypted number, for authentication of the AS by the new end station,

wherein authentication of the new end station is said to have been successful if the first and second encrypted numbers are identical.