Method and apparatus for traversing a translation device with a security protocol

• US 20040088537A1
• Filed: 10/31/2002
• Published: 05/06/2004
• Est. Priority Date: 10/31/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Swander, Brian D., Dixon, William H.
• Granted Patent: US 7,346,770 B2
• Time in Patent Office: Days
• US Class Current: 713/153
• CPC Class Codes: H04L 63/0464 using hop-by-hop encryption, i.e. wherein an intermediate entity decrypts the information and re-encrypts it before forwarding it
• CPC Class Codes: H04L 63/164 at the network layer

Abstract

The invention uses a three phase IKE protocol main mode negotiation to implement a port float algorithm that permits UDP encapsulated ESP traffic to traverse an IPSec-aware NAT. The NAT is connected to a plurality of client computers on a private network and provides an interface between the client computers and a server connected to a public network. In a first phase, a client and the server determine whether both are capable of sending UDP encapsulated ESP packets. In a second phase, the client and server conduct NAT discovery and determine whether the client, server, or both operate behind a NAT. In a third phase, the client and server initiate a port float algorithm, moving a destination UDP port specified in IKE packets from a first port value to a second port value. The server maintains a data structure that allows the server to identify the client sending IKE packets after exiting the second phase and entering the third phase.

Claims

1. A method for sending UDP encapsulated ESP packets through a NAT on a private network from a client on the private network to a server on a public network, using an IKE negotiation, comprising:

determining whether both the client and server are capable of sending the UDP encapsulated ESP packets, wherein the client '"'"'sends a first IKE packet to the server and receives a second IKE packet from the server, the first and second IKE packets using first source and destination UDP port addresses;

determining whether at least one of the client or the server operate behind the NAT;

changing the first source and destination port addresses to second source and destination port addresses, the second source and destination port addresses being distinct from the first source and destination port addresses;

sending UDP encapsulated ESP packets using the second source and destination port addresses.

2. The method of claim 1, wherein the NAT interprets the UDP encapsulated ESP packets designating the first destination port as invalid and UDP encapsulated ESP packets designating the second destination port as valid.

3. The method of claim 1, wherein the NAT interprets the UDP encapsulated ESP packets designating the first destination port as IKE packets and UDP encapsulated ESP packets designating the second destination port as non-IKE packets.

4. The method of claim 1, wherein the first and second IKE packets each include a vendor ID;

the vendor ID being a known value indicating a capability of sending UDP encapsulated ESP packets.

5. The method of claim 1, wherein the step of determining whether at least one of the client or the server operate behind the NAT further comprises;

sending a third IKE packet including a first and second NAT discovery payload, the first NAT discovery payload including a hash function of the first source port address and a source IP address as known to the client and the second NAT discovery payload including a hash function of the first destination port address and a destination IP address as known to the client;

receiving a fourth IKE packet including a third and a fourth NAT discovery payload, the third NAT discovery payload including a hash function of the first source port address and a source IP address as known to the server and the fourth NAT discovery payload including a hash function of the first destination port address and a destination IP address as known to the server; and

determining based on a comparison of the first and third NAT discovery payloads that the client operates behind the NAT.

6. The method of claim 5 further comprising:

determining based on a comparison of the second and fourth NAT discovery payloads that the server does not operate behind the NAT.

7. The method of claim 1, wherein the IKE negotiation is a main mode negotiation.

8. The method of claim 1, further comprising:

maintaining data that uniquely identifies a connection between the client and server such that the server can identify the client after the first source and destination ports are changed to the second source and destination ports.

9. The method of claim 8 wherein the data comprises an I-Cookie value assigned by the client, an R-Cookie value assigned by the server, an IP address of the client and an IP address of the server.

10. The method of claim 1, further comprising sending a request for a new cryptographic key in third IKE packet using the second source and destination port addresses.

11. A method for sending UDP encapsulated ESP packets through a NAT on a private network from a client on the private network to a server on a public network, whereby the NAT interprets UDP encapsulated ESP packets using a first destination port address as IKE packets, comprising;

conducting an IKE negotiation by sending IKE packets designating a second destination port, the second destination port being distinct from the first destination port; and

sending the UDP encapsulated ESP packets using the second destination port.

12. A method for receiving UDP encapsulated ESP packets at a server on a public network, the UDP encapsulated ESP packets being sent from a client operating behind a NAT on a private network, comprising:

receiving a first IKE packet and sending a second IKE packet, the first and second IKE packets designating a first destination port and including a vendor identification value indicating a capability to send the UDP encapsulated ESP packets;

determining that at least the client operates behind the NAT;

receiving a third IKE packet designating a second destination port, the second destination port being distinct from the first destination port;

determining that the third IKE packet is sent by the client;

receiving UDP encapsulated ESP packets, the UDP encapsulated ESP packets designating the second destination port.

13. The method of claim 12, wherein the step of determining that at least the client operates behind a NAT comprises:

receiving a discovery payload from the client, the discovery payload including a first value derived from a hash function of a client IP and port address as known to the client;

calculating a second value from a hash function of the client IP and port address as known to the server.

comparing the first value to the second value and determining that the client operates behind the NAT.

14. The method of claim 13, wherein the first value is distinct from the second value.

15. The method of claim 12, wherein the step of determining that the third IKE packet is sent by the client comprises:

comparing a unique identification within the third IKE packet to a stored value, the stored value identifying a connection between the client and the server.

16. The method of claim 15, wherein the stored valued comprises an I-Cookie value assigned by the client, an R-Cookie value assigned by the server, an IP address of the client and an IP address of the server.

17. A computer-readable medium having computer executable instruction for sending UDP encapsulated ESP packets through a NAT on a private network from a client on the private network to a server on a public network, using an IKE negotiation, comprising:

determining whether both the client and server are capable of sending the UDP encapsulated ESP packets, wherein the client sends a first IKE packet to the server and receives a second IKE packet from the server, the first and second IKE packets using first source and destination UDP port addresses;

determining whether at least one of the client or the server operate behind the NAT;

changing the first source and destination port addresses to second source and destination port addresses, the second source and destination port addresses being distinct from the first source and destination port addresses;

sending UDP encapsulated ESP packets using the second source and destination port addresses.

18. The computer-readable medium of claim 17, wherein the first and second IKE packets each include a vendor ID;

the vendor ID being a known value indicating a capability of sending UDP encapsulated ESP packets.

19. The computer-readable medium of claim 17, wherein determining whether at least one of the client or the server operate behind the NAT comprises;

sending a third IKE packet including a first and second NAT discovery payload, the first NAT discovery payload including a hash function of the first source port address and a source IP address as known to the client and the second NAT discovery payload including a hash function of the first destination port address and a destination IP address as known to the client;

receiving a fourth IKE packet including a third and a fourth NAT discovery payload, the third NAT discovery payload including a hash function of the first source port address and a source IP address as known to the server and the fourth NAT discovery payload including a hash function of the first destination port address and a destination IP address as known to the server; and

determining based on a comparison of the first and third NAT discovery payloads that the client operates behind the NAT.

20. The computer-readable medium of claim 17, further comprising:

maintaining data that uniquely identifies a connection between the client and server such that the server can identify the client after the first source and destination ports are changed to the second source and destination ports.

21. The computer-readable medium of claim 20 wherein the data comprises an I-Cookie value assigned by the client, an R-Cookie value assigned by the server, an IP address of the client and an IP address of the server.

22. The computer-readable medium of claim 17, further comprising sending a request for a new cryptographic key in a third IKE packet using the second source and destination port addresses.

23. A computer-readable medium having computer executable instructions for sending UDP encapsulated ESP packets through a NAT on a private network from a client on the private network to a server on a public network, whereby the NAT interprets UDP encapsulated ESP packets using a first destination port address as IKE packets, comprising;

conducting an IKE negotiation by sending IKE packets designating a second destination port, the second destination port being distinct from the first destination port; and

sending the UDP encapsulated ESPpackets using the second destination port.

24. A computer-readable medium having computer executable instructions for receiving UDP encapsulated ESP packets at a server on a public network, the UDP encapsulated ESP packets being sent from a client operating behind a NAT on a private network, comprising:

receiving a first IKE packet and sending a second IKE packet, the first and second IKE packets designating a first destination port and including a vendor identification value indicating a capability to send the UDP encapsulated ESP packets;

determining that at least the client operates behind the NAT;

receiving a third IKE packet designating a second destination port, the second destination port being distinct from the first destination port;

determining that the third IKE packet is sent by the client;

receiving UDP encapsulated ESP packets, the UDP encapsulated ESP packets designating the second destination port.

25. The computer-readable medium of claim 24, wherein the step of determining that at least the client operates behind a NAT comprises:

receiving a discovery payload from the client, the discovery payload including a first value derived from a hash function of a client IP and port address as known to the client.

deriving a second value from a hash function of the client IP and port address as known to the server.

comparing the first value to the second value and determining that the client operates behind the NAT.

26. The computer-readable medium of claim 25 wherein the step of determining that the third IKE packet is sent by the client comprises;

comparing a unique identification within the third IKE packet to a stored value, the stored value identifying a connection between the client and the server.

27. The computer-readable medium of claim 26, wherein the stored valued comprises an I-Cookie value assigned by the client, an R-Cookie value assigned by the server, an IP address of the client and an IP address of the server.