Method and apparatus for providing secure streaming data transmission facilities using unreliable protocols

• US 7,353,380 B2
• Filed: 02/12/2001
• Issued: 04/01/2008
• Est. Priority Date: 02/12/2001
• Status: Expired due to Fees

• Current Assignee: Aventail LLC
• Original Assignee: Aventail LLC A Subsidiary of Sonicwall Incorporated
• Inventors: VanHeyningen, Marc D.
• Primary Examiner(s): Vu; Kim
• Assistant Examiner(s): SHAW, YIN CHEN
• Application Number:
  US09/782,593
• Publication Number:
  US 20030023845A1
• Time in Patent Office: 2,605 Days
• Field of Search: 713150-151, 713/201, 380/287, 380/262, 380/44, 380/28, 726/26
• US Class Current: 713/150
• CPC Class Codes: H04L 2209/76 Proxy, i.e. using intermediary entity to perform cryptographic operations
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/166 at the transport layer
• CPC Class Codes: H04L 9/0643 Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
• CPC Class Codes: H04L 9/3242 involving keyed hash functions, e.g. message authentication codes [MACs], CBC-MAC or HMAC

Abstract

The invention provides a method and apparatus for transmitting data securely using an unreliable communication protocol, such as User Datagram Protocol. In one variation, the invention retains compatibility with conventional Secure Sockets Layer (SSL) and SOCKS protocols, such that secure UDP datagrams can be transmitted between a proxy server and a client computer in a manner analogous to conventional SOCKS processing. In contrast to conventional SSL processing, which relies on a guaranteed delivery service such as TCP and encrypts successive data records with reference to a previously-transmitted data record, encryption is performed using a nonce that is embedded in each transmitted data record. This nonce acts both as an initialization vector for encryption/decryption of the record, and as a unique identifier to authenticate the record. Because decryption of any particular record does not rely on receipt of a previously received data record, the scheme will operate over an unreliable communication protocol. The system and method allows secure packet transmission to be provided with a minimum amount of overhead.

Claims

1. A method of transmitting data securely over a computer network, comprising the steps of:

(1) establishing a communication path between a first computer and a second computer;

(2) encrypting and transmitting data records between the first computer and the second computer using an unreliable communication protocol, wherein each data record incorporates a nonce and encrypted text that has been encrypted using the nonce and a shared encryption key and without reference to a previously transmitted data record; and

(3) in the second computer, receiving and decrypting the data records transmitted in step (2) by, for each of the received data records, decrypting the incorporated encrypted text using the incorporated nonce in combination with the shared encryption key and without reference to a previously received data record,

wherein step (2) comprises the step of embedding an indicator in each of the data records indicating that the data records are encrypted according to an encryption scheme that encrypts records without regard to any previously transmitted data records, and

wherein step (3) comprises the step of determining whether the indicator is present in each record and, in response to determining that the indicator is not present, processing each such record differently than if the indicator is set.

2. The method of claim 1, wherein step (1) is performed using the Transmission Control Protocol, and wherein step (2) is performed using the User Datagram Protocol.

3. The method of claim 1, wherein step (2) is performed by a proxy server that encrypts data records received from another server.

4. A system for securely transmitting data using an unreliable protocol, comprising:

a first computer comprising a communication protocol client function operable in conjunction with an application program to transmit data records securely using an unreliable protocol; and

a second computer coupled to the first computer and comprising a communication protocol server function operable in conjunction with the communication protocol client function to receive data records securely using the unreliable communication protocol,

wherein, for each data record, the communication protocol client function encrypts text for the data record using a nonce and an encryption key and incorporates the respective encrypted text and nonce in the data record;

wherein the communication protocol server function decrypts the encrypted text in each of the data records using the respectively appended nonce and the encryption key; and

wherein the second computer comprises a record detector that determines whether an indicator has been set in each data record received from the first computer and, if the indicator has not been set, bypasses decryption in the server computer.

5. The system of claim 4, wherein the communication protocol client function exchanges encryption credentials with the communication protocol server function using a reliable communication protocol.

6. The system of claim 5, wherein the unreliable communication protocol comprises the User Datagram Protocol, and wherein the reliable communication protocol comprises the Transmission Control Protocol.

7. The system of claim 4, wherein the communication protocol client function and the communication protocol server function are compatible with the SOCKS communication protocol.

8. The system of claim 4, wherein the communication protocol client function and the communication protocol server function are compatible with the SSL/TLS communication protocol.

9. The system of claim 4, wherein the second computer comprises a proxy server that forwards the decrypted text to a server computer.

10. A method of transmitting data securely over a computer network, comprising:

establishing a communication path with a remote computer;

encrypting data records using a nonce and a shared encryption key such that each data record incorporates the nonce, and

text that is encrypted such that the remote computer can decrypt the encrypted text by using the incorporated nonce in combination with the shared encryption key and without reference to a previously received data record; and

transmitting the encrypted data records to the remote computer using an unreliable communication protocol,

wherein encrypting the data records includes embedding an indicator in each of the data records indicating that the data record incorporates text encrypted according to an encryption scheme that encrypts text without regard to any previously transmitted data records, such that the remote computer can determine whether the indicator is present in each received data record and, in response to determining that the indicator is not present, process each such received data record differently than if the indicator is set.

11. The method of claim 10, further comprising establishing a reliable communication path to the remote computer and exchanging security credentials with the remote computer over the reliable communication path.

12. The method of claim 11, wherein the step of exchanging security credentials includes exchanging an encryption key that is used to encrypt the text.

13. The method of claim 10, wherein the nonce includes a random number.

14. The method of claim 10, wherein

establishing the communication path with the remote computer is performed using the Transmission Control Protocol;

and

encrypting data records is performed using the User Datagram Protocol.

15. The method of claim 10, wherein encrypting the data records is performed by a proxy server that encrypts text received from another server.

16. A method of transmitting data securely over a computer network, comprising:

establishing a communication path with a remote computer;

receiving data records

transmitted from the remote computer using an unreliable communication protocol, and

encrypted using a nonce and a shared encryption key such that

each data record incorporates a nonce and

text that is encrypted without reference to a previously encrypted data record; and

decrypting the received data records by using the nonce in combination with the shared encryption key to decrypt each received data record without reference to a previously received data record,

wherein the received encrypted data records include an indicator indicating that the data records incorporate text that has been encrypted according to an encryption scheme that encrypts records without regard to any previously transmitted data records, and

further comprising determining whether the indicator is present in each received data record and, in response to determining that the indicator is not present in a received data record, processing such received data record differently than if the indicator is set.

17. The method of claim 16, wherein

establishing a communication path with a remote computer is performed using the Transmission Control Protocol, and

receiving the encrypted data records is performed using the User Datagram Protocol.

18. The method of claim 16, wherein the received data records are received from a proxy server that encrypts data records the proxy server received from another server.