Network gateway mechanism having a protocol stack proxy

• US 6,131,163 A
• Filed: 02/17/1998
• Issued: 10/10/2000
• Est. Priority Date: 02/17/1998
• Status: Expired due to Term

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Wiegel, Scott L.
• Primary Examiner(s): Palys, Joseph E.
• Assistant Examiner(s): ZIEMER, RITA A
• Application Number:
  US09/024,941
• Time in Patent Office: 966 Days
• Field of Search: 713/201, 713/280, 709/223, 709/224, 709/225, 380/23, 380/24, 380/25
• US Class Current: 726/12
• CPC Class Codes: H04L 63/0245 Filtering by information in the payload

Abstract

A system for a network gateway that provides computer data security using a protocol stack proxy is disclosed. The system evaluates data that arrives at a computer system that is executing a network operating system. The system comprises a protocol stack proxy, coupled between a device driver on the computer system that is configured to receive the data from a network and deliver the data according to a first protocol associated with a first network layer, and one or more components of the network operating system that receive packets according to the first protocol. The protocol stack proxy has one or more protocol proxy layers configured to (A) receive the data from the device driver; (B) pass the data to a second network layer that is higher than the first network layer; (C) evaluate the data to determine whether the data satisfies a predetermined criteria; and (D) if the data satisfies the predetermined criteria, to (D1) pass the data to the first network layer, and (D2) transmit the data to the one or more components to the network operating system.

Claims

1. A system for evaluating a data packet using a network operating system, comprising:

a protocol layer proxy stored in a kernel memory and executed in a kernel mode of said network operating system; and

a sequence of instructions stored in said kernel memory and executed in the kernel mode configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy in the kernel mode to determine whether said data packet satisfies a predetermined condition; and

passing said data packet from said protocol layer proxy to a protocol stack that is outside the kernel mode of said network operating system only if said data packet satisfies said predetermined condition.

2. The system recited in claim 1, further comprising an initial security policy defining said condition of said data packet to be evaluated and coupled to said protocol layer proxy.

3. The system recited in claim 2, further comprising:

a second security policy defining a second condition of an application protocol of said data packet to be evaluated; and

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of passing said data packet to said network operating system only if said data packet complies with said second security policy.

4. The system of claim 1, further comprising a second protocol layer proxy for evaluating different data of a second protocol different from a first protocol evaluated by said protocol layer proxy at the same time as said protocol layer proxy, and wherein said protocol layer proxy and said second protocol layer proxy are contained in a protocol proxy stack coupled to said network operating system.

5. The system of claim 1, wherein said protocol layer proxy is configured to evaluate a UDP protocol element of the data packet.

6. The system of claim 1, wherein the protocol layer proxy is configured to evaluate an Internet Control Messaging Protocol (ICMP) protocol element of the data packet.

7. A system for evaluating a data packet using a network operating system, comprising:

a protocol layer proxy in a kernel memory of said network operating system; and

a sequence of instructions stored in said kernel memory configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to said network operating system only if said data packet satisfies said predetermined condition;

an initial security policy defining said condition of said data packet to be evaluated and coupled to said protocol layer proxy;

a security policy decision tree in said kernel memory structured as a binary tree and comprising as nodes thereof said security policy and at least a second security policy.

8. The system recited in claim 7, further comprising at least one existing session identifier stored in said kernel memory;

and wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of;

passing said data packet to said protocol layer proxy only if said data packet matches an existing session identifier.

9. A system for evaluating a data packet using a network operating system comprising:

a protocol layer proxy in a kernel memory of said network operating system; and

a sequence of instructions stored in said kernel memory configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to said network operating system only if said data packet satisfies said predetermined condition;

an initial security policy defining said condition of said data packet to be evaluated and coupled to said protocol layer proxy;

a network adapter card coupled to said processor; and

a security policy decision tree in said kernel memory organized as a binary tree and comprising said security policy and a second security policy associated with said network adapter card.

10. The system recited in claim 9, wherein said security policy decision tree comprises a session identifier associated with a network adapter card identifier and with an allowed protocol for said session identifier and said network adapter card;

and

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of passing said data packet to said protocol layer proxy only if said data packet matches said session identifier.

11. The system of claim 10, wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of passing said data packet to said protocol layer proxy only if said data packet matches said allowed protocol.

12. The system of claim 11, further comprising:

a state variable stored in said protocol layer proxy describing a current state of a current network session.

13. The system of claim 10, further comprising:

a second protocol layer proxy associated with said session identifier; and

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of passing said data packet to said second protocol layer proxy for evaluation therein.

14. The system of claim 9 wherein said protocol layer proxy is coupled to a protocol stack of said network operating system and wherein one of said security policies defines an acceptable criteria for data packets directed to said protocol stack.

15. A system for evaluating a data packet using a network operating system, comprising:

a protocol layer proxy in a kernel memory of said network operating system, and

a sequence of instructions stored in said kernel memory configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to said network operating system only if said data packet satisfies said predetermined condition;

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of identifying said protocol layer proxy as a device driver to said network operating system.

16. A system for evaluating a data packet using a network operating system, comprising:

a protocol layer proxy in a kernel memory of said network operating system; and

a sequence of instructions stored in said kernel memory configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to said network operating system only if said data packet satisfies said predetermined condition;

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of instructing said network operating system that said protocol layer proxy is a network device.

17. A system for evaluating a data packet using a network operating system, comprising:

a protocol layer proxy in a kernel memory of said network operating system; and

a sequence of instructions stored in said kernel memory configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to said network operating system only if said data packet satisfies said predetermined condition;

wherein said sequence of instructions further comprises instructions configured to cause said processor to execute the step of instructing a device driver coupled to said network operating system that said protocol layer proxy is a transport layer for said device driver.

18. A system for evaluating data that arrives at a computer system that is executing a network operating system, comprising:

a protocol stack proxy stored in a kernel memory and executed in a kernel mode of the network operating system and coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

said protocol stack proxy including a proxy layer configured to

(i) receive said data from said device driver;

(ii) pass said data to a second network layer that is higher than said first network layer; and

(iii) evaluate said data to determine whether said data satisfies a predetermined criteria, and if so, pass said data to a protocol stack of the network operating system that is executed outside the kernel mode.

19. The system recited in claim 18, wherein said proxy layer is configured to test whether said data satisfies said predetermined criteria, and if so, then to pass said data to said first network layer and transmit said data to said component of said network operating system.

20. A system for evaluating data that arrives at a computer system that is executing a network operating system, comprising:

a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

said protocol stack proxy including a proxy layer configured to

(i) receive said data from said device driver;

(ii) pass said data to a second network layer that is higher than said first network layer; and

(iii) evaluate said data to determine whether said data satisfies a predetermined criteria;

an application layer proxy in an application memory of said computer system;

a first security policy coupled to said protocol stack proxy and defining said criteria;

a second security policy defining a second condition of an application protocol of said data to be evaluated; and

wherein said protocol stack proxy is configured to pass said data to said application layer proxy only if said data complies with said second security policy.

21. A system for evaluating data that arrives at a computer system that is executing a network operating system, comprising:

a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

said protocol stack proxy including a proxy layer configured to

(i) receive said data from said device driver;

(ii) pass said data to a second network layer that is higher than said first network layer; and

(iii) evaluate said data to determine whether said data satisfies a predetermined criteria;

a security policy decision tree in a kernel memory of said computer system and structured as a binary tree and comprising as nodes thereof said security policy and at least a second security policy.

22. The system recited in claim 21, further comprising at least one existing session identifier stored in said kernel memory;

and wherein said protocol stack proxy is configured to pass said data to said proxy layer only if said data matches an existing session identifier.

23. The system recited in claim 21, further comprising

a network adapter card in said computer system and coupled to said protocol stack proxy;

and

wherein the security policy decision tree comprises a first security policy and a second security policy associated with said network adapter card.

24. The system recited in claim 23, wherein said security policy decision tree comprises a session identifier associated with a network adapter card identifier and with an allowed protocol for said session identifier and said network adapter card;

and

wherein said protocol stack proxy is configured to cause said protocol stack proxy to pass said data to said proxy layer only if said data matches said session identifier.

25. The system of claim 24, wherein said protocol stack proxy is configured to pass said data to said protocol layer proxy only if said data matches said allowed protocol.

26. The system of claim 25, further comprising:

a second proxy layer associated with said session identifier; and

wherein protocol stack proxy is configured to cause said processor to pass said data to said second proxy layer for evaluation therein.

27. The system of claim 26, further comprising:

a state variable stored in said protocol layer proxy and describing a current state of a current network session.

28. A system for evaluating data that arrives at a computer system that is executing a network operating system comprising:

a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

said protocol stack proxy including a proxy layer configured to

(i) receive said data from said device driver;

(ii) pass said data to a second network layer that is higher than said first network layer; and

(iii) evaluate said data to determine whether said data satisfies a predetermined criteria;

wherein said protocol stack proxy is configured to identify said proxy layer as said device driver to said network operating system.

29. A method for evaluating data that arrives at a computer system that is executing a network operating system, comprising the steps of:

establishing in a kernel memory of said computer system a protocol stack proxy that is executed in kernel mode of the network operating system and that is coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

with a proxy layer in said protocol stack proxy, receiving said data from said device driver;

passing said data to a second network layer that is higher than said first network layer;

determining whether said data satisfies a predetermined criteria;

passing said data to a protocol stack of the network operating system that is executed outside the kernel mode when the data satisfies the predetermined criteria.

30. The method recited in claim 29, wherein said computer system comprises a plurality of device drivers and said protocol stack proxy comprises a plurality of proxy layers, each proxy layer associated with one of said device drivers, and wherein the method further comprises the steps of, in each of said proxy layers:

receiving said data from each of said device drivers;

passing said data to said second network layer; and

evaluating said data to determine whether said data satisfies said predetermined criteria.

31. The method recited in claim 29, wherein said computer system further comprises a security policy in said memory and coupled to said protocol stack proxy and defining said criteria.

32. A method for evaluating data that arrives at a computer system that is executing a network operating system, comprising the steps of:

establishing in a memory of said computer system a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

with a proxy layer in said protocol stack proxy, receiving said data from said device driver;

passing said data to a second network layer that is higher than said first network layer;

evaluating said data to determine whether said data satisfies a predetermined criteria;

wherein said computer system further comprises a security policy in said memory and coupled to said protocol stack proxy and defining said criteria;

establishing an application layer proxy in an application memory of said computer system coupled to said protocol stack proxy;

establishing a second security policy in said memory defining a second condition of an application protocol of said data to be evaluated; and

passing said data to said application layer proxy only if said data complies with said second security policy.

33. A method for evaluating data that arrives at a computer system that is executing a network operating system, comprising the steps of:

establishing in a memory of said computer system a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

with a proxy layer in said protocol stack proxy, receiving said data from said device driver;

passing said data to a second network layer that is higher than said first network layer;

evaluating said data to determine whether said data satisfies a predetermined criteria;

wherein said computer system further comprises a security policy in said memory and coupled to said protocol stack proxy and defining said criteria;

establishing a security policy decision tree in a kernel memory of said computer system structured as a binary tree and comprising as nodes thereof said security policy and at least a second security policy.

34. The method recited in claim 33, further comprising the steps of:

establishing at least one existing session identifier stored in said kernel memory; and

passing said data to said proxy layer only if said data matches an existing session identifier.

35. The method recited in claim 33, wherein said computer system further comprises a network adapter card in said computer system and coupled to said protocol stack proxy;

and wherein said method further comprises the step of;

establishing the security policy decision tree comprising a first security policy and a second security policy associated with said network adapter card.

36. The method recited in claim 35, wherein the step of establishing said security policy decision tree includes the steps of:

establishing in said security policy decision tree a session identifier associated with a network adapter card identifier and with an allowed protocol for said session identifier and said network adapter card; and

passing said data to said proxy layer only if said data matches said session identifier.

37. The method of claim 36, wherein said protocol stack proxy is configured to pass said data to said proxy layer only if said data matches said allowed protocol.

38. The method of claim 37, further comprising the steps of:

establishing in said memory a second proxy layer associated with said session identifier; and

evaluating said data in said second proxy layer to determine whether said data conforms to a second protocol associated with said second proxy layer.

39. The method of claim 38, further comprising the step of establishing a state variable stored in said proxy layer and describing a current state of a current network session.

40. A method for evaluating data that arrives at a computer system that is executing a network operating system, comprising the steps of:

establishing in a memory of said computer system a protocol stack proxy coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

with a proxy layer in said protocol stack proxy receiving said data from said device driver;

passing said data to a second network layer that is higher than said first network layer;

evaluating said data to determine whether said data satisfies a predetermined criteria;

further comprising the step of communicating from said protocol stack proxy to said network operating system an identification that said proxy layer is said device driver.

41. A system for evaluating data that arrives at a computer system that is executing a network operating system, comprising:

a protocol stack proxy coupled between a device driver on the computer system that is executed in kernel mode of the network operating system and configured to receive the data from a network and deliver the data according to a first protocol associated with a first network layer; and

one or more components of the network operating system that receive packets according to the first protocol;

the protocol stack proxy including one or more protocol proxy layers, each proxy layer associated with one of the network layers, configured to;

(A) receive the data from the device driver;

(B) pass the data to a proxy layer associated with a second network layer that is higher than the first network layer;

(C) evaluate the data to determine whether the data satisfies a predetermined criteria; and

(D) if the data satisfies the predetermined criteria, to (D1) pass the data to the proxy layer associated with the first network layer, and (D2) transmit the data to the one or more components to the network operating system.

42. A system for evaluating a data packet using a network operating system, comprising:

an application layer proxy in an application memory of said network operating system;

a protocol layer proxy stored in a kernel memory and executed in a kernel mode of said network operating system, and coupled to the application layer proxy; and

a sequence of instructions stored in said kernel memory and executed in the kernel mode configured to cause a processor under control of said network operating system to execute the steps of;

evaluating said data packet in said protocol layer proxy in the kernel mode to determine whether said data packet satisfies a predetermined condition;

passing said data packet from said protocol layer proxy to a protocol stack that is outside the kernel mode of said network operating system only if said data packet satisfies said predetermined condition;

passing the data packet to the application layer proxy when information in the data packet includes a request for an application layer service.

43. A computer-readable medium carrying one or more sequences of instructions for evaluating data that arrives at a computer system that is executing a network operating system, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of:

establishing in a kernel memory of said computer system a protocol stack proxy that is executed in kernel mode of the network operating system and that is coupled between (a) a device driver on said computer system that is configured to receive said data from a network and deliver said data according to a first protocol associated with a first network layer and (b) a component of said network operating system that receives packets according to said first protocol;

with a proxy layer in said protocol stack proxy, receiving said data from said device driver;

passing said data to a second network layer that is higher than said first network layer;

determining whether said data satisfies a predetermined criteria;

passing said data to a protocol stack of the network operating system that is executed outside the kernel mode when the data satisfies the predetermined criteria.

44. A computer-readable medium as recited in claim 43, wherein the step of establishing in a kernel memory of said computer system a protocol stack proxy includes the steps of establishing in a kernel memory of said computer system a protocol stack proxy that comprises a plurality of device drivers and a plurality of proxy layers, each proxy layer associated with one of said device drivers, and wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of, in each of said proxy layers:

receiving said data from each of said device drivers;

passing said data to said second network layer; and

evaluating said data to determine whether said data satisfies said predetermined criteria.

45. The computer-readable medium as recited in claim 43, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

establishing an application layer proxy in an application memory of said computer system coupled to said protocol stack proxy;

establishing a second security policy in said memory defining a second condition of an application protocol of said data to be evaluated; and

passing said data to said application layer proxy only if said data complies with said second security policy.

46. The computer-readable medium as recited in claim 43, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

establishing a security policy decision tree in a kernel memory of said computer system structured as a binary tree and comprising as nodes thereof said security policy and at least a second security policy.

47. The computer-readable medium as recited in claim 46, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

establishing in said security policy decision tree a session identifier associated with a network adapter card identifier and with an allowed protocol for said session identifier and said network adapter card; and

passing said data to said proxy layer only if said data matches said session identifier.

48. The computer-readable medium as recited in claim 43, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

establishing at least one existing session identifier stored in said kernel memory; and

passing said data to said proxy layer only if said data matches an existing session identifier.

49. The computer-readable medium as recited in claim 43, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

establishing the security policy decision tree in a kernel memory of said computer system organized as a binary tree and comprising a first security policy and a second security policy associated with a network adapter card in said computer system and coupled to said protocol stack proxy.

50. The computer-readable medium as recited in claim 43, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the further steps of:

communicating from said protocol stack proxy to said network operating system an identification that said proxy layer is said device driver.