Systems and methods for implementing protocol enforcement rules

• US 7,818,565 B2
• Filed: 06/10/2003
• Issued: 10/19/2010
• Est. Priority Date: 06/10/2002
• Status: Active Grant

• Current Assignee: Quest Software, Inc.
• Original Assignee: Quest Software, Inc.
• Inventors: Shapiro, Dmitry, Miller, Randy, Poling, Robert, Pugh, Richard S.
• Primary Examiner(s): Vu; Kimyen
• Assistant Examiner(s): SHAN, APRIL YING
• Application Number:
  US10/459,111
• Publication Number:
  US 20040103318A1
• Time in Patent Office: 2,688 Days
• Field of Search: 713/153, 713/150, 713/151, 713/152, 713/154, 726/14, 709/207
• US Class Current: 713/153
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 63/0281 Proxies
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 69/329 in the application layer [OSI layer 7]

Abstract

A protocol management system is capable of detecting certain message protocols and applying policy rules to the detected message protocols that prevent intrusion, or abuse, of a network'"'"'s resources. In one aspect, a protocol message gateway is configured to apply policy rules to high level message protocols, such as those that reside at layer 7 of the ISO protocol stack.

Claims

1. A method for managing one or more communication protocols, the method comprising:

providing a protocol message gateway in an enterprise network, the protocol message gateway comprising a proxy server, the protocol message gateway operative to communicate with a firewall of the enterprise network;

receiving first messages with the protocol message gateway over the enterprise network, the protocol message gateway further operative to determine whether selected ones of the first messages use a target protocol and, in response to said determination, implement policy rules associated with the target protocol;

providing a proxy enforcer within the enterprise network operative to passively listen for second messages that bypass the protocol message gateway;

detecting a selected second message with the proxy enforcer, said detecting comprising determining that a message protocol associated with the selected second message matches an instant messaging protocol definition file;

in response to detecting the selected second message, using the proxy enforcer to force the selected second message to use a defined port on the protocol message gateway; and

using the protocol message gateway to implement at least one of the policy rules in association with the selected second message.

2. The method of claim 1, wherein using the protocol message gateway to implement at least one of the policy rules comprises terminating a communication connection associated with the selected second message.

3. The method of claim 1, wherein using the protocol message gateway to implement at least one of the policy rules comprises resetting a communication connection associated with the selected second message.

4. The method of claim 1, wherein determining that a message protocol associated with the selected second message matches an instant messaging protocol definition file comprises determining whether the selected second message includes a string of five characters in an incoming packet header, wherein the five characters represent a signature of the instant messaging protocol.

5. The method of claim 1, wherein using the protocol message gateway to implement at least one of the policy rules comprises creating a log comprising information associated with the selected second message and any related messages.

6. The method of claim 1, wherein determining that a message protocol associated with the selected second message matches an instant messaging protocol definition file comprises determining whether the selected second message is directed to port 8080 or port 5190.

7. The method of claim 1, wherein determining that a message protocol associated with the selected second message matches an instant messaging protocol definition file comprises determining whether the selected second message includes a string “

?message=”

.

8. The method of claim 1, wherein determining that a message protocol associated with the selected second message matches an instant messaging protocol definition file comprises determining whether the selected second message includes a string “

?pword=%1”

.

9. A protocol enforcer, comprising:

a protocol definition file, the protocol definition file being associated with an instant messaging protocol; and

a network interface configured to interface the protocol enforcer with an enterprise network, the protocol enforcer configured for;

passively listening, via the network interface, for messages that bypass a protocol message gateway within the enterprise network,

detecting a selected message in response to said passive listening, said detecting comprising determining when the message protocol matches the protocol definition file, and

in response to detecting the selected message, performing at least one of;

forcing the selected message to use a defined communication connection on a protocol message gateway, and

applying a policy enforcement rule associated with the protocol definition file that terminates a communication connection associated with the selected message.

10. The protocol enforcer of claim 9, wherein determining when the message protocol matches the protocol definition file comprises determining if the selected message includes a string of five characters in an incoming packet header, wherein the five characters represent a signature of the instant messaging protocol.

11. The protocol enforcer of claim 9, wherein determining when the message protocol matches the protocol definition file comprises determining if the selected message is directed to port 5190.

12. The protocol enforcer of claim 9, wherein determining when the message protocol matches the protocol definition file comprises determining if the selected message includes a string ?message=.

13. The protocol enforcer of claim 9, wherein applying the policy enforcement rule comprises creating a log comprising information associated with the selected message and any related messages.

14. The protocol enforcer of claim 9, wherein determining when the message protocol matches the protocol definition file comprises determining if the selected message includes a string ?pword=%1.

15. A protocol management system, comprising:

a protocol message gateway comprising a proxy server, the protocol message gateway operative to;

receive first messages from one or more client devices over an enterprise network,

determine whether selected ones of the first messages use a first target protocol, and

in response to said determination, implement first policy rules associated with the first target protocol; and

a protocol enforcer, the protocol enforcer comprising;

a protocol definition file, and

a network interface configured to interface the protocol enforcer with the enterprise network,

the protocol enforcer configured for;

detecting a second message that bypasses the protocol message gateway, said detecting comprising inspecting a message protocol associated with the second message to determine if the message protocol is an instant messaging protocol, and

in response to detecting the second message, forcing the second message to use a defined communication connection on the protocol message gateway.

16. The protocol management system of claim 15, wherein the protocol enforcer is further configured to apply a policy enforcement rule in association with the second message, said applying comprising terminating a communication connection associated with the second message.

17. The protocol management system of claim 15, wherein inspecting a message protocol associated with the second message comprises determining whether the selected message includes a string of five characters in an incoming packet header, wherein the five characters represent a signature of the instant messaging protocol.

18. The protocol management system of claim 15, wherein the protocol enforcer is further configured to apply a policy enforcement rule in association with the second message, said applying comprising recording information associated with the second message.

19. The protocol management system of claim 18, further comprising a storage medium, and wherein applying the policy enforcement rule further comprises creating a log comprising information associated with the second message and any related messages and storing the log in the storage medium.

20. The protocol management system of claim 15, wherein inspecting a message protocol associated with the second message comprises determining whether the selected message is directed to port 8080 or port 5190.

21. The protocol management system of claim 15, wherein inspecting a message protocol associated with the second message comprises determining whether the selected message includes a string “

?message=”

.

22. The protocol management system of claim 15, wherein inspecting a message protocol associated with the second message comprises determining whether the selected message includes a string “

?pword=%1”

.