SMTP network security processing in a transparent relay in a computer network

• US 20070204341A1
• Filed: 11/17/2006
• Published: 08/30/2007
• Est. Priority Date: 11/23/2005
• Status: Active Grant

• Current Assignee: Trend Micro Inc.
• Original Assignee: Trend Micro Inc.
• Inventors: Rand, David, Moriarty, Paul, Esters, Scott, Scharf, Gerald
• Granted Patent: US 7,926,108 B2
• Time in Patent Office: Days
• US Class Current: 726/22
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: H04L 51/212 using filtering or selective blocking

Abstract

In one embodiment, a transparent relay receives diverted e-mail communications between an e-mail client and an e-mail server. The transparent relay may be configured to examine the e-mail communications for network security policy violations. E-mail communications that do not violate a network security policy may be relayed to their intended destination. Policy actions, such as discarding or redirection, may be performed on those that violate one or more network security policies. The transparent relay may include a pair of communications interfaces running in promiscuous mode, one for downstream communications and another for upstream communications. The transparent relay may decompose a network communication protocol to look network security policy violations.

Claims

1. A network security system for processing e-mail transactions, the system comprising:

an e-mail sever;

an e-mail client;

a transparent relay configured to receive and process e-mail communications between the e-mail client and the e-mail server, the transparent relay being configured to examine e-mail communications for network security policy violations, to perform policy actions on particular e-mail communications that violate a network security policy, and to relay particular e-mail communications that do not violate a network security policy; and

a router configured to divert to the transparent relay e-mail communications between the e-mail client and the e-mail server.

2. The network security system of claim 1 wherein the transparent relay includes a communications interface for each of the e-mail server and the e-mail client running in promiscuous mode to allow the transparent relay to receive and process data packets not having a destination IP address corresponding to any of the transparent relay.

3. The network security system of claim 1 wherein the transparent relay comprises an SMTP transparent relay and the e-mail server comprises a mail transfer agent.

4. The network security system of claim 1 wherein the e-mail server and the e-mail client communicate using the SMTP protocol.

5. The network security system of claim 1 wherein the transparent relay is configured to receive SMTP commands from the e-mail client and examine the SMTP commands for policy violations, perform policy actions on an SMTP command that violates a policy, and relay an SMTP command that do not violate a policy to the e-mail server.

6. The network security system of claim 1 wherein the transparent relay is configured to examine TCP packets initiating an e-mail connection between the e-mail client and the e-mail server for network security policy violations.

7. The network security system of claim 1 wherein the transparent relay is configured to examine SMTP responses from the e-mail server for network security policy violations.

8. The network security system of claim 1 wherein the policy actions include redirecting an e-mail communication to another computer.

9. A method of processing e-mail communications for network security, the method comprising:

transparently receiving diverted e-mail packets originated by an e-mail client to be sent to an e-mail server;

checking the diverted e-mail packets originated by the e-mail client for connection initiation packets configured to initiate an e-mail connection between the e-mail client and the e-mail server;

determining whether the connection initiation packets violate a first policy in a plurality of network security policies; and

performing a first policy action on the connection initiation packets if the connection initiation packets violate the first policy.

10. The method of claim 9 wherein the first policy action comprises redirecting the connection initiation packets to another computer configured to analyze the connection initiation packets for network security threats.

11. The method of claim 9 further comprising:

checking the diverted e-mail packets originated by the e-mail client for SMTP command packets;

determining whether the SMTP command packets violate a second policy in the plurality of network security policies; and

performing a second policy action on the SMTP command packets if the SMTP command packets violate the second policy.

12. The method of claim 11 wherein the second policy action comprises sending an error message to the e-mail client.

13. The method of claim 9 further comprising:

transparently receiving diverted e-mail packets originated by the e-mail server to be sent to the e-mail client;

checking the diverted e-mail packets originated by the e-mail server for SMTP response packets;

determining whether the SMTP response packets violate a third policy in the plurality of network security policies; and

performing a third policy action on the SMTP response packets if the SMTP response packets violate the third policy.

14. The method of claim 13 wherein the third policy comprises a prohibition against malformed SMTP response packets to block covert channels and the third policy action comprises blocking the SMTP response packet.

15. A method of processing computer communications for network security, the method comprising:

transparently receiving diverted packets between a client computer and a server computer communicating over a communication session in accordance with a communication protocol;

monitoring the communication session at different states of the communication protocol to check for network security policy violations; and

relaying communications between the client computer and the server computer when the monitoring of the communication session does not indicate a network security policy violation.

16. The method of claim 15 wherein the communication protocol comprises SMTP.