Detecting and countering malicious code in enterprise networks

• US 7,934,103 B2
• Filed: 04/15/2003
• Issued: 04/26/2011
• Est. Priority Date: 04/17/2002
• Status: Expired due to Fees

• Current Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Original Assignee: Computer Associates Think Inc. (Broadcom, Inc.)
• Inventors: Kidron, Yaron
• Primary Examiner(s): Orgad; Edan
• Assistant Examiner(s): Jackson; Jenise E
• Application Number:
  US10/414,117
• Publication Number:
  US 20030200464A1
• Time in Patent Office: 2,933 Days
• Field of Search: 726/2, 726 22- 25, 713/188
• US Class Current: 713/188
• CPC Class Codes: H04L 63/0218 Distributed architectures, e.g. distributed firewalls
• CPC Class Codes: H04L 63/145 the attack involving the propagation of malware through the network, e.g. viruses, trojans or worms

Abstract

A system and method for detecting and countering malicious code in an enterprise network are provided. A pattern recognition processor monitors local operations on a plurality of local machines connected through an enterprise network, to detect irregular local behavior patterns. An alert may be generated after an irregularity in behavior pattern on a local machine is detected. Irregular behavior alerts from a plurality of local machines are analyzed. If similar alerts are received from at least a threshold number of local machines over a corresponding period of time, one or more countermeasure operations are selected based on the analysis of the irregular behavior alerts. The selected countermeasure operations are communicated to the local machines and performed by the local machines.

Claims

1. A system for detecting and countering malicious code in an enterprise network, comprising:

a server; and

a plurality of local machines connected to the server through the enterprise network, each local machine comprising a pattern recognition processor, the pattern recognition processor monitoring local operations to detect irregular local behavior patterns, and generating an alert after an irregularity in local behavior pattern is detected,

wherein the server monitors for and analyzes irregular behavior alerts from the plurality of local machines, and, if similar alerts are received from at least a threshold number of local machines over a corresponding period of time, the server selects one or more countermeasure operations based on the analysis of the irregular behavior alerts and communicates to the local machines the selected countermeasure operations to be performed by the local machines, wherein a countermeasure operation communicated by the server to the local machines is identified by utility name.

17. The system of claim 1, wherein the pattern recognition processor monitors calls to the local operating system.

18. The system of claim 17, wherein each local machine further comprises a signal monitor, and the signal monitor maintains a log of local operating system calls.

19. The system of claim 1, wherein each local machine further comprises a remote control core including a network relay for communication with the server, the irregular behavior alert is communicated from the local machine through the network relay to the server, and remote control instructions are received by the local machine through the network relay.

20. The system of claim 1 further comprising a cluster manager, wherein the irregular behavior alerts are communicated from the plurality of local machines through the cluster manager to the server.

21. The system of claim 1, wherein the server is a dedicated system for monitoring suspicious activity in the enterprise network.

22. The system of claim 1, wherein the countermeasure operations include a notification to enterprise-wide administration utilities.

23. The system of claim 1, wherein the countermeasure operations include an instruction to the local machines to shutdown one or more local functionalities associated with the irregular behavior alerts.

2. A method of detecting and countering malicious code in an enterprise network system having a server and a plurality of local machines, comprising:

monitoring local operations at each local machine to detect irregular local behavior patterns, and, if an irregularity in the local behavior pattern is detected at the local machine, generating an irregular behavior alert from the local machine to the server;

analyzing at the server irregular behavior alerts received from the local machines;

determining whether a threshold number of similar irregular behavior alerts has been exceeded over a predefined period of time beginning with the reception of a first of the similar alerts; and

in response to the determining, selecting one or more countermeasure operations based on the analysis of the irregular behavior alerts and communicating the selected countermeasure operations to the local machines, wherein a countermeasure operation communicated by the server to the local machines is identified by utility name.

3. The method of claim 2 further comprising monitoring calls to the local operating system.

4. The method of claim 3 further comprising maintaining a log of the local operating system calls.

5. The method of claim 2, wherein the countermeasure operations include a notification to enterprise-wide administration utilities.

6. The method of claim 2, wherein the countermeasure operations include an instruction to the local machines to shutdown one or more local functionalities associated with the irregular behavior alerts.

7. The method of claim 2, wherein a countermeasure operation communicated by the server to the local machines is identified by library name and function call.

8. A system for detecting and countering malicious code in an enterprise network, comprising:

a server; and

a plurality of local machines connected to the server through the enterprise network, each local machine comprising a pattern recognition processor, the pattern recognition processor monitoring local operations to detect irregular local behavior patterns, and generating an irregular behavior alert after an irregularity in local behavior pattern is detected,

wherein the server is operable to;

monitors for and analyze irregular behavior alerts received from the plurality of local machines;

determine whether a threshold number of similar irregular behavior alerts has been exceeded over a predefined period of time beginning with the reception of a first of the similar alerts; and

in response to the determining, selecting one or more countermeasure operations based on the analysis of the irregular behavior alerts and communicating to the local machines the selected countermeasure operations to be performed by the local machines, wherein a countermeasure operation communicated by the server to the local machines is identified by library name and function call.

9. The system of claim 8, wherein the pattern recognition processor monitors calls to the local operating system.

10. The system of claim 9, wherein each local machine further comprises a signal monitor, and the signal monitor maintains a log of local operating system calls.

11. The system of claim 8, wherein each local machine further comprises a remote control core including a network relay for communication with the server, the irregular behavior alert is communicated from the local machine through the network relay to the server, and remote control instructions are received by the local machine through the network relay.

12. The system of claim 8 further comprising a cluster manager, wherein the irregular behavior alerts are communicated from the plurality of local machines through the cluster manager to the server.

13. The system of claim 8, wherein the server is a dedicated system for monitoring suspicious activity in the enterprise network.

14. The system of claim 8, wherein the countermeasure operations include a notification to enterprise-wide administration utilities.

15. The system of claim 8, wherein the countermeasure operations include an instruction to the local machines to shutdown one or more local functionalities associated with the irregular behavior alerts.

16. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to:

monitor local operations at each local machine to detect irregular local behavior patterns, and, if an irregularity in the local behavior pattern is detected at the local machine, generating an irregular behavior alert from the local machine to the server;

analyze at the server irregular behavior alerts received from the local machines;

determine whether a threshold number of similar irregular behavior alerts has been exceeded over a predefined period of time beginning with the reception of a first of the similar alerts; and

in response to the determining, select one or more countermeasure operations based on the analysis of the irregular behavior alerts and communicating the selected countermeasure operations to the local machines, wherein a countermeasure operation communicated by the server to the local machines is identified by utility name.

24. A system comprising:

a processor; and

a program storage device readable by the system, tangibly embodying a program of instructions executable by the machine to;

monitor local operations at each local machine to detect irregular local behavior patterns, and, if an irregularity in the local behavior pattern is detected at the local machine, generating an irregular behavior alert from the local machine to the server;

analyze at the server irregular behavior alerts received from the local machines;

determine whether a threshold number of similar irregular behavior alerts has been exceeded over a predefined period of time beginning with the reception of a first of the similar alerts; and

in response to the determining, select one or more countermeasure operations based on the analysis of the irregular behavior alerts and communicating the selected countermeasure operations to the local machines, wherein a countermeasure operation communicated by the server to the local machines is identified by utility name.