System, method and program product for detecting malicious software

• US 20050091558A1
• Filed: 10/28/2003
• Published: 04/28/2005
• Est. Priority Date: 10/28/2003
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Luke, James S., Chess, David M.
• Application Number:
  US10/696,200
• Publication Number:
  US 20050091558A1
• Time in Patent Office: Days
• US Class Current: 714/38.13
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 21/54 by adding security routines or objects to programs
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities

Abstract

System, method and program product for detecting malicious software within or attacking a computer system. In response to a system call, a hook routine is executed at a location of the system call to (a) determine a data flow or process requested by the call, (b) determine another data flow or process for data related to that of the call, (c) automatically generate a consolidated information flow diagram showing the data flow or process of the call and the other data flow or process. After steps (a-c), a routine is called to perform the data flow or process requested by the call. A user monitors the information flow diagram and compares the data flow or process of steps (a) and (b) with a data flow or process expected by said user. If there are differences, the user may investigate the matter or shut down the computer to prevent damage.

Claims

1. A method for detecting malicious software within or attacking a computer system, said method comprising the steps of:

in response to a system call, executing a hook routine at a location of said system call to (a) determine a data flow or process requested by said call, (b) determine another data flow or process for data related to that of said call, (c) automatically generate a consolidated information flow diagram showing said data flow or process of said call and said other data flow or process, and after steps (a-c), (d) call a routine to perform said data flow or process requested by said call.

2. A method as set forth in claim 1, wherein a user monitors said information flow diagram and compares the data flow or process of steps (a) and (b) with a data flow or process expected by said user.

3. A method as set forth in claim 1, wherein said information flow diagram illustrates locations of said data at stages of a processing activity.

4. A method as set forth in claim 1, wherein said system call is selected from the set of:

open file, copy file to memory, copy memory to register, mathematical functions, write to file, and network or communication functions.

5. A method as set forth in claim 1, wherein said system call is a software interrupt of an operating system.

6. A method as set forth in claim 1, wherein said system call causes a processor to stop its current activity and execute said hook routine.

7. A method as set forth in claim 1 wherein said system call is made by malicious software.

8. A system for detecting malicious software in a computer system, said system comprising:

means, responsive to a system call, for executing a hook routine at a location of said system call to (a) determine a data flow or process requested by said call, (b) determine another data flow or process for data related to that of said call, (c) automatically generate a consolidated information flow diagram showing said data flow or process of said call and said other data flow or process, and after steps (a-c), (d) call a routine to perform said data flow or process requested by said call; and

means for displaying said information flow diagram.

9. A system as set forth in claim 8, wherein said information flow diagram illustrates locations of said data at stages of a processing activity.

10. A system as set forth in claim 8, wherein said system call is selected from the set of:

open file, copy file to memory, copy memory to register, mathematical functions, write to file, and network or communication functions.

11. A system as set forth in claim 8, wherein said system call is a software interrupt of an operating system.

12. A system as set forth in claim 8, wherein said system call causes a processor to stop its current activity and execute said hook routine.

13. A system as set forth in claim 8 wherein said system call is made by malicious software.

14. A computer program product for detecting malicious software in a computer system, said computer program product comprising:

a computer readable medium;

program instructions, responsive to a system call, for executing a hook routine at a location of said system call to (a) determine a data flow or process requested by said call, (b) determine another data flow or process for data related to that of said call, (c) automatically generate a consolidated information flow diagram showing said data flow or process of said call and said other data flow or process, and after steps (a-c), (d) call a routine to perform said data flow or process requested by said call; and

wherein

said program instructions are recorded on said medium.