Method and system for tracking access to application data and preventing data exploitation by malicious programs

• US 20070240215A1
• Filed: 03/28/2006
• Published: 10/11/2007
• Est. Priority Date: 03/28/2006
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Blue Coat Systems Incorporated (Gen Digital Inc.)
• Inventors: Lu, Wei, Flores, Jose, Kaplan, Yariv, Blewer, Ronnie
• Granted Patent: US 9,171,157 B2
• Time in Patent Office: Days
• US Class Current: 726/24
• CPC Class Codes: G06F 21/55 Detecting local intrusion or implementing counter-measures
• CPC Class Codes: G06F 21/56 Computer malware detection or handling, e.g. anti-virus arrangements
• CPC Class Codes: G06F 21/566 Dynamic detection, i.e. detection performed at run-time, e.g. emulation, suspicious activities

Abstract

Provided are a method and system for tracking access to application data and preventing data exploitation by malicious programs. In one example, the method includes shimming into a running process of the system to create at least one monitoring hook to monitor a program, building an execution path of the monitored program, and monitoring a behavior of the execution path for malicious behavior using the monitoring hook.

Claims

1. A method of detecting a malicious program on a computer system, the method comprising:

shimming into a running process of the system to create at least one monitoring hook to monitor a program;

building at least one execution path of the monitored program; and

monitoring a behavior of the at least one execution path for malicious behavior using the monitoring hook.

2. The method of claim 1, further comprising:

determining if the behavior of the at least one execution path generates malicious behavior based on at least one behavior rule; and

if the behavior of the execution path generates malicious behavior, associating the behavior of the execution path with a potentially malicious instance.

3. The method of claim 2, further comprising:

determining if the potentially malicious instance communicates with at least one operating system interface function; and

if the potentially malicious instance communicates with at least one operating system interface function, monitoring additional activity of the at least one operating system interface.

4. The method of claim 3, further comprising:

determining if the additional activity of the at least one operating system interface is invoked to generate malicious behavior based on at least one behavior rule; and

if the additional activity of the at least one operating system interface is invoked to generate malicious behavior, increasing a weighting associated with the monitored program.

5. The method of claim 4, further comprising:

determining if the potentially malicious instance interacts with at least one other program module; and

if the potentially malicious instance interacts with at least one other program module, monitoring a behavior of the at least one other program module for malicious behavior based on the at least one behavior rule.

6. The method of claim 2, further comprising:

determining if the potentially malicious instance accesses an operating system resource;

if the potentially malicious instance accesses the operating system resource, increasing a weighting associated with the monitored program.

7. The method of claim 4, further comprising:

determining if the weighting of the monitored program reaches a predefined threshold of infection; and

if the weighting of the monitored program reaches the predefined threshold of infection, taking a predefined action with respect to the monitored program.

8. The method of claim 4 further comprising:

examining information associated with a module of the monitored program;

determining if the module is legitimate based on the examined information; and

modifying the weighting of the monitored program if the module is legitimate.

9. The method of claim 8 wherein the modifying includes lowering the weighting of the monitored program below the predefined threshold of infection.

10. The method of claim 1, wherein the shimming comprises:

scanning a virtual memory of the computer system to locate an operating system message dispatcher function; and

replacing the operating system message dispatcher function with a detection program dispatcher function.

11. The method of claim 10, wherein the shimming further comprises:

scanning the virtual memory of the computer system to locate a function of interest; and

replacing the function of interest with a detection program monitoring function, wherein the detection program monitoring function monitors for more than one malicious program simultaneously.

12. The method of claim 1, wherein the monitoring comprises:

identifying a list of programs in the at least one execution path capable of receiving operating system messages;

generating at least one dummy activity; and

recording a subset of programs from the list of programs triggered by the at least one dummy activity, wherein the monitored program is in the subset of programs.

13. The method of claim 12, wherein the monitoring further comprises:

determining if a program in the subset of programs has a valid signature; and

if the program in the subset of programs has a valid signature, removing the program from the subset of programs.

14. The method of claim 13, wherein the monitoring further comprises:

monitoring any remaining programs in the subset of programs for communication with other programs; and

placing a program of the remaining programs on a watch list if the program communicates with other programs.

15. The method of claim I, wherein the building comprises:

responsive to receiving input from a user of the computer system, identifying a plurality of state transitions in a logical state machine triggered by the input;

identifying a sequence of calls to functions and application programming interfaces triggered by the plurality of state transitions; and

building the at least one execution path based on the sequence of calls.

16. A method of monitoring a program to detect malicious activity, the method comprising:

replacing an operating system message dispatcher function with a detection program dispatcher function;

detecting at least one function of interest using the detection program dispatcher function;

replacing the at least one function of interest with a monitoring function;

identifying a sequence of calls to functions and application programming interfaces triggered by the program'"'"'s execution;

building at least one execution path based on the sequence of calls; and

monitoring for malicious behavior in the at least one execution path using the monitoring function.

17. The method of claim 16 wherein the monitoring comprises:

applying at least one predefined rule to the execution path to determine if the execution path generates malicious behavior; and

if the behavior of the execution path generates malicious behavior, determining if the malicious behavior includes communicating with an operating system interface function.

18. The method of claim 17 further comprising:

if the malicious behavior includes communicating with an operating system interface function, monitoring additional activity of the operating system interface to determine if the operating system interface is invoked by a malicious program to generate malicious behavior; and

if the operating system interface is invoked to generate malicious behavior, increasing a weighting associated with the program.

19. The method of claim 16, wherein the monitoring comprises:

applying at least one predefined rule to the execution path to determine if the execution path generates malicious behavior;

if the behavior of the execution path generates malicious behavior, determining if the malicious behavior includes accessing an operating system resource; and

if the malicious behavior includes accessing an operating system resource, increasing a weighting associated with the program.

20. The method of claim 18 further comprising:

determining if the weighting of the program reaches a predefined threshold; and

taking a predefined action with respect to the program if the weighting reaches the predefined threshold.

21. The method of claim 20 further comprising:

examining information associated with a module of the program;

determining if the module is legitimate based on the examined information; and

lowering the weighting of the program if the module is legitimate.

22. The method of claim 16, wherein applying the predefined rule includes examining the execution path to ensure that an operating system message processing function is never called, a function that processes key messages is never called, a call to a function that stores the key messages is never made, and a function that forwards stored messages is never called.

23. The method of claim 16, wherein applying the predefined rule includes examining the execution path to ensure that only modules that are either common to all programs or specific to a program located in the program'"'"'s installation directory are loaded, only files that are either common to all programs or specific to a program located in the program'"'"'s installation directory are accessed, system configuration files that belong to an operating system are never modified, files belonging to other programs are never deleted, and multiple connections to different machines within a short period of time are never made.

24. The method of claim 16, wherein applying the predefined rule includes examining the execution path to ensure that only requests to a specific website are generated, and only site- specific objects of a browser are used when the browser is connected to a website.

25. A malicious program detection system comprising:

a client; and

a server coupled to the client via a network, wherein the server includes a processor and a memory for executing and storing a detection program, respectively, the detection program configured to perform the steps of;

shimming into a running process of the client to create monitoring hooks in response to detecting a user login from the client;

building execution paths of a program to be monitored; and

monitoring behavior of the execution paths for malicious behavior using the monitoring hooks.