Method and system for intercepting an application program interface

• US 6,823,460 B1
• Filed: 04/28/2000
• Issued: 11/23/2004
• Est. Priority Date: 11/14/1999
• Status: Expired due to Term

• Current Assignee: McAfee, LLC
• Original Assignee: Networks Associates Technology, Inc. (McAfee, LLC)
• Inventors: Horovitz, Oded, Rachman, Ophir, Hollander, Yona
• Primary Examiner(s): Sheikh, Ayaz
• Assistant Examiner(s): ARANI, TAGHI T
• Application Number:
  US09/561,395
• Time in Patent Office: 1,670 Days
• Field of Search: 713/200, 710/260
• US Class Current: 726/3
• CPC Class Codes: G06F 12/1441 for a range
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 2209/542 Intercept
• CPC Class Codes: G06F 9/4484 Executing subprograms

Abstract

A method of intercepting application program interface, including dynamic installation of associated software, within the user portion of an operating system. An API interception control server in conjunction with a system call interception module loads into all active process spaces an API interception module. An initializer module within the API interception module hooks and patches all API modules in the active process address space. When called by the application programs, the API routines'"'"' flow of execution, by virtue of their patched code, is re-directed into a user-supplied code in a pre-entry routine of the API interception module. The API routine might be completely by-passed or its input parameters might be filtered and changed by the user code. During the operation, the API routine is double-patched by the API interception module to ensure that all simultaneous calls to the API routine will re-direct its flow of control into the API interception module. A user-supplied code in a post-entry module of the API interception module might filter or change the return values of the API.

Claims

1. In a computer system running an operating system platform, said operating system including a kernel space and a process space, a user application running in process space, said user application using application program interface (API) function, whereby said API function is executed in memory space, a method of providing user control of said API function, said method comprising steps of:

initializing API controlling routine;

hooking at least one API routine in said memory space associated with user application;

replacing said hooked API routine code with user supplied code, said user supplied code to be executed upon calling said API by said user application program; and

receiving a call from a previously hooked API and generating a predefined series of operations to control said API operation;

wherein said step of replacing said looked API routine code with user supplied code, further comprises the steps of;

storing API routine code associated with first re-direction of flow of execution to be later replaced;

storing API routine code address associated with second re-direction of flow of execution;

storing API routine code associated with second re-direction of flow of execution;

replacing said API routine code stored with user supplied code associated with first redirection of flow of execution of said API routine;

wherein enhanced privileges relating to memory space associated with said API routine are enabled;

wherein the steps are adapted for preventing intrusions.

2. The method of claim 1, wherein said step of initializing API controlling routine comprises the steps of managing operation of API controlling routine and collecting and storing information corresponding to said API routine.

3. The method according to claim 1, wherein said step of hooking at least one API routine in said memory space associated with user application comprises the steps of:

identifying said API routine;

obtaining said API routine address; and

determining an address of at least one user supplied module associated with re-direction of flow of execution of said API routine.

4. In a computer system running an operating system platform, said operating system including a kernel space and a process space, a user application running in process space, said user application using application program interface (API) function, whereby said API function is executed in memory space, a method of providing user control of said API function in conjunction with previously hooked API functions associated with user application and responsive to call made to said API function by said user application, said method comprising steps of:

restoring API routine code previously stored associated with first re-direction of flow of execution to be later replaced;

replacing API routine code with user supplied code associated with second re-direction of flow of execution of said API routine;

calling said API routine based on response generated corresponding to whether API routine is to be executed in association with user predefined rules;

replacing said API routine code stored with user supplied code associated with first redirection of flow of execution of said API routine; and

restoring previously stored API routine code associated with second re-direction of flow of executions;

wherein enhanced privileges relating to memory space associated with said API routine are enabled;

wherein the steps are adapted for preventing intrusions.

5. The method according to claim 4, initially comprising the step of limiting execution of said user application to said specific API routine corresponding to execution time of API routine based on response generated corresponding to whether API routine is to be executed in association with user predefined rules.

6. The method according to claim 4, further comprising the step of canceling limiting execution of said user application to said specific API routine corresponding to execution time of API routine.

7. The method according to claim 4, further comprising the steps of:

executing user supplied code for determining return values of said API routine; and

manipulating process level flow control structure to enable return control to user application.

8. In a computer system running an operating system platform, said operating system including a kernel space and a process space, a user application running in process space, said user application using application program interface (API) function, whereby said API function is executed in memory space, a method of inserting user supplied code into memory space of user application, said user supplied code operative to further control API execution, said method comprising steps of:

injecting loader code into active process memory space associated with said user application;

executing loader code to further load user supplied code into memory space, said user supplied code operative to further control API execution;

injecting unloader code into active process memory space associated with said user application; and

executing unloader code to further unload user supplied code from memory space;

wherein said user supplied code replaces API routine code by;

storing API routine code associated with first re-direction of flow of execution to be later replaced;

storing API routine code address associated with second re-direction of flow of execution;

storing API routine code associated with second re-direction of flow of execution;

replacing said API routine code stored with user supplied code associated with first redirection of flow of execution of said API routine;

wherein the steps are adapted for preventing intrusions.

9. In a computer system running an operating system platform, said operating system including a kernel space and a process space, a user application running in process space, said user application using application program interface (API) function, whereby said API function is executed in process space, an apparatus controlling the method of providing user control of said API function, said apparatus comprising:

an initializer for obtaining list of active processes within computer system;

an injector for injecting API Interception module into said active processes;

means for monitoring predetermined system calls operative to further injection of API Interception routine into new created process;

means for unloading API Interception routine from a process; and

means for updating said list of active processes;

means for replacing a hooked API routine code with user supplied code, comprising;

means for storing API routine code associated with first re-direction of flow of execution to be later replaced;

means for storing API routine code address associated with second redirection of flow of execution;

means for storing API routine code associated with second re-direction of flow of execution;

means for replacing said API routine code stored with user supplied code associated with first re-direction of flow of execution of said API routine;

wherein enhanced privileges relating to memory space associated with said API routine are enabled;

wherein the apparatus is adapted for preventing intrusions.

10. The apparatus according to claim 9, wherein said initializer includes a system call interception module operative to obtain list of active processes within a computer system.

11. The apparatus according to claim 9, wherein said initializer includes a system call interception module further operative to open processes within a computer system.

12. The apparatus according to claim 9, wherein said initializer includes a system call interception module further operative to issue notification associated with system calls.