Method and apparatus for isolating execution of software applications

• US 7,680,758 B2
• Filed: 09/30/2004
• Issued: 03/16/2010
• Est. Priority Date: 09/30/2004
• Status: Active Grant

• Current Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Original Assignee: Citrix Systems, Inc. (Cloud Software Group)
• Inventors: Bissett, Nicholas, Mazzaferri, Richard James, Laborczfalvi, Lee George, Borzycki, Andrew Gerard, Roychoudhry, Anil, Chin, Huai Chiun, Muir, Jeffrey Dale, Semaan, Pierre
• Primary Examiner(s): Vo; Tim T.
• Assistant Examiner(s): Morrison; Jay A
• Application Number:
  US10/711,737
• Publication Number:
  US 20060075381A1
• Time in Patent Office: 1,993 Days
• Field of Search: 707/1, 707/8, 707/9, 707/10, 707/203
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine
• CPC Class Codes: G06F 2209/542 Intercept
• CPC Class Codes: G06F 8/61 Installation
• CPC Class Codes: G06F 9/45537 Provision of facilities of other operating environments, e.g. WINE I/O emulation G06F13/105
• CPC Class Codes: G06F 9/468 Specific access rights for resources, e.g. using capability register
• CPC Class Codes: G06F 9/4856 resumption being on a different machine, e.g. task migration, virtual machine migration G06F9/5088 takes precedence
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/545 where tasks reside in different layers, e.g. user- and kernel-space
• CPC Class Codes: Y10S 707/99931 Database or file accessing

Abstract

A method for isolating access by application programs to native resources provided by an operating system redirects a request for a native resource made by an application program executing on behalf of a user to an isolation environment. The isolation environment includes a user isolation scope and an application isolation scope. An instance of the requested native resource is located in the user isolation scope corresponding to the user. The request for the native resource is fulfilled using the version of the resource located in the user isolation scope. If an instance of the requested native resource is not located in the user isolation scope, the request is redirected to an application isolation scope. The request for the native resource is fulfilled using the version of the resource located in the application isolation scope. If an instance of the requested native resource is not located in the application isolation scope, the request is redirected to a system scope.

Claims

1. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer a request for a native resource provided by an operating system and stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(b) failing to locate in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user;

(c) redirecting the request to the application isolation layer;

(d) locating in the memory element an instance of the requested native resource associated with an application isolation scope provided by the application isolation layer; and

(e) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the application isolation scope.

2. The method of claim 1 wherein step (e) comprises creating an instance of the requested native resource associated with the user isolation scope that corresponds to the instance of the requested native resource associated with the application isolation scope and responding to the request for the native resource using the created instance of the requested native resource associated with the user isolation scope.

3. The method of claim 1 wherein step (d) comprises failing to locate an instance of the requested native resource in the memory element and associated with the application isolation scope.

4. The method of claim 3 wherein step (e) comprises responding to the request for the native resource using a system-scoped native resource.

5. The method of claim 3 wherein step (e) comprises:

creating an instance of the requested native resource associated with the user isolation scope that corresponds to the instance of the requested resource associated with a system scope and responding to the request for the native resource using the created instance of the resource associated with the user isolation scope.

6. The method of claim 1 further comprising the step of hooking a request for a native resource made by a process executing on behalf of a first user.

7. The method of claim 4 further comprising the step of intercepting a request for a native resource made by a process executing on behalf of a first user.

8. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a-1) intercepting by a file system filter driver a request for a file system native resource provided by an operating system and stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(a-2) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer the request for the file system native resource;

(b) locating in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user; and

(c) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the user isolation scope.

9. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer a request for a file stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(b) locating in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user; and

(c) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the user isolation scope.

10. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer a request for a registry database entry stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(b) locating in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user; and

(c) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the user isolation scope.

11. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer a request for a native resource provided by an operating system and stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(b) locating in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user;

(c) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the user isolation scope;

(d) redirecting to the isolation environment a request for the native resource made by a second process executing on behalf of a second user;

(e) locating in the memory element an instance of the requested native resource associated with a second user isolation scope provided by the user isolation layer on behalf of the second user; and

(f) responding to the request for the native resource using the instance of the native resource located in the memory element and associated with the second user isolation scope.

12. The method of claim 11 wherein the process executes concurrently on behalf of a first user and a second user.

13. The method of claim 11 wherein step (e) comprises failing to locate an instance of the requested native resource associated with the second user isolation scope.

14. The method of claim 13 wherein step (I) comprises redirecting the request to the application isolation layer.

15. The method of claim 14 further comprising the steps of:

(g) locating in the memory element an instance of the requested native resource associated with an application isolation scope provided by the application isolation layer on behalf of an application; and

(h) responding to the request for the native resource using the instance of the native resource associated with the application isolation scope.

16. A method for isolating access by application programs to native resources provided by an operating system, the method comprising instructing a suitably programmed computer to perform the steps of:

(a) redirecting to an isolation environment comprising a user isolation layer and an application isolation layer a request for a native resource provided by an operating system and stored in a memory element provided by a computer, the request made by a process executing on behalf of a first user;

(b) locating in the memory element an instance of the requested resource associated with a user isolation scope provided by the user isolation layer on behalf of a first user

(c) responding to the request for the native resource using the instance of the requested native resource located in the memory element and associated with the user isolation scope;

(d) redirecting to the isolation environment a request for a native resource made by a second process executing on behalf of a first user;

(e) locating in the memory element an instance of the requested native resource associated with the user isolation scope; and

(f) responding to the request for the native resource using the instance of the resource associated with the user isolation scope.

17. The method of claim 16 wherein step (e) comprises failing to locate an instance of the requested native resource associated with the user isolation scope.

18. The method of claim 17 wherein step (f) comprises redirecting the request to locate an instance of the native resource associated with a second application isolation scope provided by the application isolation layer on behalf of a second application.

19. The method of claim 18 further comprising the steps of:

(g) locating an instance of the requested native resource associated with the second application isolation scope; and

(h) responding to the request for the native resource using the instance of the native resource associated with the second application isolation scope.

20. An apparatus for isolating access by application programs to native resources provided by an operating system, the apparatus comprising:

computer-readable program means for associating an instance of a native resource provided by an operating system with a user isolation scope provided by an isolation environment comprising an application isolation layer and a user isolation layer, the user isolation scope corresponding to a user;

computer-readable program means for associating an instance of a native resource with an application isolation scope provided by the isolation environment, the application isolation scope corresponding to an application; and

computer-readable program means for intercepting a request for a native resource made by a process executing on behalf of the user and redirecting the request to the instance of the resource associated with the user isolation scope.

21. The apparatus of claim 20 wherein the computer-readable program means for associating an instance of a native resource with an application isolation scope further comprises means for associating an instance of the native resource with a second application isolation scope, the second isolation scope corresponding to a second application.

22. The apparatus of claim 20 wherein the computer-readable program means for intercepting a request returns a handle to the requesting process that identifies the native resource.

23. The apparatus of claim 20 further comprising computer-readable program means for specifying behavior for the computer-readable program means for intercepting a request when redirecting the request.

24. The apparatus of claim 20 wherein the computer-readable program means for intercepting a request comprises a file system filter driver.

25. The apparatus of claim 20 wherein the computer-readable program means for intercepting a request comprises a function hooking mechanism.

26. The apparatus of claim 25 wherein the function hooking mechanism intercepts an operation selected from the group of file system operations, registry operations, operating system services, packing and installation services, named object operations, window operations, file-type association operations and Component Object Model (COM) server operations.

27. An apparatus for isolating access by application programs to native resources provided by an operating system, the apparatus comprising:

computer-readable program means for;

(i) associating an instance of a native resource provided by an operating system with a user isolation scope provided by an isolation environment comprising an application isolation layer and a user isolation layer, the user isolation scope corresponding to a user, and for (ii) associating an instance of the native resource with a second user isolation scope, the second user isolation scope corresponding to a second user; and

computer-readable program means for intercepting a request for a native resource made by a process executing on behalf of the user and redirecting the request to the instance of the resource associated with the user isolation scope.