Method of virtualizing file extensions in a computer system by determining an association between applications in virtual environment and a file extension

• US 8,387,047 B1
• Filed: 08/03/2009
• Issued: 02/26/2013
• Est. Priority Date: 08/03/2009
• Status: Active Grant

• Current Assignee: CA, Inc. (d/b/a CA Technologies) (Broadcom, Inc.)
• Original Assignee: Symantec Corporation (NortonLifeLock Inc.)
• Inventors: Mazin, Yury, McCarten, Ryan M.
• Primary Examiner(s): An, Meng
• Assistant Examiner(s): LEE, TAMMY EUNHYE
• Application Number:
  US12/534,771
• Time in Patent Office: 1,303 Days
• Field of Search: None
• US Class Current: 718/1
• CPC Class Codes: G06F 9/445 Program loading or initiating bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors

Abstract

A system and method for virtualizing file extensions. The system includes a system environment, one or more virtual environments, and a shell extension. The shell extension detects a request to perform an action on a file in the system environment. The file has an associated file extension. The system environment includes applications for opening files. In response to detecting the request, the shell extension determines that the action is to open the file in the system environment and the file extension is not associated with any of the applications in the system environment. The shell extension queries the virtual environments to find an association between the file extension and an application, identifies at least one of the virtual environments in which the extension is associated with an application, and opens the file in a selected virtual environment.

Claims

1. A computer system comprising:

a system environment; and

an application manager comprising visibility rules;

one or more virtual environments, each comprising an association table;

wherein the system environment includes a shell extension configured to detect a request to perform an action on a file in the system environment, wherein in response to detecting the request the shell extension is configured to;

determine that the action comprises opening the file;

create a processing thread in the system environment;

determine the file extension is not associated with any application in the system environment;

apply said visibility rules to identify particular virtual environments of the one or more virtual environments to be searched for an association between an application and the file extension;

query association tables in each of the particular virtual environments to determine whether the corresponding virtual environment includes an association between an application and the file extension;

identify one or more of the virtual environments in which the file extension is associated with an application;

apply selection rules to select a given virtual environment from the identified one or more virtual environments;

move the processing thread to the given virtual environment; and

open the file in the given virtual environment.

2. The system of claim 1, wherein the shell extension is registered with an operating system shell and the operating system shell is configured to convey a notification to the registered shell extension when a request to perform an action on a file is made.

3. The system of claim 2, wherein the operating system shell is configured to:

delay processing of the request until a return status is received from the registered shell extension; and

continue processing of the request responsive to receiving an indication from the registered shell extension that indicates an application associated with file extension was not found in the system environment.

4. The system of claim 2, wherein the notification includes first information identifying the file extension and second information identifying the action.

5. The system of claim 3, wherein the operating system shell is further configured to continue processing of the request responsive receiving an indication from the registered shell extension that indicates the action is not an action to open a file.

6. The system of claim 3, wherein the operating system shell is further configured to continue processing of the request responsive receiving an indication from the registered shell extension that indicates the shell extension can not identify a virtual environment in which the extension is associated with an application.

7. A method of virtualizing file extensions in a host computer system, the method comprising:

detecting a request to perform an action on a file in a system environment, wherein the file has an associated file extension;

in response to detecting the request;

determining that the action comprises opening the file;

creating a processing thread in the system environment;

determining the file extension is not associated with any application in the system environment;

applying said visibility rules to identify particular virtual environments of the one or more virtual environments to be searched for an association between an application and the file extension;

querying association tables in each of the particular virtual environments to determine whether the corresponding virtual environment includes an association between an application and the file extension;

identifying one or more of the virtual environments in which the extension is associated with an application;

applying selection rules to select a given virtual environment from the identified one or more virtual environments;

moving the processing thread to the given virtual environment; and

opening the file in the given virtual environments.

8. The method of claim 7,

wherein a shell extension is registered with an operating system shell to receive notifications when a request to perform an action on a file is made;

and

wherein said detecting a request comprises the shell extension receiving a notification of the request from the operating system shell.

9. The method of claim 8, further comprising the operating system shell:

delaying processing of the request until a return status is received from the registered shell extension; and

continuing processing of the request responsive to receiving an indication from the registered shell extension that indicates an application associated with file extension was not found in the system environment.

10. The method of claim 8, wherein the notification includes first information identifying the file extension and second information identifying the action.

11. The method of claim 9, further comprising the operating system shell continuing processing of the request responsive receiving an indication from the registered shell extension that indicates the action is not an action to open a file.

12. The method of claim 9, further comprising the operating system shell continuing processing of the request responsive receiving an indication from the registered shell extension that indicates the shell extension can not identify a virtual environment in which the extension is associated with an application.

13. A non-transitory computer readable storage medium,

wherein the non-transitory computer accessible readable storage medium stores first program instructions executable by a computer system to:

detect a request to perform an action on a file in a system environment, wherein the file has an associated file extension; and

in response to detecting the request;

determine that the action comprises opening the file;

create a processing thread in the system environment;

determine the file extension is not associated with any application in the system environment;

apply said visibility rules to identify particular virtual environments of the one or more virtual environments to be searched for an association between an application and the file extension;

query association tables in each of the particular virtual environments to determine whether the corresponding virtual environment includes an association between an application and the file extension;

identify at least one of the virtual environments in which the extension is associated with an application; and

apply selection rules to select a given virtual environment from the identified one or more virtual environments;

move the processing thread to the given virtual environment; and

open the file in the given virtual environments.

14. The non-transitory computer readable storage medium of claim 13, wherein the first program instructions are further executable by the computer system to register a shell extension with an operating system shell, wherein the operating system shell is configured to convey a notification to the registered shell extension when a request to perform an action on a file is made.

15. The non-transitory computer readable storage medium of claim 14, wherein the operating system shell is configured to:

delay processing of the request until a return status is received from the registered shell extension; and

continue processing of the request responsive to receiving an indication from the registered shell extension that indicates an application associated with file extension was not found in the system environment.

16. The non-transitory computer readable storage medium of claim 14, wherein the notification includes first information identifying the file extension and second information identifying the action.

17. The non-transitory computer readable storage medium of claim 15, wherein the operating system shell is configured to continue processing of the request responsive receiving an indication from the registered shell extension that indicates the action is not an action to open a file.