Handling instruction received from a sandboxed thread of execution

• US 8,468,600 B1
• Filed: 03/04/2011
• Issued: 06/18/2013
• Est. Priority Date: 03/04/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Kaskel, Bruce E., Holland, Paul
• Primary Examiner(s): PEARSON, DAVID J
• Application Number:
  US13/041,168
• Publication Number:
  US 20130166891A1
• Time in Patent Office: 837 Days
• Field of Search: None
• US Class Current: 726/22
• CPC Class Codes: G06F 21/53 by executing in a restricted environment, e.g. sandbox or secure virtual machine

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enveloping a thread of execution within an IDT-based secure sandbox. In one aspect, embodiments of the invention provide a method performed in a computer system, the method receiving an instruction from an execution thread where the computer system can be configured for redirection of instructions from the execution thread. The method can determine whether the instruction includes at least one of an interrupt instruction, a system call instruction and a system enter instruction. In response to determining that the instruction includes at least one of the interrupt instruction, the system call instruction and the system enter instruction, the method can further: (i) eliminate the redirection, (ii) modify a stack to specify return of control, and (iii) thereafter, pass the control to an operating system kernel.

Claims

1. A method performed in a computer system, the method comprising:

enabling a secure environment in the computer system for executing a thread comprising a plurality of instructions, wherein;

an application executing on the computer system generates the thread and initiates execution of the thread in the secure environment; and

the computer system is configured to redirect the execution of the plurality of instructions from an operating system kernel to the secure environment;

receiving an instruction from the thread;

determining whether the instruction includes a call into the operating system kernel, the instruction being one of an interrupt instruction, a system call instruction, or a system enter instruction;

in response to determining that the instruction is one of the interrupt instruction, the system call instruction, or the system enter instruction, determining that the instruction need not be executed in the secure environment; and

in response to determining that the instruction need not be executed in the secure environment;

eliminating the redirection of the execution of the instruction to the secure environment,

modifying a stack to specify return of control for the thread when the execution of the instruction is completed, and

passing the control for the thread to the operating system kernel for execution of the instruction.

2. The method of claim 1, further comprising, in response to receiving the instruction from the thread, comparing the instruction to one or more entries in a table, and wherein determining that the instruction need not be executed in the secure environment comprises determining that the instruction need not be executed based on a result of the comparison.

3. The method of claim 1, in response to determining that the instruction is not one of the interrupt instruction, the system call instruction, or the system enter instruction, determining that the instruction should be executed in the secure environment;

and

in response to determining that the instruction should be executed in the secure environment;

accessing and executing replacement code associated with the instruction, the access to the replacement code indicated by a modified interrupt descriptor table entry pointed to by a modified interrupt descriptor table register.

4. The method of claim 1, further comprising providing code for reconfiguring the computer system for the redirection, the code being specified by the return of control.

5. The method of claim 4, further comprising executing the code specified by the return of control and reconfiguring the computer system for redirection.

6. The method of claim 4, wherein a client application provides the code for reconfiguring the computer system in an epilog.

7. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by a computing apparatus causes the computing apparatus to perform operations comprising:

enabling a secure environment in the computer system for executing a thread comprising a plurality of instructions, wherein;

an application executing on the computer system generates the thread and initiates execution of the thread in the secure environment; and

the computer system is configured to redirect the execution of the plurality of instructions from an operating system kernel to the secure environment;

receiving an instruction from the thread;

determining whether the instruction includes a call into the operating system kernel, the instruction being one of an interrupt instruction, a system call instruction, or a system enter instruction;

in response to determining that the instruction is one of the interrupt instruction, the system call instruction, or the system enter instruction, determining that the instruction need not be executed in the secure environment; and

in response to determining that the instruction need not be executed in the secure environment;

eliminating the redirection of the execution of the instruction to the secure environment,

modifying a stack to specify return of control for the thread when the execution of the instruction is completed, and

passing the control for the thread to the operating system kernel for execution of the instruction.

8. The non-transitory computer storage medium of claim 7, wherein operations further comprise, in response to receiving the instruction from the thread, comparing the instruction to one or more entries in a table, and wherein determining that the instruction need not be executed in the secure environment comprises determining that the instruction need not be executed based on a result of the comparison.

9. The non-transitory computer storage medium of claim 7, wherein operations further comprise:

in response to determining that the instruction is not one of the interrupt instruction, the system call instruction, or the system enter instruction, determining that the instruction should be executed in the secure environment; and

in response to determining that the instruction should be executed in the secure environment;

accessing and executing replacement code associated with the instruction, the access to the replacement code indicated by a modified interrupt descriptor table entry pointed to by a modified interrupt descriptor table register.

10. The non-transitory computer storage medium of claim 7, wherein operations further comprise providing code that includes instructions for reconfiguring the computing apparatus for the redirection, the code being specified by the return of control.

11. The non-transitory computer storage medium of claim 10, wherein operations further comprise executing the instructions included in the code specified by the return of control and reconfiguring the computing apparatus for redirection.

12. The non-transitory computer storage medium of claim 10, wherein a client application provides the instructions for reconfiguring the computer system in an epilog.

13. A system comprising:

a processor; and

a computer readable storage device having stored therein;

a client application that, when executing on the system, generates a thread comprising a plurality of instructions, initiates execution of the thread in a secure environment, and sends an instruction from the thread;

an operating system kernel;

a device driver that receives the instruction from the thread, and enables the secure environment, wherein the system is configured to redirect the execution of the instruction from the operating system kernel to the secure environment; and

a stack;

wherein the device driver;

determines whether the instruction includes a call into the operating system kernel, the instruction being one of an interrupt instruction, a system call instruction, or a system enter instruction,

in response to determining that the instruction is one of the interrupt instruction, the system call instruction, or the system enter instruction, determines that the instruction need not be executed in the secure environment, and

in response to determining that the instruction need not be executed in the secure environment;

eliminates the redirection of the execution of the instruction to the secure environment,

modifies the stack to specify return of control for the thread when the execution of the instruction is completed, and

passes the control for the thread to the operating system kernel for execution of the instruction.

14. The system of claim 13, the computer readable storage device having further stored therein a table, wherein the device driver compares the instruction to one or more entries in the table, and wherein determining that the instruction need not be executed in the secure environment comprises determining that the instruction need not be executed based on a result of the comparison.

15. The system of claim 13, the computer readable storage device having further stored therein a modified interrupt descriptor table and an interrupt descriptor table register, wherein in response to determining that the instruction is not one of the interrupt instruction, the system call instruction, or the system enter instruction, the device driver:

determines that the instruction should be executed in the secure environment; and

in response to determining that the instruction should be executed in the secure environment;

accesses and executes replacement code associated with the instruction, the access to the replacement code indicated by a modified interrupt descriptor table entry in the modified descriptor table, the entry pointed to by a modified version of the interrupt descriptor table register.

16. The system of claim 13, wherein the device driver provides code that includes instructions for reconfiguring the system for the redirection, the code being specified by the return of control.

17. The system of claim 16, wherein after execution of the instructions included in the code the device driver reconfigures the system for redirection.

18. The system of claim 16, wherein the client application provides the instructions for reconfiguring the system in an epilog.