Systems And Methods For Regulating Execution Of Computer Software

• US 20090328003A1
• Filed: 04/17/2008
• Published: 12/31/2009
• Est. Priority Date: 02/11/2003
• Status: Active Grant

• Current Assignee: Revulytics, Inc. (Flexera Software LLC)
• Original Assignee: V.I. Laboratories, Inc.
• Inventors: Merrell, Douglas C., Pensak, David A.
• Granted Patent: US 8,225,290 B2
• Time in Patent Office: Days
• US Class Current: 717/124
• CPC Class Codes: G06F 21/126 Interacting with the operating system

Abstract

A method for regulating execution of an application program includes a process for preparing the application and a process for executing the application. The preparation process divides the application program into related segments and encrypts instructions of at least one segment. The preparation process positions encrypted instructions in at least two discontiguous regions within an executable file and associates header information with at least one discontiguous region. The header identifies a decryption key and the location of the other discontiguous region. The resulting execution file thus has portions that would not execute and would cause an operating system to call a responsive process. The execution process initiates execution of the protected application when at least a portion of the application instructions are available to the computer only in encrypted form. The user is authenticated, after which the encrypted portions can be decrypted and execution resumed. The processes may also include capability to detect and respond to tampering, or the ability to block execution snooping via a debugger. Also provided are systems and methods to allow debugging of code extensions to protected applications without sacrificing protection of the application. A Secure Debugger allows extension developers to examine memory and set breakpoints in their own extensions, without permitting them to view information in the rest of the protected application.

Claims

1. A process for executing an application computer program defining a plurality of subsections, said application computer program configured to mm under the control of an operating system and incorporating an execution controller computer program, said process steps comprising:

substituting an operating system instruction that causes an interrupt for one or more bytes of at least one subsection;

launching the application computer program;

launching the execution controller computer program, where the execution controller computer program is launched by the application computer program;

attaching the execution controller computer program to the application computer program using the operating system debugger interface;

launching a secure debugger computer program, where the secure debugger computer program is launched by the execution controller computer program;

executing the application computer program until an operating system instruction that causes an interrupt is encountered;

after an operating system instruction that causes an interrupt is encountered, restoring the operating system instruction that causes an interrupt with the substituted bytes; and

resuming execution of the application computer program, starting at the restored bytes.

2. The process of claim 1, further comprising the step of:

after an operating system instruction that causes an interrupt is encountered, terminating the execution of the application computer program.

3. The process of claim 1, further comprising the step of:

after an operating system instruction that causes an interrupt is encountered, authorizing a continued execution of the application computer program.

4. The process of claim 3, where the step of authorizing a continued execution of the application computer program comprises authenticating a user of the application computer program.

5. The process of claim 1, where at least a portion of the at least one subsection has been encrypted, further comprising the steps of:

after an operating system instruction that causes an interrupt is encountered, decrypting the encrypted portion; and

executing the decrypted subsection.

6. The process of claim 5, further comprising the step of:

re-encrypting the decrypted subsection.

7. The process of claim 5, where at least a portion of the information used to decrypt the encrypted portion is obtained from a key server computer program.

8. The process of claim 5, where at least a portion of the information used to decrypt the encrypted portion is obtained from a location in the application computer program.

9. The process of claim 5, further comprising the steps of:

after an operating system instruction that causes an interrupt is encountered, detecting if there has been tampering of the application computer program; and

if tampering has been detected, reporting information related to the tampering to a user of the application computer program.

10. The process of claim 1, further comprising the steps of:

after an operating system instruction that causes an interrupt is encountered, detecting if there has been tampering of the application computer program; and

if tampering has not been detected, resuming execution of the application computer program.

11. The process of claim 1, further comprising the steps of:

after the secure debugger computer program is launched, authenticating a user of the secure debugger computer program;

12. The process of claim 11, further comprising the steps of:

after the user of the secure debugger computer program is authenticated, accepting debug commands from the user.

13. The process of claim 12, where the debug commands are entered into a window opened by the secure debugger computer program.

14. A process for executing an application computer program that includes a plurality of subsections having boundaries, said application computer program configured to run under the control of an operating system, said process steps comprising:

substituting an operating system instruction that causes an interrupt for one or more bytes of at least one subsection;

relocating the substituted bytes to a location in the application computer program this is distinct from the boundaries of the subsection;

adding an execution controller computer program into the application computer program;

launching the application computer program;

launching the execution controller computer program, where the execution controller computer program is launched by the application computer program;

attaching the execution controller computer program to the application computer program using the operating system debugger interface;

launching a secure debugger computer program, where the secure debugger computer program is launched by the execution controller computer program;

executing the application computer program until an operating system instruction that causes an interrupt is encountered;

after an operating system instruction that causes an interrupt is encountered, restoring the operating system instruction that causes an interrupt with the substituted bytes; and

resuming execution of the application computer program, starting at the restored bytes.

15. The process of claim 14, further comprising the steps of:

encrypting at least a portion of at least one subsection of the application computer program; and

after an operating system instruction that causes an interrupt is encountered, decrypting the encrypted portion.

16. The process of claim 14, further comprising the steps of:

adding tamper detection instructions and data into the application computer program;

after an operating system instruction that causes an interrupt is encountered, detecting if there has been tampering of the application computer program; and

if tampering has been detected, terminating the execution of the application computer program.

17. The process of claim 14, further comprising the steps of:

adding tamper detection instructions into the application computer program;

after an operating system instruction that causes an interrupt is encountered, detecting if there has been tampering of the application computer program; and

if tampering has been detected, reporting the tampering to a user of the application computer program.

18. The process of claim 14, further comprising the steps of:

after an operating system instruction that causes an interrupt is encountered, authenticating a user of the application computer program; and

if the user of the application computer program is authenticated, resuming the execution of the application computer program.

19. The process of claim 14, further comprising the steps of:

after an operating system instruction that causes an interrupt is encountered, authenticating a user of the application computer program; and

if the user of the application computer program is not authenticated, terminating the execution of the application computer program.

20. The process of claim 14, further comprising the steps of:

after the secure debugger computer program is launched, authenticating a user of the secure debugger computer program.