System and method for defining and detecting pestware

• US 8,065,664 B2
• Filed: 08/07/2006
• Issued: 11/22/2011
• Est. Priority Date: 08/07/2006
• Status: Active Grant

• Current Assignee: Carbonite LLC (Open Text Corporation)
• Original Assignee: Webroot Software Incorporated (Open Text Corporation)
• Inventors: Burtscher, Michael
• Primary Examiner(s): DAS, CHAMELI
• Application Number:
  US11/462,956
• Publication Number:
  US 20080052679A1
• Time in Patent Office: 1,933 Days
• Field of Search: 717/124, 717126-128, 717130-131, 717/141, 717/133, 717/158
• US Class Current: 717/126
• CPC Class Codes: G06F 21/563 by source code analysis
• CPC Class Codes: G06F 9/4484 Executing subprograms

Abstract

A system and method for defining and detecting pestware is described. One embodiment includes receiving a file and placing at least a portion of the file into a processor-readable memory of a computer. A plurality of execution paths within code of the pestware file are followed and particular instructions within the execution paths are identified. A representation of the relative locations of each of the particular instructions within the code of the file are compared against a pestware-definition file so as to determine whether the file is a potential pestware file.

Claims

1. A method for generating pestware definitions comprising:

receiving a pestware file;

placing at least a portion of the pestware file into a processor-readable memory;

following a plurality of execution paths within code of the pestware file, wherein each of the execution paths is a potential path that a processor executing the code will potentially follow;

identifying particular instructions within the execution paths;

storing, in a processor-readable pestware-definition file, a representation of the relative locations of each of the particular instructions within the code of the pestware file; and

sending the pestware-definition file to a plurality of client devices, wherein at least one of the client devices receives a file and determines whether relative locations of each of particular instructions within code of the file received by the at least one client device match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.

2. The method of claim 1, wherein the identifying includes identifying system calls within the execution paths.

3. The method of claim 1 including:

simplifying the representation of the relative locations so as to create a simplified representation of the relative locations, wherein the storing includes storing the representation as the simplified representation.

4. The method of claim 1 including:

bypassing, while following the plurality of execution paths, instructions other than system calls and jump instructions.

5. The method of claim 1, wherein identifying particular instructions includes identifying calls to addresses to portions of the processor-readable memory that are outside of the memory occupied by the code of the pestware file.

6. The method of claim 5, including:

storing a representation of each of the addresses in connection with a corresponding one of the locations so as to identify each call by an address.

7. The method of claim 6, including:

storing a representation of each of the names of the calls in connection with a corresponding one of the locations so as to identify each call by a name.

8. A method for detecting pestware on a computer comprising:

receiving a file;

placing at least a portion of the file into a processor-readable memory of the computer;

following a plurality of execution paths within code of the file, wherein each of the execution paths is a potential path that a processor executing the code will potentially follow;

identifying particular instructions within the execution paths;

comparing, against a pestware-definition file, a representation of the relative locations of each of the particular instructions within the code of the file so as to determine whether the file is a potential pestware file; and

determining whether the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.

9. The method of claim 8, wherein the identifying includes identifying system calls within the execution paths.

10. The method of claim 8 including:

simplifying the representation of the relative locations so as to create a simplified representation of relative locations, wherein the comparing includes comparing the representation as the simplified representation.

11. The method of claim 8 including:

bypassing, while following the plurality of execution paths, instructions other than system calls and jump instructions.

12. The method of claim 8, wherein identifying particular instructions includes identifying calls to addresses to portions of the processor-readable memory that are outside of the memory occupied by the code of the pestware file.

13. The method of claim 8 including:

alerting a user of the computer in the event the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware-definition file.

14. The method of claim 8 including:

quarantining the file in the event the relative locations of each of the particular instructions within the code match at least a predetermined percentage of relative locations of instructions in the pestware definition file.