Detection of code execution exploits
First Claim
Patent Images
1. A non-transitory computer readable storage device comprising instructions stored thereon to cause one or more processors to:
- determine where one or more candidate areas exist within an arbitrary file;
search at least one nearby area in front of or after each of the one or more candidate areas within the arbitrary file for an instruction candidate;
disassemble instructions starting at a found offset for the instruction candidate to create a disassembled instruction set, the found offset reflecting a location of the instruction candidate within the arbitrary file;
normalize at least a portion of the disassembled instruction set to create a normalized instruction set;
scan the normalized instruction set to determine if the normalized instruction set reflects that the disassembled instruction set has a probability of containing shellcode;
calculate a statistical probability that the instruction candidate is associated with shellcode for normalized instruction sets associated with disassembled instruction sets that were determined to reflect a probability of containing shellcode, wherein statistical probabilities are not calculated if the normalized instruction set reflects that the disassembled instruction set has no probability of containing shellcode;
for a given stream of instructions starting at the found offset, map an instruction-to-shellcode probability to each instruction in the given stream of instructions; and
sum the mapped instruction-to-shellcode probability for each instruction using Bayes'"'"' formula to generate an overall probability.
10 Assignments
0 Petitions
Accused Products
Abstract
Various embodiments include a method of detecting shell code in an arbitrary file comprising determining where one or more candidate areas exist within an arbitrary file, searching at least one nearby area surrounding each of the one or more candidate areas within the arbitrary file for an instruction candidate, and calculating for any such instruction candidate a statistical probability based on a disassembly of instructions starting at a found offset for the instruction candidate that the disassembled instructions are shellcode.
-
Citations
22 Claims
-
1. A non-transitory computer readable storage device comprising instructions stored thereon to cause one or more processors to:
-
determine where one or more candidate areas exist within an arbitrary file; search at least one nearby area in front of or after each of the one or more candidate areas within the arbitrary file for an instruction candidate; disassemble instructions starting at a found offset for the instruction candidate to create a disassembled instruction set, the found offset reflecting a location of the instruction candidate within the arbitrary file; normalize at least a portion of the disassembled instruction set to create a normalized instruction set; scan the normalized instruction set to determine if the normalized instruction set reflects that the disassembled instruction set has a probability of containing shellcode; calculate a statistical probability that the instruction candidate is associated with shellcode for normalized instruction sets associated with disassembled instruction sets that were determined to reflect a probability of containing shellcode, wherein statistical probabilities are not calculated if the normalized instruction set reflects that the disassembled instruction set has no probability of containing shellcode; for a given stream of instructions starting at the found offset, map an instruction-to-shellcode probability to each instruction in the given stream of instructions; and sum the mapped instruction-to-shellcode probability for each instruction using Bayes'"'"' formula to generate an overall probability. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A gateway comprising:
-
a memory comprising instructions stored therein; and one or more processors communicatively coupled to the memory to configure the one or more processors to operate an anti-malware engine, the anti-malware engine operable to; scan an arbitrary file and to determine if any candidate areas exist within the arbitrary file; for any given candidate area located within the arbitrary file, search at least one nearby area in front of or after the candidate area for any instruction candidates; and for each instruction candidate; disassemble instructions starting at a found offset to create a disassembled instruction set, wherein the found offset reflects a location of the each instruction candidate within the arbitrary file; normalize at least a portion of the disassembled instruction set to create a normalized instruction set; scan the normalized instruction set to determine if the normalized instruction set reflects that the disassembled instruction set has a probability of containing shell code; and calculate a statistical probability that the instruction candidate is associated with shellcode for normalized instruction sets associated with disassembled instruction sets that were determined to reflect a probability of containing shellcode, wherein statistical probabilities are not calculated if the normalized instruction set reflects that the disassembled instruction set has no probability of containing shellcode, wherein the anti-malware engine is operable to access known characteristical shellcode sequences stored in a database coupled to the anti-malware engine, and to scan the arbitrary file for the known characteristical shellcode sequences. - View Dependent Claims (11, 12, 13, 14)
-
-
15. A non-transitory computer readable storage device comprising instructions stored thereon to cause one or more processors to:
-
scan an arbitrary file to determine offset locations of one or more candidate areas that exist within the arbitrary file, the offset locations identifying starting and ending locations of each of the one or more candidate areas within the arbitrary file; for at least one of the one or more candidate areas found in the arbitrary file, first search the areas surrounding the at least one candidate area to determine if any function calls or any code branching instructions exist in the areas surrounding the at least one candidate area, wherein the areas surrounding the at least one candidate area comprise areas in front of or after the at least one candidate area; if no function calls and no code branching instructions are found, search the areas surrounding the at least one candidate area for known characteristical shellcode sequences by disassembling one or more instructions from the surrounding areas, normalizing the disassembled one or more instructions to create one or more normalized instructions, and scan the one or more normalized instructions to determine if the one or more normalized instructions reflect that the one or more disassembled instructions represent shell code; and scan the arbitrary file for data blocks with high information entropy compared to a threshold value when no known characteristical shellcode sequences are found. - View Dependent Claims (16, 17, 18, 19, 20)
-
-
21. A computer network comprising:
-
a gateway device comprising memory and one or more processors communicatively coupled to the memory; and a database coupled to the gateway device, the database including memory operable to store one or more known shellcode sequences and to provide the one or more known shellcode sequences to an anti-malware engine for comparison to instructions in an arbitrary file, wherein the memory stores instructions to cause the one or more processors to be configured to include the anti-malware engine, the anti-malware engine operable to; receive the arbitrary file; scan the arbitrary file for repetitive constructs that have a potential to overflow a buffer in a computer memory when the arbitrary file is parsed, rendered or executed; determine if any function calls or any code branching instructions exist in the areas surrounding the repetitive constructs, wherein areas surrounding the repetitive constructs comprise areas in front of or behind the repetitive constructs with respect to a location of the repetitive constructs within the arbitrary file; and generate a statistical probability representing the likelihood that the arbitrary file includes shellcode by performing a statistical analysis of the instructions starting at each found function call or branching instruction to generate an overall shellcode probability for the instructions starting at the each found function call or branching instruction, the statistical analysis of the instructions including disassembling instructions to create a disassembled instruction set, normalizing at least a portion of the disassembled instruction set to create a normalized instruction set, and scanning the normalized instruction set. - View Dependent Claims (22)
-
Specification