Methodology, system and computer readable medium for detecting file encryption

• US 20040196970A1
• Filed: 04/01/2003
• Published: 10/07/2004
• Est. Priority Date: 04/01/2003
• Status: Active Grant

• Current Assignee: Sytex, Inc. (Leidos Holdings, Inc.)
• Original Assignee: Sytex, Inc. (Leidos Holdings, Inc.)
• Inventors: Cole, Eric B.
• Granted Patent: US 7,564,969 B2
• Time in Patent Office: Days
• US Class Current: 380/1
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 9/00 Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols

Abstract

A method, computer readable medium and system for determining whether a data stream is in cyphertext format statistically analyzes the data stream to compute a resultant value indicative of a level of uniformity for a frequency distribution of the data stream'"'"'s byte values. When applied to one or more files an average byte value may be computed for the data stream and a chi-square statistical analysis of the data bytes performed, with the resultant value computed based on the chi-square value. The resultant is then compared to a pre-determined threshold value to determine whether the file has been encrypted. The computer-readable medium has executable instructions for reading the data stream portions of files to compute a resultant value for each file and control an output device to display appropriate output. The encryption detection system comprises a storage device, an output device and a processor programmed in accordance with the foregoing.

Claims

1. A computerized method for determining whether a data stream is in a cyphertext format, wherein said data stream is characterized by a plurality of data bytes, each having an associated byte value, said method comprising:

performing a statistical analysis of the associated data stream'"'"'s byte values to compute a statistical result having a resultant value indicative of a level of uniformity for a frequency distribution of the byte values in the data stream;

controlling an output device to display output corresponding to an existence of cyphertext if the resultant value indicates uniformity of the frequency distribution.

2. A computerized method according to claim 1 wherein said data stream is part of a file on a data storage medium.

3. A computerized method according to claim 2 wherein said file includes a header portion and a data portion which includes said data stream.

4. A computerized method according to claim 3 further comprising reading the header portion of the file and displaying output corresponding to an existence of cyphertext only if both the resultant value indicates non-uniformity and the header portion is uncorrelated to a known file type.

5. A computerized method according to claim 4 wherein said known file type is one of an image file format and a compressed file format.

6. A computerized method according to claim 1 whereby said statistical analysis is only performed if the data stream is of a minimum stream size.

7. A computerized method according to claim 5 wherein said minimum stream size is at least 512 bytes.

8. A computerized method according to claim 1 wherein said statistical analysis involves computing an average byte value for the data stream, performing a chi-square statistical analysis of the data bytes to compute a total chi-square value corresponding to a difference of each byte'"'"'s value from the average byte value, and computing said resultant value based upon the total chi-square value.

9. A computer-readable medium having executable instructions for performing a method comprising:

reading a data stream portion associated with each of a plurality of files, each said data stream portion characterized by a plurality of data bytes, each having an associated byte value;

computing a resultant value that is indicative of a level of uniformity for a frequency distribution of associated byte values throughout the data stream portion; and

controlling an output device to display output corresponding to an existence of data encryption if the resultant value indicates uniformity of the frequency distribution.

10. A computer-readable medium according to claim 9 wherein said executable instructions perform a statistical analysis of the data stream portion of each of the files.

11. A computer-readable medium according to claim 9 wherein each of said files includes a header portion and said data stream portion, said executable instructions only analyzing said data stream portion.

12. A computer-readable medium according to claim 10 wherein said executable instructions only perform said statistical analysis if the data stream portion has a stream size of at least 512 bytes.

13. A computer-readable medium according to claim 12 further comprising reading the header portion of the file and displaying output corresponding to an existence of cyphertext only if the header portion is uncorrelated to a known file type.

14. A computer-readable medium according to claim 13 wherein the known file type is one of an image file format and a compressed file format.

15. A computer-readable medium according to claim 10 wherein said statistical analysis involves computing an average byte value for the data stream portion, performing a chi-square statistical analysis of the data bytes to compute a chi-square value corresponding to a statistical difference of each byte'"'"'s value from the average byte value, and computing said resultant value based upon the chi-square value.

16. A computerized method for determining whether a selected file contains cyphertext, wherein the selected file includes a header portion corresponding to a file type and a data portion that includes a data stream characterized by a plurality of data bytes, each having an associated byte value, said computerized method comprising:

computing an average byte value for the data stream;

performing a chi-square statistical analysis of the data bytes to compute a total chi-square value corresponding to a statistical difference of each byte'"'"'s value from the average byte value;

computing a resultant value based upon the chi-square value, whereby said resultant value indicates a level of uniformity for a frequency distribution of the associated byte values throughout the data stream;

comparing said resultant value to a predetermined threshold value; and

generating output indicative of data encryption upon a determination that the threshold value exceeds the resultant value.

17. A computerized method according to claim 16 whereby said chi-square statistical analysis is only performed if the data stream is at least a minimum stream size and whereby said output is displayed on if the header portion is uncorrelated to a known file type.

18. A computerized method according to claim 17 wherein said minimum stream size is 512 bytes.

19. A computerized method according to claim 17 wherein said known file type is selected one of an image file format and a compressed file format.

20. A computerized method for searching a target set of directories within a hierarchical file system of a computational device to determine whether any files within each of said directories is in a cyphertext format, wherein each of said data files includes a header portion and a data portion that includes a data stream characterized by a plurality of data bytes, each having an associated byte value, said computerized methodology comprising, sequentially with respect to each of said files in each of said directories:

computing an average byte value for the data stream;

performing a statistical analysis of the data bytes to compute a statistical value corresponding to a statistical difference of each byte'"'"'s value from the average byte value;

computing a resultant value based upon the statistical value, whereby said resultant value indicates a level of uniformity for a frequency distribution of the associated byte values throughout the data stream;

comparing said resultant value to a predetermined threshold value and generating one of;

first output indicative of data non-encryption if said resultant value is less than the threshold value; and

second output indicative of data encryption if said threshold value exceeds the resultant value and said header portion is uncorrelated to a known file type.

21. A computerized method according to claim 20 whereby said statistical analysis is only performed if the data stream is at least a minimum stream size.

22. A computerized method according to claim 21 further comprising maintaining a tabulation of a total number of files searched, a total number of files deemed to be to small to search, and a total number of each known file type encountered.

23. A computerized method according to claim 22 further comprising displaying output corresponding to said tabulation.

24. An encryption detection system for identifying an existence of cyphertext in a data stream that is characterized by a plurality of data bytes each having an associated byte value, said system comprising:

a storage device;

an output device; and

a processor programmed to;

perform a statistical analysis of the data stream'"'"'s associated byte values to compute a statistical result having a resultant value indicative of a level of uniformity for a frequency distribution of the associated byte values throughout the data stream; and

control an output device to display output corresponding to an existence of cyphertext if the resultant value indicates uniformity of the frequency distribution.

25. An encryption detection system according to claim 24 including an input device for generating input data corresponding location of the data stream.

27. An encryption detection system according to claim 24 including an input device for generating input data corresponding location of the data stream.

26. An encryption detection system for identifying an existence of cyphertext in respective data streams, wherein each of said data streams is characterized by a plurality of data bytes each having an associated byte value, said system comprising:

storage means;

output means; and

processing means for;

performing a statistical analysis of each data stream'"'"'s associated byte values to compute a statistical result having a resultant value indicative of a level of uniformity for a frequency distribution of the associated byte values throughout the data stream; and

controlling an output device to display output corresponding to an existence of cyphertext if the resultant value indicates uniformity of the frequency distribution.