Forensic feature extraction and cross drive analysis

• US 7,779,032 B1
• Filed: 09/06/2006
• Issued: 08/17/2010
• Est. Priority Date: 07/13/2005
• Status: Active Grant

• Current Assignee: Sleuth KIT Labs LLC
• Original Assignee: Basis Technology Corporation
• Inventors: Garfinkel, Simson L.
• Primary Examiner(s): Pham; Khanh B
• Application Number:
  US11/470,517
• Time in Patent Office: 1,441 Days
• Field of Search: 707/687, 707/688, 707/776, 707/822, 707/823
• US Class Current: 707/776
• CPC Class Codes: G06F 16/10 File systems; File servers

Abstract

Computer-based systems and methods enable analysts to manage and explore the information that hard drives and other storage devices or sources of data may contain, and for extracting forensic features and performing cross drive analysis.

Claims

1. A computer-based method for determining whether a second digital information storage medium accessible for forensic analysis relates to a social network with which a first digital information storage medium has previously been identified, the method comprising:

accessing first and second respective digital images generated from digital information retrieved from or present on the first and second digital information storage media, respectively, the first and second digital images comprising first and second representations, respectively, of digital information from the first and second digital information storage media;

executing, on the first and second representations, a feature extractor function to extract occurrences, from within the representations of digital information from the first and second digital information storage media, respectively, of a selected feature, thereby to generate a feature extractor output;

detecting, based on the feature extractor output, pseudo-unique information from the first and second digital information storage media, respectively;

detecting a degree of commonality between pseudo-unique information from the first and second digital information storage media, respectively; and

,

if a sufficient commonality of pseudo-unique information is detected, designating the second digital information storage medium as relating to the social network with which the first digital information storage medium has previously been identified.

2. The method of claim 1 wherein the feature extractor function stores feature extractor output in a feature file for further processing or analysis.

3. The method of claim 1 wherein the feature extractor function comprises any of an email address extractor, an email message ID extractor, an email Subject extractor, an email date extractor, a cookie extractor, a Social Security number extractor, or a credit card number extractor.

4. The method of claim 1 wherein the feature extractor function comprises differentiating between values that are common and those that are relatively rare within the data extracted from across a plurality of digital information storage media.

5. The method of claim 1 wherein the digital information storage media comprise disk drives having disk sectors, and further comprising utilizing hash codes of individual disk sectors for analysis, detection or correlation, rather than directly using the extracted digital information.

6. The method of claim 1 further comprising:

first extracting 8-bit, 16-bit, and/or Unicode strings from the digital images; and

then using the strings for subsequent processing or analysis, rather than executing processing or analysis based directly on the digital images.

7. The method of claim 1 further comprising prioritizing the digital information storage media for further analysis based on extracted features.

8. The method of claim 1 further comprising utilizing extracted features to determine the identity of a principal user of a given digital information storage medium.

9. The method of claim 8 wherein determining the identity of a principal user of a given digital information storage medium comprises generating a histogram of occurrences of an extracted feature.

10. The method of claim 1 further comprising scanning a digital information storage medium to identify the likely existence of information designated or required to have been expunged.

11. The method of claim 1 wherein the digital information storage media comprise disk drives having disk sectors, and further comprising determining whether a given file is likely to have been present on a disk drive, based on detection of the presence on the disk of a pseudo-unique sector, or sector containing pseudo-unique information identifiable with the given file.

12. A computer-based method for discovering social networks with which ones of a plurality of digital information storage media accessible for forensic analysis may be identified, the method comprising:

accessing respective digital images generated from digital information retrieved from or present on the plurality of digital information storage media, respectively, the respective digital images comprising respective representations of digital information from corresponding ones of the plurality of digital information storage media;

executing, on the representations, a feature extractor function to extract occurrences, from within the representations of digital information from respective ones of the digital information storage media, of a selected feature, thereby to generate a feature extractor output;

detecting, based on the feature extractor output, pseudo-unique information from respective ones of the plurality of digital information storage media;

detecting a degree of commonality between pseudo-unique information from respective ones of the plurality of digital information storage media; and

,

if at least a selected degree of commonality of pseudo-unique information is detected, designating respective ones of the plurality of digital information storage media for which such commonality has been detected as relating to a potential social network, thereby to identify a potential social network.

13. The method of claim 12 further comprising identifying, based on the detecting of at least a selected degree of commonality of pseudo-unique information thereon, clusters of digital information storage media as identifiable with a given social network or organizational entity.

14. A computer-based method of analyzing digital information present on or retrieved from a plurality of digital information storage media, the method comprising:

generating digital images representative of digital information from respective ones of the plurality of digital information storage media;

applying a feature extractor function to extract features from the digital images or string files generated from the digital images, the feature extractor function comprising scanning for selected pseudo-unique identifiers to identify features for extraction, and storing extracted features for subsequent access or further analysis, whereby the feature extraction function extracts features from across at least a subset of the plurality of digital information storage media;

applying a first order cross-media analysis, wherein the results of applying a feature extractor function are compared across multiple ones of the plurality of digital information storage media, to identify digital information storage media among the plurality of digital information storage media having a selected or maximal value or number of occurrences of a selected feature;

applying a second order cross media analysis to generate correlations between ones of the plurality of digital information storage media, wherein the correlations are generated based on analysis of extracted features, to enable correlation of selected features across the plurality of digital information storage media and detection of a selected degree of commonality of features extracted from given digital information storage media within the plurality of digital information storage media; and

whereby the method is operable to enable an operator to analyze and correlate information from across multiple ones of the plurality of digital information storage media.

15. The method of claim 14 wherein applying a second order cross media analysis further comprises executing a multi-source correlation function, wherein the correlation function comprises reading a plurality of feature files and generating from the reading of the plurality of feature files an output containing, for each selected feature, a list comprising the number of digital information storage media on which the selected feature was detected, the total number of times the selected feature was detected on all digital information storage media under analysis, and a listing of digital information storage media on which the selected feature occurs.

16. The method of claim 15 wherein the correlation function is characterized by an input comprising a set of digital images in a feature to be correlated, and an output comprising a list of (feature, drive-list) tuples.

17. The method of claim 16 wherein the correlation function further comprises executing a scoring function comprising scoring the correlation to generate a report of digital information storage media that are most highly correlated.

18. The method of claim 17 wherein the scoring function comprises summing the number of features that two selected digital information storage media have in common.

19. The method of claim 17 wherein the scoring function comprises a weighting function that discounts features by the number of digital information storage media on which they appear.

20. The method of claim 17 wherein the scoring function comprises a weighting function that accords increased weight to rare features present in high concentrations on the selected digital information storage media.

21. The method of claim 14 wherein the method further comprises automatically identifying “

hot”

digital information storage media, the “

hot”

media comprising media within the plurality of digital information storage media under analysis that contain one or more features of interest to an operator, and that should be accorded higher priority for further analysis.

22. The method of claim 21 further comprising assigning priority values to ones of the plurality of digital information storage media as a function of detection of media having the largest numbers of occurrences of selected features.

23. The method of claim 22 further comprising automatically identifying media containing a high concentration of selected features that are selected as being of interest to an analyst.

24. The method of claim 14 further comprising generating a “

stop list”

of feature values that can be disregarded, substantially without loss of generality, by subsequent analysis processes, wherein the listed feature values are substantially ubiquitous across the plurality of digital information storage media under analysis.

25. The method of claim 24 further comprising using the stop list to suppress output relating to the listed feature values of the stop list.

26. The method of claim 14 further comprising identifying social network membership and determining whether a newly-ingested digital information storage medium was previously utilized by an element of the social network.

27. A computer-based method of analyzing digital information present on or retrieved from a plurality of digital information storage media, the method comprising:

generating digital images representative of digital information from respective ones of the plurality of digital information storage media;

generating from the images an image file;

extracting strings from the image file;

applying a feature extractor function to extract features from the strings extracted from the digital images, the feature extractor function comprising scanning for selected pseudo-unique identifiers to identify features for extraction, wherein the feature extraction function extracts features from across at least a subset of the plurality of digital information storage media and writes the results to one or more feature files;

applying a first order cross-media analysis, wherein the results of applying a feature extractor function are compared across multiple ones of the plurality of digital information storage media, to identify digital information storage media among the plurality of digital information storage media having a selected or maximal value or number of occurrences of a selected feature; and

applying a second order cross media analysis to generate correlations between ones of the plurality of digital information storage media, wherein the correlations are generated based on analysis of extracted features, to enable correlation of selected features across the plurality of digital information storage media and detection of a selected degree of commonality of features extracted from given digital information storage media within the plurality of digital information storage media;

whereby the method is operable to enable an operator to analyze and correlate information from across multiple ones of the plurality of digital information storage media.

28. The method of claim 27 wherein the strings are extracted using a file format-aware program in multiple passes, including one for 8-bit characters, one for 16-bit characters in LSB format, and one for 16-bit characters in MSB format.

29. The method of claim 28 wherein extracted features from newly-ingested drives are run against a watch list, and hits against the watch list are reported to a human operator.

30. The method of claim 29 wherein the feature files are read by indexers operable to build indexes, in a database server, of the identified features.

31. The method of claim 30 further comprising executing a multi-drive correlation to determine whether a newly accessioned drive contains features in common with any drives that are on a drive watch list.

32. The method of claim 31 further comprising providing a user interface operable to enable a human operator to interact with the images, files, or results.

33. The method of claim 32 wherein the user interface is a multi-user interface operable to enable multiple operators to simultaneously interact with the images, files or results.

34. A computer-based system for determining whether a second digital information storage medium accessible for forensic analysis relates to a social network with which a first digital information storage medium has previously been identified, the system comprising:

means for accessing first and second respective digital images generated from digital information retrieved from or present on the first and second digital information storage media, respectively, the first and second digital images comprising first and second representations, respectively, of digital information from the first and second digital information storage media;

means for executing, on the first and second representations, a feature extractor function to extract occurrences, from within the representations of digital information from the first and second digital information storage media, respectively, of a selected feature, thereby to generate a feature extractor output;

means for detecting, based on the feature extractor output;

pseudo-unique information from the first and second digital information storage media, respectively;

means for detecting a degree of commonality between pseudo-unique information from the first and second digital information storage media, respectively; and

,

means for, if a sufficient commonality of pseudo-unique information is detected, designating the second digital information storage medium as relating to the social network with which the first digital information storage medium has previously been identified.

35. A computer program product operable within a computer, the computer program product being operable to enable the computer to determine whether a second digital information storage medium accessible for forensic analysis relates to a social network with which a first digital information storage medium has previously been identified, the computer program product comprising:

a non-transitory computer readable medium having computer-executable program code stored thereon, the computer-executable program code comprising program code executable by the computer to enable the computer to;

access first and second respective digital images generated from digital information retrieved from or present on the first and second digital information storage media, respectively, the first and second digital images comprising first and second representations, respectively, of digital information from the first and second digital information storage media;

execute, on the first and second representations, a feature extractor function to extract occurrences, from within the representations of digital information from the first and second digital information storage media, respectively, of a selected feature, thereby to generate a feature extractor output;

detect, based on the feature extractor output, pseudo-unique information from the first and second digital information storage media, respectively;

detect a degree of commonality between pseudo-unique information from the first and second digital information storage media, respectively; and

,

if a sufficient commonality of pseudo-unique information is detected, designate the second digital information storage medium as relating to the social network with which the first digital information storage medium has previously been identified.