Information security apparatus and methods for credential dump authenticity verification

• US 9,871,797 B2
• Filed: 02/09/2016
• Issued: 01/16/2018
• Est. Priority Date: 02/09/2016
• Status: Active Grant

• Current Assignee: ZeroFOX, Inc. (ZeroFox Holdings, Inc.)
• Original Assignee: Lookingglass Cyber Solutions, LLC
• Inventors: Weinstein, Steven, Lewis, Jason, Parker, Douglas
• Primary Examiner(s): Kabir, Jahangir
• Application Number:
  US15/019,259
• Publication Number:
  US 20170230372A1
• Time in Patent Office: 707 Days
• Field of Search: 726 4- 7, 726 22, 726 25
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 21/00 Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: H04L 63/083 using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1425 Traffic logging, e.g. anomaly detection

Abstract

In some embodiments, an apparatus includes a memory, storing processor-executable instructions, blacklist terms, and credential dump records, and a processor. The processor receives repository data from targeted remote repositories and stores the repository data as a potential credential dump in the memory when the repository data includes a credential dump attribute. The processor stores the potential credential dump as a probable credential dump when the potential credential dump does not include a blacklist term, in which case the processor also detects a format and delimiter of the probable credential dump. Based on the format and delimiter, pairs of usernames and associated passwords are identified and hashed. If a percentage of the hashes not associated with the credential dump records exceeds a predetermined threshold, the probable credential dump is deemed authentic.

Claims

1. An apparatus, comprising:

a memory storing processor-executable instructions, a plurality of blacklist terms associated with an instruction to ignore data, and a plurality of credential dump records, each credential dump record from the plurality of credential dump records including an associated plurality of hashes; and

at least one processor, operably coupled to the memory and configured to execute the processor-executable instructions to;

receive repository data from a plurality of targeted remote repositories;

in response to a determination that a credential dump attribute is identified in the repository data;

access each blacklist term from the plurality of blacklist terms that are previously-identified and stored;

determine whether a blacklist term from the plurality of blacklist terms is identified in the repository data; and

in response to a determination that a blacklist term from the plurality of blacklist terms is not identified in the repository data;

detect a common format and a common delimiter of the repository data;

identify a plurality of pairs of usernames and associated passwords of the repository data based on the common format and the common delimiter;

generate a hash for each pair of usernames and associated passwords from the plurality of pairs of usernames and associated passwords to produce a plurality of hashes;

compare the plurality of hashes to the plurality of credential dump records stored in the memory;

determine a percentage of the plurality of hashes that are not associated with the plurality of credential dump records;

send a signal indicating that the repository data is an authentic credential dump in response to the determination that the percentage is larger than a predetermined threshold;

identify an intrusion into a computer system associated with the repository data in response to the determination that the percentage is larger than the predetermined threshold; and

the credential dump attribute including at least one of;

the repository data including more than a predetermined number of usernames and the repository data including a password-type field.

2. The apparatus of claim 1, wherein the instruction to generate uses a Secure Hash Algorithm (SHA) hash function.

3. The apparatus of claim 2, wherein the hash function is SHA-256.

4. The apparatus of claim 1, wherein the repository data is received from the plurality of targeted remote repositories, and the instruction to receive is performed repeatedly and at a predetermined rate.

5. The apparatus of claim 1, wherein the repository data is received from the plurality of targeted remote repositories when a change is detected at a targeted remote repository or the plurality of targeted remote repositories.

6. The apparatus of claim 1, wherein:

the repository data is received from a first targeted remote repository of the plurality of targeted remote repositories at a first rate that is a function of the first targeted remote repository, and

the repository data is received from a second targeted remote repository of the plurality of targeted remote repositories at a second rate that is a function of the second targeted remote repository.

7. The apparatus of claim 1, wherein detecting the common delimiter of the repository data includes identifying a predetermined number of consecutive lines of the repository data that each include a common delimiter type, the detecting includes detecting the common delimiter when the predetermined number exceeds a threshold.

8. The apparatus of claim 1, wherein detecting the common format and the common delimiter of the repository data includes identifying, a predetermined number of consecutive lines of the repository data in which respective usernames of the consecutive lines of the repository data are indexed at a common index position, and the detecting includes detecting the common format and the common delimiter when the predetermined number exceeds a threshold.

9. The apparatus of claim 8, wherein the predetermined number of consecutive lines is at least 5.

10. The apparatus of claim 1, wherein each pair of usernames and associated passwords from the plurality of pairs of usernames and associated passwords includes the associated username concatenated with the associated password.

11. The apparatus of claim 1, wherein each username of the plurality of pairs of usernames and associated passwords is an email address.

12. The apparatus of claim 1, wherein receiving repository data from the plurality of targeted remote repositories is performed using web scraping.

13. The apparatus of claim 1, wherein:

the plurality of blacklist terms includes a list of terms that are known to not be included in the authentic credential dump.

14. A method, comprising:

receiving, using a processor, remote source data from a plurality of targeted remote sources;

in response to a determination that a credential dump attribute is identified in the remote source data;

accessing each blacklist term from the plurality of blacklist terms that are previously-identified and stored; and

determining whether a blacklist term from the plurality of blacklist terms is identified in the remote source data;

in response to a determination that a blacklist term from the plurality of blacklist terms is not identified in the remote source data;

storing (1) a plurality of credential pairs of the remote source data and (2) metadata associated with the plurality of credential pairs in a memory that is operably coupled to the processor;

detecting a format of the remote source data including identifying a plurality of usernames and the plurality of passwords;

normalizing, using, the processor, the plurality of credential pairs into a concatenated, delimiter-free format, the normalizing is based on the plurality of usernames and the plurality of passwords;

converting, using the processor, the normalized plurality of credential pairs into a plurality of hashes;

comparing, using the processor, the plurality of hashes to previously-collected credential dump data to determine a percentage of the plurality of hashes that are not included in the previously-collected credential dump data;

sending, using the processor, a signal indicating, that the remote source data includes an authentic credential dump in response to the determination that the percentage of the plurality of hashes that are not included in the previously-collected credential dump data is larger than a predetermined threshold; and

identifying an intrusion into a computer system associated with the remote source data in response to the determination that the percentage of the plurality of hashes that are not included in the previously-collected credential dump data is larger than the predetermined threshold.

15. The method of claim 14, wherein:

the detecting includes detecting a delimiter that recurs on a consecutive plurality of lines of the remote source data, the normalizing being based on the delimiter.

16. The method of claim 15, wherein:

each username of the plurality of usernames is disposed in the remote source data before a delimiter of the detected recurring delimiters; and

each password of the plurality of passwords is disposed in the remote source data after a delimiter of the detected recurring delimiters.

17. The method of claim 15, wherein:

each username of the plurality of usernames is disposed in the remote source data after a delimiter of the detected recurring delimiters; and

each password of the plurality of passwords is disposed in the remote source data before a delimiter of the detected recurring delimiters.

18. A method, comprising:

storing a plurality of blacklist terms associated with an instruction to ignore data, and a plurality of credential dump records;

receiving, using a processor, remote source data from a plurality of targeted remote sources;

in response to a determination that a credential dump attribute is identified in the remote source data;

accessing each blacklist term from the plurality of blacklist terms that are previously-identified and stored; and

determining whether a blacklist term from the plurality of blacklist terms is identified in the repository data;

in response to a determination that a blacklist term from the plurality of blacklist terms is not identified in the remote source data;

storing (1) a plurality of credential pairs of the remote source data, each credential pair of the plurality of credential pairs including an associated username and an associated password, and (2) metadata associated with the plurality of credential pairs, in a memory that is operably coupled to the processor;

comparing, using the processor, the plurality of credential pairs to previously-collected credential dump data to determine a percentage of the plurality of credential pairs that are not included in the previously-collected credential dump data;

sending, using the processor, a signal indicating that the remote source data includes an authentic credential dump in response to the determination that the percentage of the plurality of credential pairs that are not included in the previously-collected credential dump data is larger than a predetermined threshold;

identifying, using the processor, an intrusion into a computer system associated with the remote source data in response to the determination that the percentage of the plurality of credential pairs that are not included in the previously-collected credential dump data is larger than the predetermined threshold; and

the credential dump attribute including at least one of;

the repository data including more than a predetermined number of usernames and the repository data including a password-type field.

19. The method of claim 18, wherein the receiving the remote source data is performed periodically and at a predetermined rate.

20. The method of claim 18, wherein:

the receiving the remote source data includes receiving the remote source data from a first targeted remote source of the plurality of targeted remote sources at a first rate that is a function of the first targeted remote source, and from a second targeted remote source of the plurality of targeted remote sources at a second rate that is a function of the second targeted remote source.

21. The method of claim 18, wherein the receiving the remote source data is performed using web scraping.