SYSTEM FOR CORRELATION OF DOMAIN NAMES

• US 20160065534A1
• Filed: 11/10/2015
• Published: 03/03/2016
• Est. Priority Date: 07/06/2011
• Status: Abandoned Application

• Current Assignee: Akamai Technologies, Inc.
• Original Assignee: Nominum Incorporated (Akamai Technologies, Inc.)
• Inventors: Liu, Hongliang, Kullberg, Mikael, Yuzifovich, Yuriy, Paugh, James, Wilbourn, Robert S.
• Application Number:
  US14/937,616
• Publication Number:
  US 20160065534A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: H04L 2463/144 Detection or countermeasures against botnets
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002

Abstract

Provided are methods and systems for correlation of domain names. An example method includes receiving Domain Name System (DNS) data associated with a plurality of domain names, generating multidimensional vectors based on the DNS data such that each of the domain names is associated with one of the multidimensional vectors, calculating similarity scores for each pair of the plurality of domain names based on comparison of corresponding multidimensional vectors, and clustering one or more sets of domain names selected from the plurality of domain names based on the similarity scores and such that a difference between the similarity scores corresponding to each pair of the domain names in each of clusters is below a predetermined threshold.

Claims

1. A computer-implemented method for correlating domain names, the method comprising:

receiving Domain Name System (DNS) data associated with a plurality of domain names;

based on the DNS data, generating multidimensional vectors, wherein each of the domain names is associated with one of the multidimensional vectors;

calculating similarity scores for each pair of the plurality of domain names based on a comparison of corresponding multidimensional vectors; and

based on the similarity scores, clustering one or more sets of domain names selected from the plurality of domain names such that a difference between the similarity scores corresponding to each pair of the domain names in each of clusters being below a predetermined threshold.

2. The method of claim 1, further comprising:

receiving a correlation request associated with a target domain name;

determining that the target domain name is included in a dictionary, wherein the dictionary includes the plurality of domain names associated with the multidimensional vectors; and

based on the determination that the target domain name is included in the dictionary, selecting a cluster associated with the target domain name.

3. The method of claim 1, further comprising:

receiving a correlation request associated with a target domain name;

determining that the target domain name is not included in a dictionary, wherein the dictionary includes the plurality of domain names associated with the multidimensional vectors;

ascertaining DNS data associated with the target domain name;

generating a multidimensional vector for the target domain name;

calculating similarity scores between the multidimensional vector for the target domain name and the multidimensional vectors of the plurality of the domain names in the dictionary; and

assigning the target domain name to a cluster based on the calculation.

4. The method of claim 1, further comprising:

training a classifier using the DNS data, wherein the classifier is configured to convert each of the domain names into one of the multidimensional vectors;

wherein the DNS data is associated with a plurality of DNS queries, and wherein the DNS data comprises, for each of the DNS queries, an Internet Protocol (IP) address of a client created a DNS request, a time stamp of the DNS request, a DNS query name, and a DNS query type.

5. The method of claim 4, wherein the training of the classifier comprises performing a forward propagation process to obtain a dictionary of the domain names with corresponding multidimensional vectors.

6. The method of claim 4, further comprising:

grouping the DNS queries by IP addresses of clients.

7. The method of claim 4, further comprising:

sorting the DNS queries by the time stamp.

8. The method of claim 1, further comprising:

filtering the DNS data by removing DNS queries of predetermined types.

9. The method of claim 8, wherein the predetermined types of DNS queries include:

DNS queries associated with malicious attacks, Address and Routing Parameter Area (ARPA) queries, and same DNS queries that appear less than a predetermined number of times in the training data.

10. The method of claim 1, wherein the receiving the DNS data associated with the plurality of domain names comprises collecting the DNS queries from multiple Internet Service Providers (ISPs) for a predetermined period of time, wherein the predetermined period of time is between about 1 minute and about 24 hours.

11. The method of claim 1, wherein the multidimensional vectors of the domain names include numeric representation vectors that reflect semantic similarities between the domain names.

12. The method of claim 1, further comprising:

selecting the pairs of the plurality of domain names based on a skip-gram model.

13. The method of claim 1, further comprising:

ranking two or more of the domain names in at least one of the clusters to create a ranked list of the domain names.

14. The method of claim 1, wherein each of the clusters of the domain names reflects operational behavior of the domain names in the cluster.

15. The method of claim 1, further comprising:

projecting the multidimensional vectors onto two-dimensional (2D) space by performing a dimension reduction technique.

16. The method of claim 15, further comprising:

visualizing at least one of the clusters of the domain names via a user graphical interface by displaying graphical representations of the multidimensional vectors projected onto the 2D space.

17. The method of claim 16, wherein the visualizing comprises displaying domain name maps, wherein each of the domain name maps is associated with an individual graphical representation such that the domain name maps are visually different from each other.

18. The method of claim 1, further comprising:

receiving DNS data associated with a plurality of domain names having trusted categorization data;

based on the DNS data, generating multidimensional vectors, wherein each of the domain names having the trusted categorization data is associated with one of the multidimensional vectors;

receiving at least one domain name with no categorization data or having untrusted categorization data;

generating a multidimensional vector of the at least one domain name with no categorization data or having untrusted categorization data;

calculating similarity scores between the multidimensional vector of the at least one domain name with no categorization data or having untrusted categorization data and each of the multidimensional vectors associated with the domain names having trusted categorization data; and

based on the similarity scores, assigning a category to the at least one domain name with no categorization data or having untrusted categorization data.

19. A computer-implemented system comprising at least one processor and a memory storing processor-executable codes, wherein the at least one processor is configured to:

receive Domain Name System (DNS) data associated with a plurality of domain names;

based on the DNS data, generate multidimensional vectors, wherein each of the domain names is associated with one of the multidimensional vectors;

calculate similarity scores for each pair of the plurality of domain names based on comparison of corresponding multidimensional vectors; and

based on the similarity scores, cluster one or more sets of domain names selected from the plurality of domain names such that a difference between the similarity scores corresponding to each pair of the domain names in each of clusters being below a predetermined threshold.

20. A non-transitory processor-readable medium having instructions stored thereon, which when executed by one or more processors, cause the one or more processors to implement a method, comprising:

receiving Domain Name System (DNS) data associated with a plurality of domain names;

based on the DNS data, generating multidimensional vectors, wherein each of the domain names is associated with one of the multidimensional vectors;

calculating similarity scores for each pair of the plurality of domain names based on comparison of corresponding multidimensional vectors; and

based on the similarity scores, clustering one or more sets of domain names selected from the plurality of domain names such that a difference between the similarity scores corresponding to each pair of the domain names in each of clusters being below a predetermined threshold.