Distributional Similarity Based Method and System for Determining Topical Relatedness of Domain Names

• US 20090282027A1
• Filed: 05/02/2009
• Published: 11/12/2009
• Est. Priority Date: 09/23/2008
• Status: Abandoned Application

• Current Assignee: Paxfire, Inc.
• Original Assignee: Paxfire, Inc.
• Inventors: Subotin, Michael, Sullivan, Alan
• Application Number:
  US12/434,626
• Publication Number:
  US 20090282027A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]

Abstract

Systems, computer software and methods for calculating relatedness scores of domain names, which are indicative of relatedness of pairs of domain names requested by clients are described. The method includes receiving DNS traffic data, where the DNS traffic data includes at least domain names requested by the clients and identities of the clients requesting the domain names; generating, based on the identities of the clients, vectors including the requested domain names, where entries in the vectors correspond to client sessions in which the client has requested the domain names; reducing a dimensionality of the vectors by applying a dimensionality reduction method for generating reduced vectors; applying a similarity metric to the reduced vectors to calculate the relatedness scores; and storing the relatedness scores of the domain names.

Claims

1. A method for calculating relatedness scores of domain names, which are indicative of relatedness of pairs of domain names requested by clients, the method comprising:

receiving domain name system (DNS) traffic data, wherein the DNS traffic data includes at least domain names requested by the clients and identities of the clients requesting the domain names;

generating, based on the identities of the clients, vectors including the requested domain names, wherein entries in the vectors correspond to client sessions in which the client has requested the domain names;

reducing a dimensionality of the vectors by applying a dimensionality reduction method for generating reduced vectors;

applying a similarity metric to the reduced vectors to calculate the relatedness scores; and

storing the relatedness scores of the domain names.

2. The method of claim 1, further comprising:

constructing, based on the vectors, a matrix W having elements w_ij when a domain name “

i”

appears at least once in a client session “

j” and

zero otherwise, wherein w_ij is a real number.

3. The method of claim 2, further comprising:

applying singular value decomposition to matrix W to obtain three matrices U, Σ

, and V.

4. The method of claim 3, wherein the step of reducing further comprises:

truncating the Σ

matrix to Σ

_k, which has a rank k, where k is an integer and is smaller than a rank r of the matrix Σ

; and

calculating UΣ

_k.

5. The method of claim 4, further comprising:

identifying rows of the calculated UΣ

_k matrix as the reduced vectors.

6. The method of claim 5, wherein the applying a similarity metric step further comprises:

calculating a cosine of an angle between i-th and j-th rows of UΣ

_k for determining the relatedness score between domains i and j.

7. The method of claim 1, further comprising:

calculating the relatedness score for all pairs of available domain names in an Internet service provider; and

generating a database that stores the calculated relatedness scores for the available domain names.

8. A server for calculating relatedness scores of domain names, which are indicative of relatedness of pairs of domain names requested by clients, the server comprising:

an input/output interface configured to receive domain name system (DNS) traffic data, wherein the DNS traffic data includes at least domain names requested by the clients and identities of the clients requesting the domain names;

a processor connected to the input/output interface and configured to,

generate, based on the identities of the clients, vectors including the requested domain names, wherein entries in the vectors correspond to client sessions in which the client has requested the domain names,

reduce a dimensionality of the vectors by applying a dimensionality reduction method for generating reduced vectors, and

apply a similarity metric to the reduced vectors to calculate the relatedness scores; and

a memory connected to the processor and configured to store the relatedness scores of the domain names.

9. The server of claim 8, wherein the processor is further configured to,

construct, based on the vectors, a matrix W having non-zero entries w_ij when a domain name “

i”

appears at least once in a client session “

j” and

zero entries otherwise, wherein w_ij is a real number.

10. The server of claim 9, wherein the processor is further configured to,

apply singular value decomposition to matrix W to obtain three matrices U, Σ

, and V.

11. The server of claim 10, wherein the processor is further configured to,

truncate the Σ

matrix to Σ

_k, which has a rank k, where k is an integer and is smaller than a rank r of the matrix Σ

; and

calculate UΣ

_k.

12. The server of claim 11, wherein the processor is further configured to,

identify rows of the calculated UΣ

_k matrix as the reduced vectors.

13. The server of claim 12, wherein the processor is further configured to

calculate a cosine of an angle between i-th and j-th rows of UΣ

_k for determining the relatedness score between domains i and j.

14. The server of claim 8, wherein the processor is further configured to

calculate the relatedness score for all pairs of available domain names in an Internet service provider;

and

generate a database that stores the calculated relatedness scores for the available domain names.

15. A computer readable medium including computer executable instructions, wherein the instructions, when executed, implement a method for calculating relatedness scores of domain names, which are indicative of relatedness of pairs of domain names requested by clients, the method comprising:

providing a system comprising distinct software modules, wherein the distinct software modules comprise a domain name system (DNS) traffic module, a vector generating module, and a mathematical module;

receiving DNS traffic data via the DNS traffic module, wherein the DNS traffic data includes at least domain names requested by the clients and identities of the clients requesting the domain names;

generating in the vector generating module, based on the identities of the clients, vectors including the requested domain names, wherein entries in the vectors correspond to client sessions in which the client has requested the domain names;

reducing in the mathematical module dimensionality of the vectors by applying a dimensionality reduction method for generating reduced vectors;

applying a similarity metric to the reduced vectors to calculate the relatedness scores; and

storing the relatedness scores of the domain names.

16. The medium of claim 15, further comprising:

constructing, based on the vectors, a matrix W having non-zero entries w_ij when a domain name “

i”

appears at least once in a client session “

j” and

zero entries otherwise, wherein w_ij is a real number.

17. The medium of claim 16, further comprising:

applying singular value decomposition to matrix W to obtain three matrices U, Σ

, and V.

18. The medium of claim 17, wherein the step of reducing further comprises:

truncating the Σ

matrix to Σ

_k, which has a rank k, where k is an integer and is smaller than a rank r of the matrix Σ

; and

calculating UΣ

_k.

19. The medium of claim 18, further comprising:

identifying rows of the calculated UΣ

_k matrix as the reduced vectors.

20. The medium of claim 19, wherein the processor is further configured to,

calculate a cosine of an angle between i-th and j-th rows of UΣ

_k for determining the relatedness score between domains i and j.