System and method for collecting and processing information of an internet user via IP-web correlation

• US 9,049,117 B1
• Filed: 10/21/2009
• Issued: 06/02/2015
• Est. Priority Date: 10/21/2009
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: Narus, Inc. (Gen Digital Inc.)
• Inventors: Nucci, Antonio, Robinson, Joshua, Ranjan, Supranamaya, Bhagat, Smriti, Keralapura, Ramakrishna
• Primary Examiner(s): Serrao, Ranodhi
• Assistant Examiner(s): RICHARDSON, THOMAS W
• Application Number:
  US12/603,578
• Time in Patent Office: 2,050 Days
• US Class Current: 1/1
• CPC Class Codes: H04L 43/0811 by checking connectivity
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 63/302 gathering intelligence information for situation awareness or reconnaissance

Abstract

A method is provided for collecting and processing information of a target who is a user of a communication network. The method includes obtaining a first identifier of the target, accessing, based on a handle of the first identifier, a first public webpage associated with the target in a first Internet site identified based on a domain of the first identifier, extracting content of the first public webpage for including in target data of the target, obtaining a third identifier of the target, intercepting a document associated with the target from a private portion of communication network traffic identified based on a domain of the third identifier, extracting content of the document for including in the target data, determining a second identifier by searching the target data, associating the second identifier with the target based on a pre-determined criterion, and collecting information of the target based on the second identifier.

Claims

1. A method for collecting and processing information of a target comprising a user of a communication network, comprising:

obtaining a first identifier of the target, wherein the first identifier comprises a first domain and a first handle;

identifying a first Internet site based on the first domain;

accessing, in the first Internet site and based on the first handle, a first public webpage associated with the target;

selecting a first parsing rule from a plurality of parsing rules that is associated with the first Internet site;

extracting, by a central processing unit (CPU) of a computer, a second identifier from the first public webpage using the first parsing rule;

determining, by the CPU and in response to extracting the second identifier, a similarity measure by comparing the second identifier to the first identifier using a pre-determined algorithm;

determining, in response to the similarity measure exceeding a pre-determined threshold and without based on any prior association between the first identifier and the second identifier, that the first identifier and the second identifier identify the same person as the user of the communication network based on a pre-determined criterion; and

collecting information of the target based on the second identifier for including in target data of the target.

2. The method of claim 1, further comprising:

obtaining a third identifier of the target, wherein the third identifier comprises a third domain and a third handle;

intercepting, based on the third handle, a document associated with the target from a private portion of communication network traffic identified based on the third domain; and

extracting content of the document for including in the target data of the target, wherein the content of the document is extracted using a third parsing rule, of the plurality of parsing rules, associated with the private portion of the communication network traffic identified based on the third domain.

3. The method of claim 1,

wherein the second identifier is determined from at least one selected from a group consisting of email address, webpage login name, chat user name, social networking user name, telephone number, and voice over IP (VoIP) phone number,

wherein the second domain is determined explicitly when determined from the email address, and

wherein the second domain is determined implicitly when determined from at least one selected from a group consisting of webpage login name, chat user name, social networking user name, telephone number, and VoIP phone number.

4. The method of claim 1, further comprising:

adding the second identifier to a list of identifiers belonging to the target,

wherein the second identifier comprises a second domain and a second handle.

5. The method of claim 4,

wherein the pre-determined algorithm comprises a Longest Common Subsequence Windowed (LCS^W) signature algorithm.

6. The method of claim 5, further comprising:

generating a string by concatenating a plurality of attributes of the target to the first identifier,

wherein the plurality of attributes of the target are stored in a target profile of the target,

wherein similarity measure is further determined by comparing the second identifier to the string using the LCS^W signature algorithm, and

wherein the plurality of attributes of the target are determined based in part by searching the target data of the target using the plurality of parsing rules.

7. The method of claim 6, further comprising:

generating an alpha-text string and a numerical string from the string,

wherein the similarity measure is further determined by at least one selected from a group consisting of comparing a alpha-text portion of the second identifier to the alpha-text string using the LCS^W signature algorithm and comparing a numerical portion of the second identifier to the numerical string using the LCS^W signature algorithm.

8. The method of claim 4,

wherein the second identifier is found in a user profile portion of the first public webpage, and

wherein the user profile portion of the first public webpage is identified based on the first parsing rule associated with the first Internet site.

9. The method of claim 4, wherein collecting information of the target based on the second identifier comprises:

accessing, based on the second handle, a second public webpage in a second Internet site identified based on the second domain; and

extracting content of the second public webpage for including in the target data of the target, wherein the content of the second public webpage is extracted using a second parsing rule, of the plurality of parsing rules, associated with the second Internet site.

10. The method of claim 4, wherein collecting information of the target based on the second identifier comprises:

intercepting, based on the second handle, a document associated with the target from a private portion of the communication network traffic identified based on the second domain; and

extracting content of the document for including in the target data of the target, wherein the content of the document is extracted using a second parsing rule, of the plurality of parsing rules, associated with the private portion of the communication network traffic identified based on the second domain.

11. The method of claim 10, wherein the document comprises at least one selected from a group consisting of an email, a telephone recording, and a VoIP recording.

12. The method of claim 4, further comprising:

storing the second identifier in a target profile of the target;

detecting the second identifier in another target profile of another target comprising another user of the communication network by searching a plurality of target profiles of a plurality of targets comprising a plurality of users of the communication network;

merging the target profile of the target with the another target profile; and

;

merging the target data of the target with target data of the another target.

13. The method of claim 11, further comprising:

determining, using the CPU, a biometric identifier of the target by analyzing the target data of the target using at least one selected from a group consisting of speaker recognition algorithm, authorship recognition algorithm, and facial recognition algorithm; and

determining an identity of the target based on the biometric identifier.

14. The method of claim 1, further comprising:

determining, using the CPU, the second identifier by further searching target data of a plurality of associates in an associate list of the target using the plurality of parsing rules.

15. The method of claim 1, further comprising:

determining a third identifier by searching the first public webpage using the first parsing rule, wherein the third identifier comprises a third domain and a third handle;

determining, using the CPU, the third identifier as belonging to an associate of the target, wherein the associate comprises another user of the communication network, wherein the third identifier is determined as belonging to the associate by applying a plurality of association rules to the target data of the target with respect to the another user;

adding the another user to a list of associates of the target; and

adding the third identifier to a list of identifiers belonging to associates of the target.

16. The method of claim 15, further comprising:

determining, using the CPU, at least one selected from a group consisting of relationships, topics, and localities by analyzing the target data of the target,

wherein the plurality of association rules comprise at least one selected from a group consisting of a relational association rule, a topical association rule, and a spatial association rule.

17. A non-transitory computer readable medium, embodying instructions when executed by the computer to collect and process information of a target comprising a user of a communication network, the instructions comprising functionality for:

obtaining a first identifier of the target, wherein the first identifier comprises a first domain and a first handle;

identifying a private portion of communication network traffic based on the first domain;

intercepting, from the private portion of communication network traffic and based on the first handle, a document associated with the target;

selecting a first parsing rule from a plurality of parsing rules that is associated with the private portion of the communication network traffic;

extracting a second identifier from the document using the first parsing rule;

determining, in response to extracting the second identifier, a similarity measure by comparing the second identifier to the first identifier using a pre-determined algorithm;

determining, in response to the similarity measure exceeding a pre-determined threshold and without based on any prior association between the first identifier and the second identifier, that the first identifier and the second identifier identify the same person as the user of the communication network based on a pre-determined criterion; and

collecting information of the target based on the second identifier.

18. The non-transitory computer readable medium of claim 17, the instructions when executed by the processor further comprising functionality for:

obtaining a third identifier of the target, wherein the third identifier comprises a third domain and a third handle;

accessing, based on the third handle, a public webpage associated with the target in a Internet site identified based on the third domain;

extracting content of the public webpage for including in the target data of the target, wherein the content of the public webpage is extracted using a third parsing rule, of the plurality of parsing rules, associated with the Internet site.

19. A system for collecting and processing information of a target comprising a user of a communication network, comprising:

a repository storing a target profile of the target and target data of the target, wherein the target profile comprises a list of identifiers associated with the target, wherein the list of identifiers associated with the target comprises a list of identifiers belonging to the target and a list of identifiers belonging to associates of the target;

a target data population engine comprising;

a web crawler configured to extract contents of Internet web pages based on the identifiers associated with the target for including in the target data of the target, wherein the contents of the Internet web pages are extracted using a plurality of parsing rules corresponding to the Internet web pages;

a target data analysis engine comprising;

an identifier retrieval engine configured to associate an identifier of the identifiers with the target as belonging to the target; and

an association retrieval engine configured to associate another identifier of the identifiers with the target as belonging to an associate of the target;

a processor; and

memory storing instructions when executed by the processor comprising functionalities for;

obtaining a first identifier of the target, wherein the first identifier comprises a first domain and a first handle;

identifying a first Internet site based on the first domain;

accessing, in the first Internet site and based on the first handle, a first public webpage associated with the target;

selecting a first parsing rule from a plurality of parsing rules that is associated with the first Internet site;

extracting a second identifier from the first public webpage using the first parsing rule;

determining, in response to extracting the second identifier, a similarity measure by comparing the second identifier to the first identifier using a pre-determined algorithm;

determining, in response to the similarity measure exceeding a pre-determined threshold and without based on any prior association between the first identifier and the second identifier, that the first identifier and the second identifier identify the same person as the user of the communication network based on a pre-determined criterion; and

collecting information of the target based on the second identifier for including in target data of the target.

20. The system of claim 19,

wherein the target data population engine further comprises:

a IP sniffer configured to extract contents of documents intercepted from private portions of communication network traffic based on the identifiers associated with the target for including in the target data of the target, wherein the contents of the documents are extracted using a plurality of parsing rules corresponding to the private portions of the communication network traffic;

wherein the instructions when executed by the processor further comprises functionalities for;

obtaining a third identifier of the target, wherein the third identifier comprises a third domain and a third handle;

intercepting, based on the third handle, a document associated with the target from a private portion of the communication network traffic identified based on the third domain; and

extracting content of the document for including in the target data of the target, wherein the content of the document is extracted using a third parsing rule, of the plurality of parsing rules, associated with the private portion of the communication network traffic identified based on the third domain.

21. The system of claim 19,

wherein the second identifier is determined from at least one selected from a group consisting of email address, webpage login name, chat user name, social networking user name, telephone number, and voice over IP (VoIP) phone number,

wherein the second domain is determined explicitly when determined from the email address, and

wherein the second domain is determined implicitly when determined from at least one selected from a group consisting of webpage login name, chat user name, social networking user name, telephone number, and VoIP phone number.

22. The system of claim 19, wherein the instructions when executed by the processor further comprises functionalities for:

adding the second identifier to the list of identifiers belonging to the target,

wherein the second identifier comprises a second domain and a second handle.

23. The system of claim 22, wherein the pre-determined algorithm comprises a Longest Common Subsequence Windowed (LCSW) signature algorithm.

24. The system of claim 22, wherein the instructions when executed by the processor further comprises functionalities for:

generating a string by concatenating a plurality of attributes of the target to the first identifier,

wherein the plurality of attributes of the target are stored in a target profile of the target,

wherein similarity measure is further determined by comparing the second identifier to the string using the LCS^W signature algorithm, and

wherein the plurality of attributes of the target are determined based in part by searching the target data of the target using the plurality of parsing rules.

25. The system of claim 23, wherein the instructions when executed by the processor further comprises functionalities for:

generating an alpha-text string and a numerical string from the string,

wherein the similarity measure is further determined by at least one selected from a group consisting of comparing a alpha-text portion of the second identifier to the alpha-text string using the LCS^W signature algorithm and comparing a numerical portion of the second identifier to the numerical string using the LCS^W signature algorithm.

26. The system of claim 22,

wherein the second identifier is found in a user profile portion of the first public webpage, and

wherein the user profile portion of the first public webpage is identified based on the first parsing rule associated with the first Internet site.

27. The system of claim 22, wherein collecting information of the target based on the second identifier comprises:

accessing, based on the second handle, a second public webpage in a second Internet site identified based on the second domain; and

extracting content of the second public webpage for including in the target data of the target, wherein the content of the second public webpage is extracted using a second parsing rule, of the plurality of parsing rules, associated with the second Internet site.

28. The system of claim 22, wherein collecting information of the target based on the second identifier comprises:

intercepting, based on the second handle, a document associated with the target from a private portion of the communication network traffic identified based on the second domain; and

extracting content of the document for including in the target data of the target, wherein the content of the document is extracted using a second parsing rule, of the plurality of parsing rules, associated with the private portion of the communication network traffic identified based on the second domain.

29. The system of claim 28, wherein the document comprises at least one selected from a group consisting of an email, a telephone recording, and a VoIP recording.

30. The system of claim 22, wherein the instructions when executed by the processor further comprises functionalities for:

storing the second identifier in the target profile of the target;

detecting the second identifier in another target profile of another target comprising another user of the communication network by searching a plurality of target profiles of a plurality of targets comprising a plurality of users of the communication network;

merging the target profile of the target with the another target profile; and

;

merging the target data of the target with target data of the another target.

31. The system of claim 30, wherein the instructions when executed by the processor further comprises functionalities for:

determining a biometric identifier of the target by analyzing the target data of the target using at least one selected from a group consisting of speaker recognition algorithm, authorship recognition algorithm, and facial recognition algorithm; and

determining an identity of the target based on the biometric identifier.

32. The system of claim 19,

wherein the target profile further comprises an associate list of the target;

wherein the instructions when executed by the processor further comprises functionalities for;

determining, using the CPU, the second identifier by further searching target data of a plurality of associates in the associate list of the target using the plurality of parsing rules.

33. The system of claim 19,

wherein the target profile further comprises an associate list of the target;

wherein the instructions when executed by the processor further comprises functionalities for;

determining a third identifier by searching the first public webpage using the first parsing rule, wherein the third identifier comprises a third domain and a third handle;

determining, using the association retrieval engine, the third identifier as belonging to an associate of the target, wherein the associate comprises another user of the communication network, wherein the third identifier is determined as belonging to the associate by applying a plurality of association rules to the target data of the target with respect to the another user;

adding the another user to the list of associates of the target; and

adding the third identifier to the list of identifiers belonging to associates of the target.

34. The system of claim 33, further comprising:

determining at least one selected from a group consisting of relationships, topics, and localities by analyzing the target data of the target,

wherein the plurality of association rules comprise at least one selected from a group consisting of a relational association rule, a topical association rule, and a spatial association rule.