Internet profile service

• US 9,292,612 B2
• Filed: 04/22/2009
• Issued: 03/22/2016
• Est. Priority Date: 04/22/2009
• Status: Active Grant

• Current Assignee: VeriSign, Inc.
• Original Assignee: VeriSign, Inc.
• Inventors: Orentas, Leonard Linas, Kosters, Mark Alan, Moston, Dave, Shyamsunder, Karthik, Sunkara, Srinivas, Jimenez, Eduardo, Pattapu, Phani, Holmes, Alex, Simpson, Andrew
• Primary Examiner(s): Joshi, Suraj
• Assistant Examiner(s): Mesa, Joel
• Application Number:
  US12/428,208
• Publication Number:
  US 20100274836A1
• Time in Patent Office: 2,526 Days
• Field of Search: 709/224, 709/223
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 43/08 Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
• CPC Class Codes: H04L 61/4511 using domain name system [DNS]
• CPC Class Codes: H04L 61/5007 Internet protocol [IP] addresses
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/30 Profiles

Abstract

Systems and methods for creating and using a domain profile include identifying a status of a first page associated with a domain. The first page is retrieved and additional pages from the domain are identified based on hyperlinks from the first page. The status of the additional pages is identified and the hyperlinks are prioritized based on the status and/or a comparison with predetermined data. Content is extracted from the first page and selected pages from among the additional pages. The specific additional pages may be selected based on the prioritization. The retrieved content may be processed through a signature marker set to determine a contextual match. A purpose of the domain is determined according to the status of the first page, the status of the additional pages and results of the processing of the content. The domain profile can be displayed, stored, sent and/or searched to identify web sites or attributes of interest.

Claims

1. A computer-implemented method for generating a domain profile, comprising:

determining, using a processor, a purpose associated with a first domain by accessing a first web page associated with the first domain, wherein the determining comprises;

determining whether content is obtainable by resolving the first domain and accessing the first web page;

in response to determining that content is obtainable, determining whether the first web page results in a redirection;

in response to determining that the first web page does not result in a redirection, determining whether retrieved content of the first web page is content from another web page;

comparing the retrieved content with predetermined data to determine contextual matches;

in response to determining that the retrieved content is not content from another web page, determining, based on the contextual matches, whether the purpose associated with the first domain is to receive advertising traffic; and

in response to determining that the purpose of the first domain is not to receive advertising traffic, determining, based on the contextual matches, whether the purpose of the first domain is commercial;

determining, based on the contextual matches, a category associated with the first domain, wherein the category associated with the first domain is distinct from the purpose associated with the first domain and defines a commercial sector of the first domain; and

storing, in a memory, the purpose associated with the first domain and the category associated with the first domain.

2. The computer-implemented method of claim 1, further including:

retrieving, from the first web page, at least one hyperlink corresponding to one or more subsequent web pages;

determining a purpose of the one or more subsequent web pages;

extracting content from the first web page and content from the one or more subsequent web pages;

identifying contextual matches that exist in the content that is extracted from the first web page and the content that is extracted from the one or more subsequent web pages based on a table of predetermined associated data elements;

determining a second purpose of the first domain based on the purpose of the first web page, the purpose of the one or more subsequent web pages, and the contextual matches that are identified;

determining, based on the contextual matches that are identified, a second category associated with the first domain; and

storing, in the memory, the second purpose that is determined and the second category associated with the first domain, wherein the category is different than the second category.

3. The computer-implemented method of claim 2, further including:

comparing the at least one hyperlink retrieved from the first web page with predetermined data;

prioritizing the at least one hyperlink based on the comparing; and

selecting a web page of the one or more subsequent web pages based on the prioritization of the at least one hyperlink.

4. The computer-implemented method of claim 1, further including:

identifying one or more sub-domains based on the purpose of the first domain;

determining a respective purpose associated with the one or more sub-domains;

determining a category associated with the one or more sub-domains; and

storing, in the memory, the respective purpose associated with the one or more sub-domains and the category associated with the one or more sub-domains.

5. The computer-implemented method of claim 4, wherein the determining the category associated with the one or more sub-domains comprises:

identifying one or more sub-domain web pages associated with the one or more sub-domains;

determining a respective purpose corresponding to each of the one or more sub-domain web pages; and

determining the category associated with the one or more sub-domains based on the respective purpose corresponding to each of the one or more sub-domain web pages.

6. The computer-implemented method of claim 1, further including:

receiving at least one of a purpose and a category from a user;

adding the first domain to a list of identified domains based on a determination that the at least one of a purpose and a category from the user corresponds to at least one of the purpose and the category associated with the first domain; and

outputting the list of identified domains to the user.

7. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for generating a domain profile, comprising:

determining a purpose associated with a first domain by accessing a first web page associated with the first domain, wherein the determining comprises;

determining whether content is obtainable by resolving the first domain and accessing the first web page;

in response to determining that content is obtainable, determining whether the first web page results in a redirection;

in response to determining that the first web page does not result in a redirection, determining whether retrieved content of the first web page is content from another web page;

comparing the retrieved content with predetermined data to determine contextual matches;

in response to determining that the retrieved content is not content from another web page, determining, based on the contextual matches, whether the purpose associated with the first domain is to receive advertising traffic; and

in response to determining that the purpose of the first domain is not to receive advertising traffic, determining, based on the contextual matches, whether the purpose of the first domain is commercial;

determining, based on the contextual matches, a category associated with the first domain, wherein the category associated with the first domain is distinct from the purpose associated with the first domain and defines a commercial sector of the first domain; and

storing, in a memory, the purpose associated with the first domain and the category associated with the first domain.

8. The non-transitory computer-readable storage medium of claim 7, the method further including:

retrieving, from the first web page, at least one hyperlink corresponding to one or more subsequent web pages;

determining a purpose of the one or more subsequent web pages;

extracting content from the first web page and content from the one or more subsequent web pages;

identifying contextual matches that exist in the content that is extracted from the first web page and the content that is extracted from the one or more subsequent web pages based on a table of predetermined associated data elements;

determining a second purpose of the first domain based on the purpose of the first web page, the purpose of the one or more subsequent web pages, and the contextual matches that are identified;

determining, based on the contextual matches that are identified, a second category associated with the first domain; and

storing, in the memory, the second purpose that is determined and the second category associated with the first domain, wherein the category is different than the second category.

9. The non-transitory computer-readable storage medium of claim 8, the method further including:

comparing the at least one hyperlink retrieved from the first web page with predetermined data;

prioritizing the at least one hyperlink based on the comparing; and

selecting a web page of the one or more subsequent web pages based on the prioritization of the at least one hyperlink.

10. The non-transitory computer-readable storage medium of claim 7, the method further including:

identifying one or more sub-domains based on the purpose of the first domain;

determining a respective purpose associated with the one or more sub-domains;

determining a category associated with the one or more sub-domains; and

storing, in the memory, the respective purpose associated with the one or more sub-domains and the category associated with the one or more sub-domains.

11. The non-transitory computer-readable storage medium of claim 10, wherein the determining the category associated with the one or more sub-domains comprises:

identifying one or more subsequent web pages associated with the one or more sub-domains;

determining a respective purpose corresponding to each of the one or more subsequent web pages; and

determining the category associated with the one or more sub-domains based on the respective purpose corresponding to each of the one or more subsequent web pages.

12. The non-transitory computer-readable storage medium of claim 7, the method further including:

receiving at least one of a purpose and a category from a user;

adding the first domain to a list of identified domains based on a determination that the at least one of a purpose and a category from the user corresponds to at least one of the purpose and the category associated with the first domain; and

outputting the list of identified domains to the user.

13. An apparatus comprising:

a memory storing data and instructions; and

a processor configured to access the memory and execute the instructions, causing the processor to;

determine a purpose associated with a first domain by accessing a first web page associated with the first domain, wherein the determining comprises;

determining whether content is obtainable by resolving the first domain and accessing the first web page;

in response to determining that content is obtainable, determining whether the first web page results in a redirection;

in response to determining that the first web page does not result in a redirection, determining whether retrieved content of the first web page is content from another web page;

comparing the retrieved content with predetermined data to determine contextual matches;

in response to determining that the retrieved content is not content from another web page, determining, based on the contextual matches, whether the purpose associated with the first domain is to receive advertising traffic; and

in response to determining that the purpose of the first domain is not to receive advertising traffic, determining, based on the contextual matches, whether the purpose of the first domain is commercial;

determine, based on the contextual matches, a category associated with the first domain, wherein the category associated with the first domain is distinct from the purpose associated with the first domain and defines a commercial sector of the first domain; and

store, in the memory, the purpose associated with the first domain and the category associated with the first domain.

14. The apparatus of claim 13, wherein the processor is further configured to:

retrieve, from the first web page, at least one hyperlink corresponding to one or more subsequent web pages;

determine a purpose of the one or more subsequent web pages;

extract content from the first web page and content from the one or more subsequent web pages;

identify contextual matches that exist in the content that is extracted from the first web page and the content that is extracted from the one or more subsequent web pages based on a table of predetermined associated data elements;

determine a second purpose of the first domain based on the purpose of the first web page, the purpose of the one or more subsequent web pages, and the contextual matches that are identified;

determine, based on the contextual matches that are identified, a second category associated with the first domain; and

store, in the memory, the second purpose that is determined and the second category associated with the first domain, wherein the category is different than the second category.

15. The apparatus of claim 14, wherein the processor is further configured to:

compare the at least one hyperlink retrieved from the first web page with predetermined data;

prioritize the at least one hyperlink based on the comparing; and

select a web page of the one or more subsequent web pages based on the prioritization of the at least one hyperlink.

16. The apparatus of claim 13, wherein the processor is further configured to:

identify one or more sub-domains based on the purpose of the first domain;

determine a respective purpose associated with the one or more sub-domains;

determine a category associated with the one or more sub-domains; and

store, in the memory, the respective purpose associated with the one or more sub-domains and the category associated with the one or more sub-domains.

17. The apparatus of claim 16, wherein when the processor is configured to determine the category associated with the one or more sub-domains, the processor is further configured to:

identify one or more sub-domain web pages associated with the one or more sub-domains;

determine a respective purpose corresponding to each of the one or more sub-domain web pages; and

determine the category associated with the one or more sub-domains based on the respective purpose corresponding to each of the one or more sub-domain web pages.

18. The apparatus of claim 13, wherein the processor is further configured to:

receive at least one of a purpose and a category from a user;

add the first domain to a list of identified domains based on a determination that the at least one of a purpose and a category from the user corresponds to at least one of the purpose and the category associated with the first domain; and

output the list of identified domains to the user.

19. A computer-implemented method for generating a domain profile, comprising:

determining, using a processor, a purpose associated with a first domain, wherein the determining comprises;

determining whether content is obtainable by resolving the first domain and accessing the first web page;

in response to determining that content is obtainable, determining whether the first web page results in a redirection;

in response to determining that the first web page does not result in a redirection, determining whether retrieved content of the first web page is content from another web page, wherein the retrieved content comprises at least one hyperlink corresponding to one or more subsequent web page;

comparing the retrieved content with predetermined data to determine first contextual matches;

determining a purpose of the one or more subsequent web pages, wherein the determining comprises determining whether content is obtainable by accessing the one or more subsequent web page;

in response to determining that content is obtainable from the one or more subsequent web page, determining whether the one or more subsequent web page results in a redirection;

in response to determining that the one or more subsequent web page does not result in a redirection, determining whether retrieved content of the one or more subsequent web page is content from another web page;

comparing the retrieved content of the one or more subsequent web page with predetermined data to determine second contextual matches;

in response to determining that the retrieved content of the first web page and the retrieved content of the one or more subsequent web page are not content from another web page, determining, based on the first contextual matches and the second contextual matches, whether the purpose associated with the first domain is to receive advertising traffic; and

in response to determining that the purpose of the first domain is not to receive advertising traffic, determining, based on the first contextual matches and the second contextual matches, whether the purpose of the first domain is commercial;

determining, based on the first contextual matches and the second contextual matches, a category associated with the first domain, wherein the category associated with the first domain is distinct from the purpose associated with the first domain and defines a commercial sector of the first domain; and

storing, in a memory, the purpose of the first domain that is determined.

20. The computer-implemented method of claim 19, further including:

identifying one or more sub-domains based on the purpose of the first domain;

determining a purpose associated with each of the one or more sub-domains based on the one or more subsequent web page that are respectively associated with each of the one or more sub-domains;

determining, based on the purpose associated with the each of the one or more sub-domains, a category associated with the each of the one or more sub-domains; and

storing, in the memory, the respective purpose associated with the each of the one or more sub-domains and the category associated with the each of the one or more sub-domains.

21. The computer-implemented method of claim 19, further comprising:

receiving at least one of a purpose and a category from a user;

adding the first domain to a list of identified domains based on a determination that the at least one of a purpose and a category from the user corresponds to at least one of the purpose and the category associated with the first domain; and

outputting the list of identified domains to the user.