Search ranger system and double-funnel model for search spam analyses and browser protection

• US 9,430,577 B2
• Filed: 05/31/2007
• Issued: 08/30/2016
• Est. Priority Date: 05/31/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Wang, Yi-Min, Ma, Ming
• Primary Examiner(s): KUDDUS, DANIEL A
• Application Number:
  US11/756,600
• Publication Number:
  US 20080301139A1
• Time in Patent Office: 3,379 Days
• Field of Search: 709/224
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9566 URL specific, e.g. using aliases, detecting broken or misspelled links
• CPC Class Codes: H04L 63/1441 Countermeasures against malicious traffic countermeasures against attacks on cryptographic mechanisms H04L9/002
• CPC Class Codes: H04L 63/1483 service impersonation, e.g. phishing, pharming or web spoofing detection of rogue wireless access points H04W12/12

Abstract

An exemplary system for monitoring search spam and protecting against search spam includes a self-monitoring subsystem to uncover spam patterns and a self-protection subsystem to protect against spam by providing spam-related information to strengthen a relevance ranking algorithm. An exemplary architecture for monitoring search spam includes a first component to receive one or more spammer targeted keywords and to search, scan and analyze URLs based at least in part on the one or more spammer targeted keywords, a second component to receive one or more URLs from the first component and to verify one or more of these URLs as a spam URL and a third component to collect spammer targeted keywords associated with one or more spam URLs and to provide one or more of the spammer targeted keywords to the first component.

Claims

1. One or more computing devices comprising:

one or more processors; and

one or more computer readable storage media storing instructions that, when executed by the one or more processors, configure the one or more computing devices to;

identify one or more spammer targeted keywords relating to keywords used in search queries submitted to a search engine by one or more users, the identifying being based in part on a popularity of the one or more spammer targeted keywords amongst advertisers, wherein the popularity of the one or more spammer targeted keywords amongst the advertisers is based in part on a number of bids provided by the advertisers, and wherein the one or more spammer targeted keywords is associated with a syndication business, the syndication business including at least a publisher, an advertiser, and a syndicator;

perform one or more queries for the one or more spammer targeted keywords to retrieve a set of uniform resource locators (URLs) that provide search results;

scan the set of URLs based in part on the one or more spammer targeted keywords by accessing at least one URL in the set of URLs;

determine that the at least one URL redirects to a known spammer domain, the at least one URL including an associated URL domain;

determine that the at least one URL of the set of URLs comprises a spam URL;

update a list of known spam domains to include the associated URL domain; and

determine that the associated URL domain is associated with a spam syndication program, the spam syndication program including at least a spam publisher associated with a doorway page for redirecting a browser to a redirection domain associated with the spam publisher.

2. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to analyze a ranking algorithm for ranking the search results based in part on the spam URL.

3. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to analyze a ranking algorithm for ranking the search results based in part on the one or more spammer targeted keywords.

4. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to:

identify a spam heavy domain associated with the spam URL; and

analyze a ranking algorithm for ranking the search results based in part on the spam heavy domain.

5. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to confirm spam URLs and perform a backward discovery of one or more forums supporting the spam URLs.

6. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to receive a list of one or more spammed forums and extract one or more spam URL suspects from a spammed forum.

7. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to patrol spam heavy domains or forums.

8. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to receive information associated with the number of bids provided by the advertisers from the syndicator.

9. The one or more computing devices of claim 1, wherein the one or more computing devices are further configured to remove the spam URL from a search result.

10. The one or more computing devices of claim 1, wherein the accessing the at least one URL comprises:

accessing the at least one URL without running one or more scripts associated with a website located at the at least one URL, the accessing to receive a first webpage;

accessing the at least one URL including running the one or more scripts associated with the website, the accessing to receive a second webpage;

determining a difference between the first webpage and the second webpage; and

determining that the at least one URL includes a click-through spammer URL.

11. A method comprising:

determining at least one spammer targeted keyword relating to a common keyword used in commerce search queries, the determining being based in part on a popularity of the at least one spammer targeted keyword amongst advertisers, wherein the popularity of the at least one spammer targeted keyword amongst the advertisers is based in part on a number of bids provided by the advertisers, and wherein the at least one spammer targeted keyword is associated with a syndication business, the syndication business including at least a publisher, an advertiser, and a syndicator;

inputting the at least one spammer targeted keyword to a search engine to generate search results including a plurality of uniform resource locators (URLs);

accessing, by one or more processors, one or more URLs of the plurality of URLs;

recording the one or more URLs, wherein the recording comprises redirection tracking that intercepts redirection traffic;

grouping the one or more recorded URLs using similarity-based grouping;

verifying that at least one of the one or more URLs comprises a spam URL; and

determining that the spam URL is associated with a spam syndication program, the spam syndication program including at least a spam publisher associated with a doorway page for redirecting a browser to a redirection domain associated with the spam publisher.

12. The method of claim 11, wherein the one or more recorded URLs comprise at least one redirection URL.

13. The method of claim 11, wherein the similarity-based grouping comprises grouping based in part on doorway-domain similarity.

14. The method of claim 11, wherein the grouping further comprises identifying a domain associated with one or more doorways.

15. The method of claim 11, wherein the grouping comprises grouping at least one of the one or more recorded URLs with a preexisting group.

16. The method of claim 11, wherein the verifying comprises at least one of:

determining that redirection occurred to a new domain with an IP address associated with a known spammer,

determining that link-query results include one or more forums associated with spammers,

determining that a page uses scripting on-off cloaking to achieve crawler-browser cloaking,

determining that a page uses click-through cloaking,

determining that a page is an ads portal that forwards ads click-through traffic to a known spam-traffic aggregator domain,

determining that a page is hosted on a known spam heavy domain,

determining that characteristics of an image matches characteristics of an image of a known spammer, and

determining that a script matches a script of a known spammer.

17. The method of claim 16, further comprising generating at least one of a list of domains ranked based on a number of unique confirmed spam URLs hosted, a list of domains ranked based on a number of spam appearances in search results, and a list of domains ranked based on spam percentages.

18. The method of claim 11, further comprising receiving information associated with the number of bids provided by the advertisers from the syndicator.

19. The method of claim 11, further comprising removing the spam URL from a search result.

20. The method of claim 11, wherein the accessing the one or more URLs comprises:

accessing the one or more URLs without running one or more scripts associated with a website located at the one or more URLs, the accessing to receive a first webpage;

accessing the one or more URLs including running the one or more scripts associated with the web site, the accessing to receive a second webpage;

determining a difference between the first webpage and the second webpage; and

determining that the one or more URLs includes a click-through spammer URL.

21. A method comprising:

identifying one or more spammer targeted keywords related to keywords used in search queries, the identifying being based in part on a popularity of the one or more spammer targeted keywords amongst advertisers, wherein the popularity of the one or more spammer targeted keywords amongst the advertisers is based in part on a number of bids provided by the advertisers, and wherein the one or more spammer targeted keywords is associated with a syndication business, the syndication business including at least a publisher, an advertiser, and a syndicator;

implementing a search based in part on the one or more spammer targeted keywords;

accessing, by one or more processors, individual uniform resource locators (URLs) of a set of URLs that comprise a search result to the search;

recording one or more redirection URLs associated with the set of URLs based in part on the accessing;

determining that the one or more redirection URLs redirect to a known spammer domain, the one or more redirection URLs including one or more associated URL domains;

classifying, based in part on the recording and the determining, one or more URLs of the set of URLs as one or more classified spam URLs;

updating a list of known spam domains to include the one or more associated URL domains; and

determining that the one or more associated URL domains are associated with a spam syndication program, the spam syndication program including at least a spam publisher associated with a doorway page for redirecting a browser to a redirection domain associated with the spam publisher.

22. The method of claim 21, further comprising verifying that the one or more classified spam URLs are one or more spam URLs.

23. The method of claim 22, further comprising removing the spam URLs from one or more search results.

24. The method of claim 21, further comprising receiving information associated with the number of bids provided by the advertisers from the syndicator.