URL-based content categorization

• US 8,078,625 B1
• Filed: 09/11/2006
• Issued: 12/13/2011
• Est. Priority Date: 09/11/2006
• Status: Expired due to Fees

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: AOL Inc. (Apollo Global Management, Inc.)
• Inventors: Zhang, Jianping, Qin, Jinshui, Yan, Qiuming
• Primary Examiner(s): Vo, Tim T
• Assistant Examiner(s): TRAN, ANHTAI V
• Application Number:
  US11/530,752
• Time in Patent Office: 1,919 Days
• Field of Search: None
• US Class Current: 707/748
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: Y10S 707/952 Malicious software

Abstract

Content may be categorized by accessing a URL associated with the content, determining a set of n-grams contained in the URL, and determining a category of the content based on the set of n-grams.

Claims

1. A computer implemented method of categorizing content, the method comprising:

accessing a uniform resource locator associated with content requested by a user;

processing the uniform resource locator to generate a set of multiple n-grams contained in the uniform resource locator, wherein the processing includes using a plurality of different n values to generate the set of multiple n-grams, the set including n-grams of different character lengths;

providing the set of multiple n-grams to a classifier for determination of a classification score based on the set of multiple n-grams, wherein each of the n-grams is weighted according to the formula;

2. The method of claim 1 wherein processing the uniform resource locator to generate a set of n-grams includes replacing an “

@”

character with an ‘

a’

character.

3. A computer implemented method of categorizing content, the method comprising:

accessing, from computer storage, a uniform resource locator associated with content;

determining a set of n-grams contained in the uniform resource locator, wherein a plurality of different n values are used to determine the set of n-grams, the set of n-grams including n-grams of different character lengths;

determining a category of the content based on the set of n-grams, wherein each of the n-grams is weighted according to the formula;

4. The method of claim 3 further comprising processing the uniform resource locator to remove characters unrepresentative of content.

5. The method of claim 3 wherein processing the uniform resource locator includes removing a non-alphabetical character and replacing the character with one or more alphabetical characters.

6. The method of claim 3 wherein the plurality of different n values includes a range of n'"'"'s greater than or equal to three and less than or equal to ten.

7. The method of claim 3 wherein determining a category of the content comprises:

using a classifier to determine a score based on the set of n-grams; and

comparing the score to a threshold.

8. The method of claim 3 wherein determining a category of the content comprises determining a category for the content based on the set of n-grams and an analysis of the content.

9. The method of claim 3 wherein determining a category of the content comprises:

using a first classifier to determine a first score based on the set of n-grams;

using a second classifier to determine a first score based on the set of n-grams;

using a second classifier to determine a second score based on the content, wherein the first score is separate from the second score; and

determining a category of the content based on both the first score and the second score.

10. The method of claim 3 wherein the content is included in a web page.

11. The method of claim 4 wherein processing the uniform resource locator includes removing a non-alphabetical character.

12. The method of claim 9 wherein determining a category of the content based on the first score and the second score comprises generating a sum of the first score and the second score and comparing the sum to a threshold.

13. The method of claim 9 wherein determining a category of the content based on the first score and the second score comprises determining if either of the first score or the second score is above a threshold.

14. The method of claim 9 wherein determining a category of the content based on the first score and the second score comprises combining the first score with the second score to generate a combined classification score such that either the first score or the second score contributes more to the combined classification score than the other, and comparing the combined classification score to a threshold.

15. A system for categorizing content, the system comprising:

at least one processing device;

a storage storing instructions for causing the at least one processing device to implement;

an n-gram generator configured to access a uniform resource locator (URL) associated with content and determine a set of n-grams contained in the uniform resource locator;

a classifier configured to determine a URL score based on the set of n-grams, wherein each of the n-grams is weighted according to the formula;

16. The system of claim 15 wherein, to categorize the content based on the URL score, the categorizer is configured to compare the URL score to a threshold.

17. The system of claim 15 wherein, to determine the set of n-grams contained in the uniform resource locator, the n-gram generator is configured to determine a set of n-grams that includes n-grams for a range of n'"'"'s.

18. The system of claim 15 further comprising:

a content-based classifier configured to determine a content score based on the content; and

wherein the categorizer is configured to categorize the content based on the URL score and the content score.

19. The system of claim 15 wherein the content is included in a web page.

20. The system of claim 18 wherein the categorizer is further configured to:

categorize the content based on the URL score and the content score by combining the URL score with the content score to generate a combined classification score, wherein either the URL score or the content score contributes more to the combined classification score than the other; and

compare the combined classification score to a threshold.

21. A system for categorizing content, the system comprising:

at least one processing device;

a storage storing instructions for causing the at least one processing device to implement;

a URL-based classifier configured to produce a first score based on a set of n-grams contained in a uniform resource locator associated with content, wherein the set of n-grams is based on multiple different n-values and includes n-grams of different character lengths, wherein each of the n-grams is weighted according to the formula;

23. The system of claim 21 wherein, to determine a category of the content based on both the first score and the second score, the integration engine is configured to determine if the first score exceeds a threshold and, if the first score does not exceed the threshold, to determine if the second score exceeds the threshold.

24. The system of claim 21 where the integration engine is further configured to:

determine a category of the content based on both the first score and the second score by combing the first score with the second score to generate a combined classification score, wherein either the first score or the second score contributes more to the combined classification score than the other; and

compare the combined classification score to a threshold.

22. The system of 21 wherein, to determine a category of the content based on both the first score and the second score, the integration engine is configured to generate a sum of the first score and the second score and compare the sum to a threshold.

25. The system of claim 22 wherein, to generate the sum of the first score and the second score, the integration engine is configured to generate a weighted linear sum of the first score and the second score.

26. The system of claim 22 wherein, to generate the sum of the first score and the second score, the integration engine is configured to generate a probabilistic sum of the first score and the second score.

27. A computer implemented method of categorizing content, the method comprising:

accessing a uniform resource locator (URL) associated with the content;

processing the uniform resource locator to generate a set of multiple n-grams contained in the uniform resource locator, wherein a plurality of different n values are used to process the uniform resource locator, and the set of multiple n-grams includes n-grams of different character lengths;

providing the set of multiple n-grams to a URL classifier, wherein the URL classifier is configured to determine a first classification score based on the set of multiple n-grams, wherein each of the n-grams is weighted according to the formula;

28. The method of claim 27 wherein combining the first classification score with the second classification score to generate a combined classification score includes generating a sum of the first score and the second score.

29. The method of claim 27 wherein combining the first classification score with the second classification score to generate a combined classification score includes combining the first classification score with the second classification score such that either the first classification score or the second classification score contributes more to the combined classification score than the other.

30. The method of claim 28 wherein generating a sum of the first score and the second score includes generating a weighted linear sum of the first score and the second score.

31. The method of claim 28 wherein generating a sum of the first score and the second score included generating a probabilistic sum of the first score and second score.