Spam detection for user-generated multimedia items based on keyword stuffing

• US 8,752,184 B1
• Filed: 01/17/2008
• Issued: 06/10/2014
• Est. Priority Date: 01/17/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kacholia, Varun, Garg, Ashutosh, Stoutamire, David
• Primary Examiner(s): Brown, Christopher
• Assistant Examiner(s): OLION, BRIAN L
• Application Number:
  US12/015,986
• Time in Patent Office: 2,336 Days
• Field of Search: 726/26, 1/1, 715/208
• US Class Current: 726/26
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A system, a method, and various software tools enable a video hosting website to automatically identify posted video items that contain spam in the metadata associated with a respective video item. A spam detection tool for user-generated video items based on keyword stuffing is provided that facilitates the detection of spam in the metadata associated with a video item.

Claims

1. A computer-implemented method for spam detection in a collection of multimedia items, comprising:

storing the collection of multimedia items in a memory of a computer system, each multimedia item including a description of the item, the description including a plurality of tokens; and

for at least one multimedia item;

selecting a plurality of portions of the description of the item and for each selected portion counting a total number of unique tokens in the selected portion, wherein unique tokens appearing more than once in the selected portion are counted only once;

determining a distribution of unique tokens for the multimedia item using the total number of unique tokens in each selected portion of the multimedia item; and

responsive to the distribution of unique tokens exceeding a distribution threshold, marking the multimedia item for a future spam filtering action.

2. The computer implemented method of claim 1, further comprising:

indexing the multimedia items remaining in the collection in the index.

3. The computer-implemented method of claim 1, wherein the future spam filtering action is removing the multimedia item from the collection in memory.

4. The computer-implemented method of claim 1, wherein the future spam filtering action is demoting a rank of the multimedia item.

5. The computer-implemented method of claim 1, wherein the distribution is determined using a total number of unique tokens in all portions of the plurality of portions.

6. The computer-implemented method of claim 1, wherein the distribution is determined using an average number of unique tokens in all portions of the plurality of portions.

7. The computer-implemented method of claim 1, wherein the distribution is determined using a minimum number of unique tokens in any portion of the plurality of portions.

8. The computer-implemented method of claim 1, wherein the distribution is determined using a percentage of unique tokens in all portions of the plurality of portions.

9. The computer-implemented method of claim 1, wherein the distribution is determined using a maximum of percentage of unique tokens in any portion of the plurality of portions.

10. The computer-implemented method of claim 1, wherein the distribution is determined using a ratio of number of unique tokens in any portion of the plurality of portions to expected number of unique tokens in the portion.

11. The computer-implemented method of claim 1, wherein the distribution is determined using a ratio of number of instances of a most frequently occurring token to total number of tokens in any portion of the plurality of portions.

13. The computer-implemented method of claim 1, wherein the multimedia items are videos.

12. A non-transitory computer readable storage medium containing program code for spam detection in a collection of multimedia items, the program code comprising code for:

storing the collection of multimedia items, each multimedia item including a description of the item, the description including a plurality of tokens; and

for at least one multimedia item;

selecting a plurality of portions of the description of the item and for each selected portion counting a total number of unique tokens in the selected portion, wherein unique tokens appearing more than once in the selected portion are counted only once;

determining a distribution of unique tokens for the multimedia item using the total number of unique tokens in each selected portion of the multimedia item; and

responsive to the distribution of unique tokens exceeding a distribution threshold, marking the multimedia item for a future spam filtering action.

14. The computer readable storage medium of claim 12, wherein the multimedia items are videos.