Software web crowler and method therefor

• US 7,693,872 B2
• Filed: 08/16/2007
• Issued: 04/06/2010
• Est. Priority Date: 08/17/2006
• Status: Active Grant

• Current Assignee: Intersections Incorporated (Aura Sub, LLC)
• Original Assignee: Anchorfree Incorporated
• Inventors: Gavrilov, Dimitry, Malobrodsky, Eugene, Gorodyansky, David
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): MUELLER, KURT A
• Application Number:
  US11/839,789
• Publication Number:
  US 20080046240A1
• Time in Patent Office: 964 Days
• Field of Search: 707/1
• US Class Current: 1/1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

System for crawling the web for multimedia files and indexing the files based on sound analysis and algorithmic translation.

Claims

1. A computerized system comprising:

a crawler obtaining multimedia files from a network, the crawler comprising;

a multi-threaded downloader that downloads web pages;

a queue storing links corresponding to the download web pages;

a scheduler obtaining the stored links from the queue and passing the obtained links to the multi-threaded downloader, wherein the multi-threaded downloader downloads multiple multimedia files concurrently from said links;

a multimedia processor receiving said multimedia files from the crawler and processing said multimedia files by translating speech in the multimedia files into a textual representation,

wherein said multimedia processor determines sound effects in said multimedia files by comparing said sound effects in said multimedia files against a predetermined set of sounds, wherein generated metadata is determined by the comparison, and wherein said metadata comprises keywords identifying a type of said sound effects;

a data mining module that extracts text information from the textual representation; and

an indexer that indexes the multimedia files based on said keywords and said text information.

2. The computerized system of claim 1, wherein the multimedia processor is further operable to detect a type of music in the multimedia files.

3. The computerized system of claim 1, wherein the data mining algorithm:

a. detects frequency of keywords in the generated metadata;

b. adds weights to the keywords, and

c. compiles the keywords for each multimedia file.

4. The computerized system of claim 1, wherein the multimedia files are audio files.

5. The computerized system of claim 1, wherein the multimedia files are video files.

9. The computerized system of claim 1, wherein the multimedia files are audio files.

10. The computerized system of claim 1, wherein the multimedia files are video files.

6. A computer implemented method comprising:

obtaining multimedia files from a network by utilizing a multi-threaded downloader that download multiple multimedia files concurrently from links acquired from downloaded web pages;

processing, using a processor, the obtained multimedia files by translating speech in the multimedia files into a textual representation,

wherein said processing further determines sound effects in said multimedia files by comparing said sound effects in said multimedia files against a predetermined set of sounds, wherein generated metadata is determined by the comparison, and wherein said metadata comprises keywords identifying a type of said sound effects;

extracting text information from the textual representation, and;

indexing the multimedia files based on said keywords and said text information.

7. The method of claim 6, wherein processing further comprises detecting a type of music in the multimedia files.

8. The method of claim 6, wherein extracting further comprises:

a. detecting frequency of keywords in the generated metadata;

b. adding weights to the keywords; and

c. compiling the keywords for each multimedia file.