System, method, and computer program product for processing a prefix tree file utilizing a selected agent
First Claim
Patent Images
1. A method, comprising:
- identifying a first file including a prefix tree, wherein the prefix tree includes an ordered hierarchical data structure which stores arrays of data with a common prefix, wherein the prefix tree stores at least a portion of uniform resource locators, each of the uniform resource locators having a common prefix;
selecting an agent to process the first file; and
processing the first file utilizing the agent, wherein the agent is selected based on a type of the prefix tree included in the first file, and wherein the type of the prefix tree is indicated by a file extension of the first file in which the prefix tree is included;
wherein selecting the agent includes identifying that the prefix tree is a crawl trie based upon the file extension and selecting an agent configured to perform a crawling function based upon identifying that the prefix tree is a crawl trie; and
wherein processing the first file utilizing the agent includes downloading a second file identified using a uniform resource locator stored in the prefix tree and analyzing the second file to identify unwanted content in the second file.
9 Assignments
0 Petitions
Accused Products
Abstract
A system, method, and computer program product are provided for processing a prefix tree file utilizing a selected agent. In use, a file including a prefix tree is identified. Additionally, an agent is selected from a plurality of agents to process the file. Further, the file is processed utilizing the agent.
29 Citations
20 Claims
-
1. A method, comprising:
-
identifying a first file including a prefix tree, wherein the prefix tree includes an ordered hierarchical data structure which stores arrays of data with a common prefix, wherein the prefix tree stores at least a portion of uniform resource locators, each of the uniform resource locators having a common prefix; selecting an agent to process the first file; and processing the first file utilizing the agent, wherein the agent is selected based on a type of the prefix tree included in the first file, and wherein the type of the prefix tree is indicated by a file extension of the first file in which the prefix tree is included; wherein selecting the agent includes identifying that the prefix tree is a crawl trie based upon the file extension and selecting an agent configured to perform a crawling function based upon identifying that the prefix tree is a crawl trie; and wherein processing the first file utilizing the agent includes downloading a second file identified using a uniform resource locator stored in the prefix tree and analyzing the second file to identify unwanted content in the second file. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A method, comprising:
-
identifying a first file including a prefix tree, wherein the prefix tree includes an ordered hierarchical data structure which stores arrays of data with a common prefix, wherein the prefix tree stores at least a portion of uniform resource locators, each of the uniform resource locators having a common prefix; selecting an agent to process the first file based on a type of the prefix tree included in the file, and wherein the type of the prefix tree is indicated by a file extension of the file in which the prefix tree is included, and wherein selecting the agent includes identifying that the prefix tree is a crawl trie based upon the file extension and selecting an agent configured to perform a crawling function based upon identifying that the prefix tree is a crawl trie; and processing the first file utilizing the agent, wherein identifying the file includes selecting the file from a queue of files, and wherein processing the file utilizing the agent includes downloading a second file identified using a uniform resource locator stored in the prefix tree and analyzing the second file for gathering information associated with the uniform resource locator.
-
-
16. A non-transitory computer readable storage medium for storing instructions for performing operations comprising:
-
identifying a first file including a prefix tree, wherein the prefix tree includes an ordered hierarchical data structure which stores arrays of data with a common prefix, wherein the prefix tree stores at least a portion of uniform resource locators, each of the uniform resource locators having a common prefix; selecting an agent to process the first file based on a type of the prefix tree included in the file, and wherein the type of the prefix tree is indicated by a file extension of the file in which the prefix tree is included, and wherein selecting the agent includes identifying that the prefix tree is a crawl trie based upon the file extension and selecting an agent configured to perform a crawling function based upon identifying that the prefix tree is a crawl trie; and processing the first file utilizing the agent, wherein processing the file utilizing the agent includes downloading a second file identified using a uniform resource locator stored in the prefix tree and analyzing the second file for gathering information associated with the uniform resource locator. - View Dependent Claims (17, 18, 19, 20)
-
Specification