System and method for dynamically identifying the best search engines and searchable databases for a query, and model of presentation of results - the search assistant

• US 20060288001A1
• Filed: 06/20/2006
• Published: 12/21/2006
• Est. Priority Date: 06/20/2005
• Status: Abandoned Application

• Current Assignee: Daniel Santos Murta De Oliveira, Rafael Rego Pinto Rodrigues Da Costa, Rodrigo Barreto Dos Santos
• Original Assignee: Daniel Santos Murta De Oliveira, Rafael Rego Pinto Rodrigues Da Costa, Rodrigo Barreto Dos Santos
• Inventors: dos Santos, Rodrigo Barreto, Oliveira, Daniel Santos Murta de, Costa, Rafael Rego Pinto Rodrigues da
• Application Number:
  US11/472,181
• Publication Number:
  US 20060288001A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The invention is directed to a system and method for dynamically identifying the best search engines and searchable databases for a given query comprising a model where given a query, the more relevant search engines and searchable databases will be retrieved and presented as response to the query.

Claims

1. A method for dynamically identifying search engines or searchable databases for a given query comprising:

performing a query treatment, said query treatment including at least one step selected from a group consisting of;

interpreting key-words defining strings,

performing stemming words,

setting correlated words,

understanding the association between words, and

setting capital or small letters when it is necessary;

checking a stored list to determine if there is already a result list stored related to said key-words,

and, if there is valid result list stored in said stored list, skipping an offline ranking

process and proceeding to an output ranking of search engines and searchable databases;

performing said offline ranking process, in which said search engines and searchable databases

are retrieved based on representative indexes without performing any online consult;

storing the data such that when the same key-words are searched again in the future, the results

can be immediately retrieved; and

output ranking of said search engines and searchable databases as a page of results.

2. The method of claim 1, wherein said offline ranking process further comprises an offline consult to the representative index of each search engine by:

identifying a plurality of search engines and searchable databases that brings reference to the searched key-words;

assigning offline scores to each search engine and database based on the analysis of the elements uploaded in the offline consult to the representative index; and

ranking the search engines and searchable databases according to their scores, being the highest scored engines placed in the top of the list.

3. The method of claim 2, wherein said elements are chosen from the group consisting of inverted files (index) common elements such as word hits or term frequencies, document frequencies, inverted document frequencies, collection frequencies, inverted collection frequencies, words positioning or additional techniques such as the use of thesaurus and complementary dictionaries in order to help determining which search engines and searchable databases are the most likely to address the query searched.

4. The method of claim 1, wherein said representative index comprises a summary of the content of each search engine and searchable database built by querying each one with sample queries and by building an inverted file based on a sample group of documents of each search engine and searchable database.

5. The method of claim 4, wherein said sample queries used in order to request the pages of results, extract documents, and build the representative index for each search engine and searchable database comprise words from a static repository.

6. The method of claim 4, wherein said sample queries used in order to request the pages of results, extract documents, and build the representative index for each search engine and searchable database comprise words from a dynamically generated repository.

7. The method of claim 4, wherein said representative index associated with said thesaurus and with a previous categorized and classified repository of words can generate automated classification of the content and enable to automatically categorize each Search Engine and Searchable Databases into areas of knowledge.

8. The system and method of claim 1, wherein said offline ranking process is combined with an online ranking process, in order to reach a more comprehensive output rank of search engines.

9. The method of claim 8, wherein an offline rank of search engines is used as a filtering list, being the “

n”

first engines selected to pass through online consult, so as to upload search parameters from the called interaction rules database, in order to format the query to every search engine, maintaining the search preferences, and parse each search engines'"'"' page of results to the query, and eventually their results, for important information, such as word hits, term frequency, string matches, hierarchical analysis, in order to assign called online scores, wherein online scores can also make use of additional relevance elements such as hyperlinks networking, user'"'"'s implicit feedback, and other statistics of use and once online scores are assigned, the search engines and searchable databases are ranked according to these scores, being the highest scored engines placed in the top of the list, wherein said list is an online rank of search engines, wherein combining the said offline scores with the online scores to determine a combined rank of search engines.

10. The method of claim 9, wherein said interaction rules database comprises the parameters needed in order to send and receive pages of results to every Search Engine and Searchable Databases.

11. The method of claims 9, wherein said output rank brings hyperlinks pointing to the URL of the page of results each search engine and searchable database has to the given query, wherein said URL is dynamically generated every time a user requests it, comprising:

a user clicking on a particular search engine or searchable database;

uploading search parameters from said interaction rules database;

formatting the URL of the page of results to the query; and

redirecting user to said URL of the page of results of the clicked search engine.

12. The method of claim 10, wherein said parameters are chosen from a group consisting of search methods (POST, GET), target URL (URL of the page of results), Boolean operators (All words/Any Words/Phrase Match/Near/Not), wrappers for results extraction and HTML hierarchic structure.

13. The method of claim 12 further comprising building an operating database to extract said interaction rules and to perform said representative index.

14. The method of claim 12, wherein said building said operating database comprises:

using a special crawler configured to be a search engine finder to find search engines and searchable databases to be set to operate in the said search assistant;

using the search engine finder to continuously seek the world wide web for search engines and searchable databases;

finding pages with search fields or pages where html tags such as <

form> and

<

input>

can be found, and where there are evidences that they are related to a search field, wherein such evidence can be an attribute named search, research, find, seek, fetch, or any other similar word, in english or in any other language;

forwarding said identified search engines and searchable databases to the called learning module having a series of operations to automatically discover and store the said interaction rules;

automatically sending a set of queries comprising possible queries and impossible queries to each search engine;

analyzing results in order to capture said search parameters; and

storing said representative index for each database by querying of a search engine with sample queries in order to capture results and index them, and thereafter performing a representation of its database.

15. The method of claim 9 wherein said output rank of search engines and searchable databases is based on the said online ranking process, comprising the use of the said online scores in order to perform the ranking of search engines and searchable databases.

16. The method of claim 9 wherein said output rank of search engines and searchable databases is replaced by an output rank of documents found inside those engines, said output rank of documents comprising the results each search engine and searchable database brings to the searched query.

17. The method of claim 16, wherein said output rank of documents comprises:

within said online ranking process, assigning scores to the results found inside the page of results of each search engine and searchable database, such scores being based on the content of each document and on the scores of the search engine or searchable database the document belongs to, wherein said relevance factor comprises a named document score; and

building a list merging said documents found inside all search engines and searchable databases by placing these documents ordered by said document scores.

18. The method of claim 1, wherein said output rank brings hyperlinks pointing to the URL of the page of results each search engine and searchable database has to the given query, wherein said URL is dynamically generated every time a user requests it, comprising:

a user clicking on a particular search engine or searchable database;

uploading search parameters from said interaction rules database;

formatting the URL of the page of results to the query; and

redirecting user to said URL of the page of results of the clicked search engine.

19. The method of claim 18, wherein said page of results each search engine and searchable database has to the given query is presented in a frame inside the search assistant'"'"'s page of results to the query, comprising:

user clicking on a particular search engine or searchable database;

uploading search parameters from said interaction rules database;

formatting the URL of the page of results to the query; and

opening said URL of the page of results of the clicked search engine in a frame inside the search assistant'"'"'s page of results.

20. The method of claim 14, wherein said output rank brings hyperlinks pointing to the URL of the page of results each search engine and searchable database has to the given query, wherein said URL is dynamically generated every time a user requests it, comprising:

a user clicking on a particular search engine or searchable database;

uploading search parameters from said interaction rules database;

formatting the URL of the page of results to the query; and

redirecting user to said URL of the page of results of the clicked search engine.