Computer-implemented system and method for analyzing search queries

• US 7,577,641 B2
• Filed: 09/07/2004
• Issued: 08/18/2009
• Est. Priority Date: 09/07/2004
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Scott, Kevin L., Pope, David C., Koch, Donald O.
• Primary Examiner(s): Truong; Cam Y T
• Application Number:
  US10/935,734
• Publication Number:
  US 20060053095A1
• Time in Patent Office: 1,806 Days
• Field of Search: 707/3, 707/10, 707/6
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

Systems and methods for handling input search engine data. A data store is searched based upon one or more data items contained within the input search engine data. The result of searching the data store is used to determine which data items in the input search engine data might be a search term. Search terms are extracted based upon the result of searching the data store.

Claims

1. A method for operation upon a data processing device to determine which terms in a weblog are search terms, wherein the search terms are from different types of search engines, wherein the different types of search engines include a first type of search engine and a second type of search engine, the method comprising:

receiving, by using the data processing device, input search engine data from the weblog, wherein the input search engine data is a collection of search strings that are query inputs for use by the different types of search engines in performing queries to generate search results, the data processing device includes a memory;

wherein the input search engine data includes click-stream referrers;

using one or more data items contained within the input search engine data to search an indexed data store in the memory, wherein the indexed data store does not include a list of the different types of search engines;

determining which of the one or more data items are search terms by comparing a portion of a query input to data in the indexed data store;

extracting search terms from the input search engine data when the portion of the query input matches the data in the indexed data store, wherein the extracted search terms include a first search term and a second search term, the first search term comprising a query input to the first type of search engine, and the second search term comprising a query input to the second type of search engine;

wherein the indexed data store is a hash table containing pre-selected search term parameter names, and wherein the extracting the search terms from the input search engine data is based upon whether a data item from the input search engine data is found within the hash table;

wherein the input search engine data includes query strings formatted according to a hypertext transfer protocol, the query strings including name/value pairs associated with a referrer, and wherein data items used in searching the hash table include data items from the name/value pairs;

performing additional search engine tests upon data items from the input search engine data, the additional search engine tests including a test to determine whether a name data item matches a pattern of known search engines, a test to determine whether a value data item matches a pattern of known search strings that have been identified as not being search terms, a test to determine whether a value data item satisfies a pre-selected minimum length requirement, and a test to determine whether a value data item matches a pattern describing only numbers or non-alphanumeric characters;

updating the hash table with a data item that is not found within the hash table when one or more of the additional search engine tests were satisfied with respect to the data item that is not found within the hash table; and

generating, by using the extracted search terms, a report indicating user searching profiles.

2. The method of claim 1, wherein the extracting the search terms from the input search engine data is search engine independent;

wherein at least two of the search engines are different types of search engines that use different formats and identifiers for search terms within respective query inputs, a query input including user input used by a search engine to perform a search; and

wherein the input search engine data includes the user input used by the search engine and stored in a weblog.

3. The method of claim 2, wherein the extracting the search terms obviates a need to verify that a referrer provided in the input search engine data is a known search engine.

4. The method of claim 3, wherein new types of search engines arise over time and the searching of the indexed data store is performed independently of type of search engine.

5. The method of claim 1, wherein the extracting the search terms is substantially independent of search engine type, and wherein the different types of search engines use different formats and identifiers for search terms within respective query inputs.

6. The method of claim 1, wherein the input search engine data includes click-stream data generated from one or more users of internet web pages.

7. The method of claim 1, wherein a data mining engine processes and analyzes the extracted search terms in order to determine the user searching profiles contained in the report.

8. The method of claim 1, wherein the report is used by e-business web site managers for analyzing website activity.

9. The method of claim 1, wherein the searching the indexed data store includes using data items generated by processing name/value pairs of a query string associated with a referrer.

10. The method of claim 9, further comprising:

comparing the name/value pairs to data stored in the indexed data store to determine whether a value should be considered a valid search term.

11. The method of claim 1, wherein the input search engine data includes a referrer, the method further comprising:

parsing the referrer to identify the query input;

parsing the query input into name/value pairs; and

using names or values as data items for searching the indexed data store to determine search terms.

12. The method of claim 1, wherein a look-up table containing a list of non-search terms is used to determine whether a data item is a search term.

13. The method of claim 12, wherein a look-up table containing a list of possible search terms is used to determine whether a data item is a search term.

14. A system for operation upon a data processing device to determine which terms in a weblog are search terms, wherein the search terms are from different types of search engines, wherein the different types of search engines include a first type of search engine and a second type of search engine, the system comprising:

a processor;

a memory containing instructions for use in execution by the processor to perform operations including;

receiving input search engine data from the weblog, wherein the input search engine data is a collection of search strings that are query inputs for use by the different types of search engines in performing queries to generate search results;

wherein the input search engine data includes click-stream referrers;

using one or more data items contained within the input search engine data to search an indexed data store, wherein the indexed data store does not include a list of the different types of search engines;

determining which of the one or more data items are search terms by comparing a portion of a query input to data in the indexed data store;

extracting search terms from the input search engine data when the portion of the query input matches the data in the indexed data store, wherein the extracted search terms include a first search term and a second search term, the first search term comprising a query input to the first type of search engine, and the second search term comprising a query input to the second type of search engine;

wherein the indexed data store is a hash table containing pre-selected search term parameter names, and wherein the extracting the search terms from the input search engine data is based upon whether a data item from the input search engine data is found within the hash table;

wherein the input search engine data includes query strings formatted according to a hypertext transfer protocol, the query strings including name/value pairs associated with a referrer, and wherein data items used in searching the hash table include data items from the name/value pairs;

performing additional search engine tests upon data items from the input search engine data, the additional search engine tests including a test to determine whether a name data item matches a pattern of known search engines, a test to determine whether a value data item matches a pattern of known search strings that have been identified as not being search terms, a test to determine whether a value data item satisfies a pre-selected minimum length requirement, and a test to determine whether a value data item matches a pattern describing only numbers or non-alphanumeric characters;

updating the hash table with a data item that is not found within the hash table when one or more of the additional search engine tests were satisfied with respect to the data item that is not found within the hash table; and

generating, by using the extracted search terms, a report indicating user searching profiles.