System, method and computer program product for performing unstructured information management and automatic text analysis, including a search operator functioning as a weighted and (WAND)

• US 7,512,602 B2
• Filed: 11/30/2006
• Issued: 03/31/2009
• Est. Priority Date: 05/30/2003
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Zien, Jason, Broder, Andrei Z, Carmel, David, Herscovici, Michael, Soffer, Aya
• Primary Examiner(s): ROBINSON, GRETA LEE
• Application Number:
  US11/607,080
• Publication Number:
  US 20070112763A1
• Time in Patent Office: 852 Days
• Field of Search: 704/7, 704/9, 707/3, 707/4, 707/5, 707/100
• US Class Current: 1/1
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

Disclosed is a system architecture, components and a searching technique for an Unstructured Information Management System (UIMS). The UIMS may be provided as middleware for the effective management and interchange of unstructured information over a wide array of information sources. The architecture generally includes a search engine, data storage, analysis engines containing pipelined document annotators and various adapters. The searching technique makes use of a two-level searching technique. A search query includes a search operator containing of a plurality of search sub-expressions each having an associated weight value. The search engine returns a document or documents having a weight value sum that exceeds a threshold weight value sum. The search operator is implemented as a Boolean predicate that functions as a Weighted AND (WAND).

Claims

1. A method for processing stored data, comprising:

storing a collection of data units, said data units comprising documents;

retrieving at least one data unit in response to a query, the query comprising a search operator comprised of a plurality of search sub-expressions each having an associated weight value; and

outputting as a result to the query the retrieved at least one data unit, where said retrieved at least one data unit has a weight value sum that exceeds a threshold weight value sum, where said search operator comprises a weighted AND function, where varying the threshold weight value sum varies the operation of the weighted AND function from being substantially a logical OR function to being substantially a logical AND function, where at least one of the weight value sum and the threshold weight value sum is variable during a search.