System and method for multi-dimensional aggregation over large text corpora

• US 7,720,837 B2
• Filed: 03/15/2007
• Issued: 05/18/2010
• Est. Priority Date: 03/15/2007
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: International Business Machines Corporation
• Inventors: Nguyen, Linda A., Kusnitz, Jeffrey A., Meredith, Daniel N.
• Primary Examiner(s): Alam; Shahid A
• Assistant Examiner(s): Hershley; Mark E
• Application Number:
  US11/686,639
• Publication Number:
  US 20080228718A1
• Time in Patent Office: 1,160 Days
• Field of Search: 707/3
• US Class Current: 707/711
• CPC Class Codes: G06F 16/319 Inverted lists

Abstract

Systems and methods for multi-dimensional aggregation. Exemplary embodiments include a method for retrieving data from an inverted list index within a computer system, wherein the index comprises annotated postings, the method including receiving a query in a system, converting the query into a query language, scanning at least one list of postings for data from the query, aggregating the data in the list, thereby resulting in an aggregated list, wherein the aggregating includes recording the occurrence of unique values from the list, mapping the values using a user-provided definition to an alternate value, grouping the values by a user-provided mapping of values to groups, recording and mutating data associated with the unique value in the list, relating the recorded data values with other values in the index and returning the requested data from the aggregated list in a return format.

Claims

1. A method for retrieving data from an inverted list index within a computer system, wherein the index comprises annotated postings, the method consisting of:

receiving a query in a system;

converting the query into a query language;

scanning at least one list of postings for data from the query;

aggregating the data in the list, thereby resulting in an aggregated list, wherein the aggregating includes;

recording the occurrence of unique values from the list;

mapping the values using a user-provided definition to an alternate value;

grouping the values by a user-provided mapping of values to groups;

recording and mutating data associated with the unique value in the list;

relating the recorded data values with other values in the index; and

returning the requested data from the aggregated list in a return format;

aggregating counts of the unique values over at least one aggregation key;

aggregating counts of the mappings of the values over the at least one aggregation key;

aggregating counts of the values over at least one set of values associated with the at least one aggregation key;

aggregating the mappings of the values over at least one set of values associated with the at least one aggregation key; and

aggregating mappings of alternate values over an aggregation of the values over the at least one aggregation key,

wherein the annotated postings contain document identification, occurrence identification, and occurrence related data, wherein alternately occurrence related data is accessible using document identification and occurrence identification,

wherein the unique value is the result of a computation on a pre-existing value,

wherein recording data associated with the unique value takes place during query processing,

wherein mutating data associated with the unique value includes numeric calculations, or referential mappings.