Methods and apparatus for statistical data analysis and reduction for an enterprise application

• US 20030158851A1
• Filed: 11/21/2002
• Published: 08/21/2003
• Est. Priority Date: 07/27/2001
• Status: Active Grant

• Current Assignee: Metatomix, Inc. (ESW Capital, LLC)
• Original Assignee: Metatomix, Inc. (ESW Capital, LLC)
• Inventors: Kumar, Ashok, Britton, Colin P., Bigwood, David, Greenblatt, Howard
• Granted Patent: US 7,302,440 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 16/972 Access to data in other repository systems, e.g. legacy data or dynamic Web page generation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

The invention provides methods of time-wise data reduction that include the steps of inputting data from a source; summarizing that data according to one or more selected epochs in which it belongs; and generating for each such selected epoch one or more RDF triples characterizing the summarized data. The data source may be, for example, a database, a data stream or otherwise. The selected epoch may be a second, minute, hour, week, month, year, or so forth. The triples may be output in the form of RDF document objects. These can be stored, for example, in a hierarchical data store such as, for example, a WebDAV server. Triples parsed from the document objects may be maintained in a relational store that is organized, for example, according to a hashed with origin approach.

Claims

1. A method of time-wise data reduction and storage, comprising

A. inputting data from at least one source,

B. summarizing that data according to a specified epoch in which it belongs,

C. generating for each such epoch, one or more RDF triples characterizing the summarized data.

2. The method of claim 1, comprising outputting the RDF triples in one or more RDF document objects.

3. The method of claim 2, comprising storing the RDF document objects in a hierarchical data store.

4. The method of claim 3, comprising store the RDF document objects in accord with a WebDAV protocol.

5. The method of claim 1, comprising storing the RDF triples in a relational data store.

6. The method of claim 5, comprising storing the RDF triples in a relational data organized according to a hashed with origin approach.

7. A method of time-wise data reduction and storage, comprising

A. querying one or more data sources,

B. summarizing data received from the data sources in response to querying, where the data is summarized by selected epoch,

C. generating for each such epoch, one or more RDF triples characterizing the summarized data,

D. storing the RDF triples to one or more data stores, along with further RDF triples characterizing the data from which the summaries were generated, where the one or more data stores include any of a hierarchical data store and a relational data store.

8. The method of claim 7, comprising summarizing data received from the data sources with respect to multiple epochs of differing length.

9. The method of claim 7, comprising querying one or more data sources in an SQL format.

10. The method of claim 7, comprising parsing an XML file that identifies one or more of the data sources, one or more fields thereof to be summarized, and/or one or more epochs for which those fields are to be summarized.

11. The method of claim 7, comprising responding data received from a data source by updating a store associated with an epoch of shorter duration.

12. The method of claim 11, comprising updating a store associated with an epoch of longer duration based on information maintained in an epoch of shorter duration.

13. A method of time-wise data reduction and storage, comprising

A. at least one of querying and filtering data from one or more data sources,

B. summarizing the data received in one or more selected epochs of differing length,

C. generating RDF document objects comprising one or more RDF triples characterizing the summarized data,

D. storing the RDF documents to a first, hierarchical data store,

E. storing the triples therein to a second, relational data store.

14. The method of claim 13, comprising querying one or more data sources in an SQL format.

15. The method of claim 13, comprising parsing an XML file that identifies one or more of the data sources, one or more fields thereof to be summarized, and/or one or more epochs for which those fields are to be summarized.

16. The method of claim 13, comprising responding data received from a data source by updating a store associated with an epoch of shorter duration.

17. The method of claim 16, comprising updating a store associated with an epoch of longer duration based on information maintained in an epoch of shorter duration.

18. The method of claim 13, comprising generating a display or other presentation based on the RDF triples characterizing the summarized data.