Efficient processing of time series data
First Claim
1. A method for processing data, comprising:
- receiving time series measurement data;
writing said time series measurement data to a first format suited for fast writing; and
converting said time series measurement data from said first format to a second format suited for compact storage.
3 Assignments
0 Petitions
Accused Products
Abstract
A system receiving time series data will store the data in a first format suited for quick writing. That data will subsequently be converted from the first format to a second format better suited than the first format for compact storage and queries. Over time, the data in the second format can be aggregated and/or down sampled. In one embodiment, the first format includes a set of blocks of information, with each block of information storing data for a set of metrics during a time period. The second format includes a header, a set of blocks of data and a footer. Each bock of data in the second format stores information for a metric over multiple time periods. The footer includes pointers to each block of data.
-
Citations
48 Claims
-
1. A method for processing data, comprising:
-
receiving time series measurement data;
writing said time series measurement data to a first format suited for fast writing; and
converting said time series measurement data from said first format to a second format suited for compact storage. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A method for processing data, comprising:
-
receiving time series measurement data;
writing said time series measurement data to a first format suited for fast writing; and
converting said time series measurement data from said first format to a second format suited for performing queries faster than said first format. - View Dependent Claims (14, 15, 16, 17, 18, 19)
-
-
20. A method for processing data, comprising:
-
receiving measurement data about a system, said measurement data includes a set of data for each time period of a set of time periods, each set of data includes information for a set of metrics about said system during a respective time period;
saving said measurement data in a first format such that information from a particular set of data is stored together; and
converting said measurement data from said first format to a second format such that information for each metric is stored together. - View Dependent Claims (21, 22, 23, 24, 25, 26, 27)
-
-
28. A method for processing data, comprising:
-
receiving time series data that includes multiple samples of said data over time;
writing said time series data to a first format suited for fast writing; and
converting said time series data from said first format to a second format having blocks of records, said converting includes compressing said records individually, said blocks have data from multiple samples of said data. - View Dependent Claims (29, 30, 31, 32)
-
-
33. A method for processing data, comprising:
-
receiving time series data;
writing said time series data to a first format suited for fast writing during a first time period;
converting said time series data from said first format to a second format suited long term storage after said first time period;
receiving a query of said time series data during said first time period;
performing said query of said time series data during said first time period using said time series data in said first format;
receiving a query of said time series data after said first time period; and
performing said query of said time series data after said first time period using said time series data in said second format. - View Dependent Claims (34)
-
-
35. A method for processing data about a system, comprising:
-
adding probes to a software application;
acquiring performance data about said software application using said probes in real time;
sending said performance data to a data manager;
storing said performance data in one or more searchable files according to a first format; and
periodically converting a subset of said performance data to searchable files according to a second format. - View Dependent Claims (36, 37, 38)
-
-
39. One or more processor readable storage devices storing processor readable code for programming one or more processors, said processor readable code comprising:
-
a writer, said writer receives data about a set of metrics and stores said data in a first format having blocks sorted by time; and
a converter, said converter accesses said data in said first format and converts said data from said first format to a second format having blocks sorted by metric. - View Dependent Claims (40, 41, 42, 43)
-
-
44. An apparatus for processing data, comprising:
-
a communication interface;
a storage device; and
one or more processors in communication with said storage device and said communication interface, said one or more processors perform a method comprising;
receiving measurement data about a system, said measurement data includes a set of data for a set of metrics during each time period of a set of time periods, storing said measurement data in real time in a first format, converting said stored measurement data from said first format to a second format and storing said measurement data in said second format in a set of one or more files associated with contiguous time periods, aggregating said set of one or more files; and
down sampling said aggregated set of one or more files. - View Dependent Claims (45)
-
-
46. One or more processor readable storage devices storing processor readable code for programming one or more processors, said processor readable code comprising:
-
code means for receiving time series measurement data;
code means for writing said time series measurement data to a first format suited for fast writing; and
code means for converting said time series measurement data from said first format to a second format suited for compact storage. - View Dependent Claims (47, 48)
-
Specification