Processing time series data from multiple sensors

• US 9,645,736 B2
• Filed: 10/31/2013
• Issued: 05/09/2017
• Est. Priority Date: 10/31/2012
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chen, Xiao Yan, Huang, Sheng, Liu, Kai, Wang, Chen
• Primary Examiner(s): Nguyen, An T
• Application Number:
  US14/068,559
• Publication Number:
  US 20140122022A1
• Time in Patent Office: 1,286 Days
• Field of Search: 702181
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2322 using timestamps
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2477 Temporal data queries
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06Q 40/04 Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: Y04S 40/18 Network protocols supporting networked applications, e.g. including control of end-device applications over a network

Abstract

Processing time sequence data for multiple sensors, wherein the multiple sensors are divided into multiple sensor groups and each data comprises a time stamp and a value associated with the timestamp. The method comprises: receiving time series data from each sensor; assigning the time series data received to a sensor group to which the sensor belongs; storing time series data in a first database of a first memory, such that multiple time series data assigned to the same sensor group in the multiple sensor groups are stored in at least one database record of the first database; obtaining the time series data of each sensor among the multiple sensors from the first database; storing time series data in a second database of a second memory, such that the multiple time series data from the same sensor are stored in at least one database record of the second database.

Claims

1. A method for processing time series data from multiple sensors, wherein the multiple sensors are divided into multiple sensor groups and each time series data comprises a time stamp and a value associated with the timestamp, the method comprising:

a data receiving step of receiving time series data from each sensor of the multiple sensors;

a data grouping step of assigning the time series data received from each sensor to a sensor group to which the sensor belongs among the multiple sensor groups;

a first storing step of storing time series data in a first database of a first memory, such that multiple time series data assigned to the same sensor group in the multiple sensor groups are stored in at least one database record of the first database;

a data re-organizing step of obtaining the time series data of each sensor among the multiple sensors from the first database; and

a second storing step of storing time series data as per the values in a second database of a second memory, such that the multiple time series data from the same sensor are stored in at least one database record of the second database, the second storing step storing said time series data as per the values comprising;

calculating a probability distribution function PDF for values from each sensor;

dividing the values from each sensor into multiple value ranges based on the calculated PDF; and

storing time series data based on the divided value ranges, such that one database record comprises time series data corresponding to values within the same value range.

2. The method according to claim 1, wherein the second storing step further comprises:

storing time series data as per the timestamps.

3. The method according to claim 1, further comprising:

a data compressing step of compressing time series data for each sensor to generate compressed time series data.

4. The method according to claim 3, wherein the compressing time series data comprises one or more of the following:

compressing the timestamps in the time series data; and

performing linear compression to the values in the time series data from each sensor.

5. The method according to claim 4, wherein the storing time series data in a first database of a first memory comprises:

storing the compressed time series data in the first database of the first memory.

6. The method according to claim 4, wherein the compressing time series data comprises:

performing linear compression to values in time series data from each sensor, and the storing time series data in a second database of a second memory comprises;

storing the compressed time series data in the second database of the second memory.

7. The method according to claim 3, further comprising:

a query receiving step of receiving a query to the time series data; and

a query step of performing a query to the time series data in the second memory.

8. The method according to claim 7, wherein the query step comprises:

a query converting step of converting the received query into a query to the compressed time series data in the second memory;

a compressed data query step of performing a query to the compressed time series data in the second memory to obtain a query result; and

a decompressing step of decompressing the compressed time series data in the obtained query result.

9. An apparatus for processing time series data from multiple sensors, wherein the multiple sensors are divided into multiple sensor groups and each time series data comprises a time stamp and a value associated with the timestamp, the apparatus comprising:

a hardware processor;

a memory storage unit for storing programming instructions configuring the hardware processor to;

receive time series data from each sensor of the multiple sensors;

assign the time series data received from each sensor to a sensor group to which the sensor belongs among the multiple sensor groups;

store time series data in a first database of a first memory, such that multiple time series data assigned to the same sensor group in the multiple sensor groups are stored in at least one database record of the first database;

obtain the time series data of each sensor among the multiple sensors from the first database; and

store time series data as per the values in a second database of a second memory, such that the multiple time series data from the same sensor are stored in at least one database record of the second database, wherein to store said times series data as per the values, said hardware processor is further configured to;

calculate a probability distribution function PDF for values from each sensor, and

divide the values from each sensor into multiple value ranges based on the calculated PDF; and

store time series data based on the divided value ranges, such that one database record comprises time series data corresponding to values within the same value range.

10. The apparatus according to claim 9, wherein the hardware processor is further configured to store time series data as per the timestamps.

11. The apparatus according to claim 9, wherein the hardware processor is further configured to:

compress time series data for each sensor to generate compressed time series data.

12. The apparatus according to claim 11, wherein the hardware processor is further configured to:

compress the timestamps in the time series data; and

perform linear compression to the values in the time series data from each sensor.

13. The apparatus according to claim 12, wherein the hardware processor is further configured to store the compressed time series data in the first database of the first memory.

14. The apparatus according to claim 12, wherein the hardware processor is further configured to store the compressed time series data in the second database of the second memory.

15. The apparatus according to claim 11, wherein the hardware processor is further configured to:

receive a query to the time series data; and

perform a query to the time series data in the second memory.

16. The apparatus according to claim 15, wherein the hardware processor is further configured to:

convert the received query into a query to the compressed time series data in the second memory;

perform a query to the compressed time series data in the second memory to obtain a query result; and

decompress the compressed time series data in the obtained query result.