Sensor data locating

• US 9,355,106 B2
• Filed: 04/23/2013
• Issued: 05/31/2016
• Est. Priority Date: 04/27/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: An, Wen Hao, Duan, Ning, Fan, Liya, Gao, Bo, Hu, Ke, Sun, Wei, Wang, Yu Ying, Wang, Zhi Hu
• Primary Examiner(s): PENG, HUAWEN A
• Application Number:
  US13/868,559
• Publication Number:
  US 20130311480A1
• Time in Patent Office: 1,134 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/148 File search processing

Abstract

A method, an apparatus, and a system for locating sensor data. The method includes the steps of: obtaining an index table; intercepting a query for sensor data in runtime; extracting a characteristic parameter from a query condition; locating a block identifier of matching sensor data storage blocks in the index table by using the characteristic parameter; and loading the storage blocks into a memory space of a working processor; where the index table contains mapping relationships between block identifiers of sensor data storage blocks and characteristic attributes of sensor data.

Claims

1. A method for locating a sensor data stored in a distributed data system, comprising the steps of:

obtaining an index table for the distributed data system;

intercepting a query for sensor data in runtime;

extracting a characteristic parameter from a condition of the query, wherein the characteristic parameter includes at least one of a temporal parameter and a spatial parameter of the sensor data;

locating, in the index table, a block identifier of one or more storage blocks of the distributed data system that include sensor data corresponding to the characteristic parameter, wherein the block identifier identifies a location of the one or more storage blocks in the distributed data system; and

loading only the one or more storage blocks of the distributed data system into a memory space of a working processor that correspond to the block identifier;

wherein said index table contains mapping relationships between block identifiers of storage blocks of the distributed data system and characteristic attributes of the sensor data, wherein the characteristic attributes of the sensor data includes at least one of a temporal parameter and a spatial parameter of the sensor data and wherein the index table is created prior to intercepting the query.

2. The method of claim 1, further comprising the step of creating said index table.

3. The method of claim 2, wherein creating said index table is realized by scanning data stored in storage blocks on data nodes managed by a Hadoop-like Distributed File System.

4. The method of claim 2, wherein creating said index table is realized during a process of writing sensor data into data nodes managed by a Hadoop-like Distributed File System.

5. The method of claim 1, wherein said index table further contains relationships between said block identifiers of sensor data storage blocks and other attributes of sensor data.

6. The method of claim 1, wherein the method is realized on a name node of a Hadoop-like architecture, so as to realize seamless integration of the method and said Hadoop-like architecture.

7. An system for locating a sensor data stored in a distributed data system, the system comprising a processor configured to:

obtain an index table for the distributed data system;

intercept a query for sensor data in runtime, and extract a characteristic parameter from a condition of the query, wherein the characteristic parameter includes at least one of a temporal parameter and a spatial parameter of the sensor data; and

locate, in the index table, a block identifier of one or more storage blocks of the distributed data system that include sensor data corresponding to the characteristic parameter wherein the block identifier identifies a location of the one or more storage blocks in the distributed data system, and

load only the one or more storage blocks into a memory space of a working processor that correspond to the block identifier;

wherein said index table contains mapping relationships between block identifiers of storage blocks of the distributed data system and characteristic attributes of the sensor data, wherein the characteristic attributes of the sensor data includes at least one of a temporal parameter and a spatial parameter of the sensor data and wherein the index table is created prior to intercepting the query.

8. The apparatus of claim 7, wherein the processor is configured to create said index table.

9. The apparatus of claim 8, wherein the processor is further configured to create said index table by scanning data stored in storage blocks on data nodes managed by a Hadoop-like Distributed File System.

10. The apparatus of claim 8, wherein the processor is further configured to create said index table during a process of writing sensor data into data nodes managed by a Hadoop-like Distributed File System.

11. The apparatus of claim 7, wherein said index table further contains relationships between a block identifier of sensor data storage blocks and other attributes of sensor data.

12. The apparatus of claim 7, wherein the apparatus is realized on a name node of a Hadoop-like architecture, so as to realize a seamless integration of the apparatus and said Hadoop-like architecture.

13. A system for locating a sensor data in a distributed data system, comprising:

a memory;

a processor device communicatively coupled to said memory;

a sensor data locating module communicatively coupled to said memory and said processor device, wherein said sensor data locating module is configured to perform steps of a method comprising;

obtaining an index table for the distributed data system;

intercepting a query for sensor data in runtime;

extracting a characteristic parameter from a condition of the query, wherein the characteristic parameter includes at least one of a temporal parameter and a spatial parameter of the sensor data;

locating, in the index table, a block identifier of one or more storage blocks of the distributed data system that include sensor data corresponding to the characteristic parameter wherein the block identifier identifies a location of the one or more storage blocks in the distributed data system; and

loading only the one or more storage blocks into said memory that correspond to the block identifier;

wherein said index table contains mapping relationships between block identifiers of storage blocks f the distributed data system and characteristic attributes of the sensor data, wherein the characteristic attributes of the sensor data includes at least one of a temporal parameter and a spatial parameter of the sensor data and wherein the index table is created prior to intercepting the query.

14. The system of claim 13, wherein said index table further contains relationships between a block identifier of sensor data storage blocks and other attributes of sensor data.

15. The system of claim 13, wherein said sensor data locating module is further configured to also perform the step of creating an index table.

16. The system of claim 15, wherein said sensor data locating module is further configured to create said index table by scanning data stored in storage blocks on data nodes managed by a Hadoop-like Distributed File System.

17. The system of claim 15, wherein said sensor data locating module is further configured to create said index table during a process of writing sensor data into data nodes managed by a Hadoop-like Distributed File System.