SENSOR DATA PROCESSING IN A DISTRIBUTED SENSOR SYSTEM
First Claim
1. A method in a distributed sensor system, the distributed sensor system including a plurality of spatially distributed sensor nodes in communication with a data network, each sensor node comprising at least one sensor configured to measure an air quality parameter or an environmental parameter, the method comprising:
- providing a sensor data database including data tables for storing raw sensor data and processed sensor data;
providing a metadata database including metadata tables used to construct definitions for the data tables in the sensor data database;
receiving sensor data messages sent by one or more sensors in the sensor nodes of the distributed sensor system, each sensor data message including generically defined data fields adapted for reporting sensor data associated with different types of sensors;
storing the sensor data messages in a message queue;
providing the sensor data messages in the message queue on a first-in-first-out basis to a plurality of data processors having a parallel processing configuration, where each sensor data message is provided to one data processor;
maintaining the sensor data messages sent out to one or more data processors in the message queue;
processing, at each data processor, sensor data contained in the sensor data message;
storing raw and processed sensor data in the sensor data database;
in response to a sensor data message having completed processing by a data processor, discarding the sensor data message from the message queue;
generating, at the data processors, configuration update information based at least in part on the sensor data messages;
providing, by the data processors, the configuration update information to the metadata database to modify the metadata tables, there by modifying the definitions for the data tables for the sensor data database; and
providing, on a given time interval, modified table definitions to the sensor data database to modify or create data tables for storing raw or processed sensor data.
4 Assignments
0 Petitions
Accused Products
Abstract
A sensor data processing system and method is implemented in a distributed sensor system to facilitate the collection of sensor data generated from multiple sensors deployed in the field. The sensor data processing system and method processes the large volume of sensor data to generate sensor data with high data quality. More specifically, the sensor data processing system is implemented as a generically defined, metadata driven system to allow the distributed sensor system to be expanded or modified with ease. Furthermore, the sensor data processing system and method applies backend calibration to the sensor data to enhance the accuracy of the sensor data. Finally, the sensor data processing system and method of the present invention implements parallel sensor data processing using a processing pipeline to process each sensor data message for scalability and increased reliability.
-
Citations
16 Claims
-
1. A method in a distributed sensor system, the distributed sensor system including a plurality of spatially distributed sensor nodes in communication with a data network, each sensor node comprising at least one sensor configured to measure an air quality parameter or an environmental parameter, the method comprising:
-
providing a sensor data database including data tables for storing raw sensor data and processed sensor data; providing a metadata database including metadata tables used to construct definitions for the data tables in the sensor data database; receiving sensor data messages sent by one or more sensors in the sensor nodes of the distributed sensor system, each sensor data message including generically defined data fields adapted for reporting sensor data associated with different types of sensors; storing the sensor data messages in a message queue; providing the sensor data messages in the message queue on a first-in-first-out basis to a plurality of data processors having a parallel processing configuration, where each sensor data message is provided to one data processor; maintaining the sensor data messages sent out to one or more data processors in the message queue; processing, at each data processor, sensor data contained in the sensor data message; storing raw and processed sensor data in the sensor data database; in response to a sensor data message having completed processing by a data processor, discarding the sensor data message from the message queue; generating, at the data processors, configuration update information based at least in part on the sensor data messages; providing, by the data processors, the configuration update information to the metadata database to modify the metadata tables, there by modifying the definitions for the data tables for the sensor data database; and providing, on a given time interval, modified table definitions to the sensor data database to modify or create data tables for storing raw or processed sensor data. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A sensor data processing system in a distributed sensor system, the distributed sensor system including a plurality of spatially distributed sensor nodes in communication with a data network, each sensor node comprising at least one sensor configured to measure an air quality parameter or an environmental parameter, the sensor data processing system comprising:
-
a message queue to store sensor data messages sent by one or more sensors in the sensor nodes of the distributed sensor system, each sensor data message including generically defined data fields adapted for reporting sensor data associated with different types of sensors; a plurality of data processors configured for parallel processing, the data processors to receive sensor data messages from the message queue on a first-in-first-out basis, each data processor processing raw sensor data contained in the sensor data message and generating processed sensor data; a sensor data database in communication with the plurality of data processors to store raw sensor data and processed sensor data in data tables; and a sensor metadata database including metadata tables used to construct definitions for the data tables in the sensor data database, wherein the message queue provides the sensor data messages to the plurality of data processors on a first-in-first-out basis and maintains the sensor data messages sent out to one or more data processors in the message queue until the sensor data messages have completed processing at the data processors, and in response to a sensor data message having completed processing by a data processor, discarding the sensor data message from the message queue; and wherein the data processors generate configuration update information based at least in part on the sensor data messages and provide the configuration update information to the metadata database to modify the metadata tables, there by modifying the definitions for the data tables for the sensor data database; and
the sensor data database receiving, on a given time interval, modified table definitions to modify or create data tables for storing raw or processed sensor data. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
Specification