LOAD SHEDDING IN A DATA STREAM MANAGEMENT SYSTEM
First Claim
1. A data stream management system, DSMS, comprising:
- a data receiving module configured to receive;
an input data stream from each of a plurality of data stream sources; and
respective location information associated with sets of at least one of the data stream sources;
a continuous query execution module configured to execute a continuous query against data items received via the input data streams to generate at least one output data stream for output to a client of the DSMS;
a load shedding module configured to execute a load shedding process when the DSMS is overloaded with data from the input data streams, so as to discard a plurality of the data items one of;
as received by the DSMS; and
that have been partially processed by the DSMS;
a learning module configured to determine, when the DSMS is not overloaded and for the location information associated with each of the sets of the at least one data stream source, a respective utility value indicating a utility to the client of the DSMS of data from the set of the at least one data stream source, by processing a feedback signal that is based on the at least one output data stream, the learning module being configured to store the location information in association with the corresponding data utility value; and
the load shedding module being configured to control the load shedding process by using location information received by the data receiving module when the DSMS is overloaded, together with the location information and data utility values stored by the learning module, to identify at least one input data stream whose data items are to be discarded.
1 Assignment
0 Petitions
Accused Products
Abstract
A data stream management system (DSMS) receives an input data stream from data stream sources and respective location information associated with sets of the data stream sources. A continuous query is executed against data items received via the input data streams to generate at least one client output data stream. A load shedding process is executed when the DSMS is overloaded with data from the input data streams. When the DSMS is not overloaded and for the location information associated with each of the data stream source sets, a respective utility value is determined indicating a utility to the client of data from the data stream source sets. The location information is stored in association with the corresponding data utility value. The location information received when the DSMS is overloaded is used, together with the data utility values, to identify input data streams whose data items are to be discarded.
13 Citations
22 Claims
-
1. A data stream management system, DSMS, comprising:
-
a data receiving module configured to receive; an input data stream from each of a plurality of data stream sources; and respective location information associated with sets of at least one of the data stream sources; a continuous query execution module configured to execute a continuous query against data items received via the input data streams to generate at least one output data stream for output to a client of the DSMS; a load shedding module configured to execute a load shedding process when the DSMS is overloaded with data from the input data streams, so as to discard a plurality of the data items one of; as received by the DSMS; and that have been partially processed by the DSMS; a learning module configured to determine, when the DSMS is not overloaded and for the location information associated with each of the sets of the at least one data stream source, a respective utility value indicating a utility to the client of the DSMS of data from the set of the at least one data stream source, by processing a feedback signal that is based on the at least one output data stream, the learning module being configured to store the location information in association with the corresponding data utility value; and the load shedding module being configured to control the load shedding process by using location information received by the data receiving module when the DSMS is overloaded, together with the location information and data utility values stored by the learning module, to identify at least one input data stream whose data items are to be discarded. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A data stream processing system, comprising:
-
a DSMS comprising; a data receiving module configured to receive; an input data stream from each of a plurality of data stream sources; and respective location information associated with sets of at least one of the data stream sources; a continuous query execution module configured to execute a continuous query against data items received via the input data streams to generate at least one output data stream for output to a client of the DSMS; a load shedding module configured to execute a load shedding process when the DSMS is overloaded with data from the input data streams, so as to discard a plurality of the data items one of; as received by the DSMS; and that have been partially processed by the DSMS; a learning module configured to determine, when the DSMS is not overloaded and for the location information associated with each of the sets of the at least one data stream source, a respective utility value indicating a utility to the client of the DSMS of data from the set of the at least one data stream source, by processing a feedback signal that is based on the at least one output data stream, the learning module being configured to store the location information in association with the corresponding data utility value; and the load shedding module being configured to control the load shedding process by using location information received by the data receiving module when the DSMS is overloaded, together with the location information and data utility values stored by the learning module, to identify at least one input data stream whose data items are to be discarded; and a DSMS client configured to receive and process the at least one output data stream generated by the DSMS, and configured to execute an action when triggered by the processing of at least one data item in the output data stream, the action comprising the sending by the DSMS client of a signal to a recipient different from the DSMS client; and a feedback loop configured to convey a feedback signal to the DSMS notifying the execution of an action by the DSMS client; the learning module of the DSMS is configured to determine, when the DSMS is not overloaded and for the location information associated with each of the sets of at least one data stream source, a respective utility value indicating a utility to the DSMS client of data from the at least one data stream source in the set, by processing the feedback signal conveyed by the feedback loop, the learning module being configured to store the location information in association with the corresponding data utility value.
-
-
11. A method of processing stream data in a data stream management system, DSMS, the method comprising the DSMS:
-
receiving an input data stream from each of a plurality of data stream sources and respective location information associated with sets of at least one er-more of the data stream sources; executing a continuous query against data items received via the input data streams to generate at least one output data stream for output to a client of the DSMS; determining, when the DSMS is not overloaded with data from the input data streams and for the location information associated with each of the sets of at least one data stream source a respective utility value indicating a utility to the client of the DSMS of data from the set of at least one data stream source, by processing a feedback signal that is based on the at least one output data stream; storing, in association with the location information associated with each of the sets of at least one data stream source the corresponding data utility value; and executing a load shedding process when the DSMS is overloaded with data from the input data streams, so as to discard some of the data items as one of received by the DSMS and that have been partially processed by the DSMS; and the load shedding process being controlled by using location information received when the DSMS is overloaded, together with the stored location information and data utility values, to identify one or more input data streams whose data items are to be discarded. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
-
21. A non-transitory computer-readable storage medium storing computer program instructions which, when executed by a processor, cause the processor to perform a method of processing stream data in a data stream management system, DSMS, the method comprising the DSMS:
-
receiving an input data stream from each of a plurality of data stream sources and respective location information associated with sets of at least one of the data stream sources; executing a continuous query against data items received via the input data streams to generate at least one output data stream for output to a client of the DSMS; determining, when the DSMS is not overloaded with data from the input data streams and for the location information associated with each of the sets of at least one data stream source, a respective utility value indicating a utility to the client of the DSMS of data from the set of at least one data stream source, by processing a feedback signal that is based on the at least one output data stream; storing, in association with the location information associated with each of the sets of at least one data stream source, the corresponding data utility value; and executing a load shedding process when the DSMS is overloaded with data from the input data streams, so as to discard some of the data items as one of received by the DSMS and that have been partially processed by the DSMS, the load shedding process being controlled by using location information received when the DSMS is overloaded, together with the stored location information and data utility values, to identify one or more input data streams whose data items are to be discarded.
-
-
22. (canceled)
Specification