Publication

This paper addresses the problem of minimizing the staleness of query results for streaming applications with update semantics under overload conditions. Staleness is a measure of how out-of-date the results are compared with the latest data arriving on the input. Real-time streaming applications are subject to overload due to unpredictably increasing data rates, while in many of them, we observe that data streams and queries in fact exhibit “update semantics” (i.e., the latest input data are all that really matters when producing a query result). Under such semantics, overload will cause staleness to build up. The key to avoid this is to exploit the update semantics of applications as early as possible in the processing pipeline. In this paper, we propose UpStream, a storage-centric framework for load management over streaming applications with update semantics.We first describe how we model streams and queries that possess the update semantics, providing definitions for correctness and staleness for the query results. Then, we show how staleness can be minimized based on intelligent update key scheduling techniques applied at the queue level, while preserving the correctness of the results, even for complex queries that involve sliding windows. UpStream is based on the simple idea of applying the updates in place, yet with great returns in terms of lowering staleness and memory consumption, as we also experimentally verify on the Borealis system.

@inproceedings{abc,
	abstract = {This paper addresses the problem of minimizing
the staleness of query results for streaming applications
with update semantics under overload conditions. Staleness
is a measure of how out-of-date the results are compared
with the latest data arriving on the input. Real-time streaming
applications are subject to overload due to unpredictably
increasing data rates, while in many of them, we observe that
data streams and queries in fact exhibit \&$\#$147;update semantics\&$\#$148;
(i.e., the latest input data are all that really matters when
producing a query result). Under such semantics, overload
will cause staleness to build up. The key to avoid this is to
exploit the update semantics of applications as early as possible
in the processing pipeline. In this paper, we propose
UpStream, a storage-centric framework for load management
over streaming applications with update semantics.We
first describe how we model streams and queries that possess
the update semantics, providing definitions for correctness
and staleness for the query results. Then, we show how staleness
can be minimized based on intelligent update key scheduling
techniques applied at the queue level, while preserving
the correctness of the results, even for complex queries that
involve sliding windows. UpStream is based on the simple
idea of applying the updates in place, yet with great returns
in terms of lowering staleness and memory consumption, as
we also experimentally verify on the Borealis system.},
	author = {Alexandru Moga and Irina Botan and Nesime Tatbul},
	booktitle = {VLDB J.},
	title = {UpStream: storage-centric load management for streaming applications with update semantics},
	url = {http://dx.doi.org/10.1007/s00778-011-0229-7},
	year = {2011}
}