Scalable data stream monitoring framework for large-scale ubiquitous services

Abstract

In this thesis, we envision the necessity of a scalable data stream monitoring framework to address the unique challenges of emerging large-scale ubiquitous services, proliferating in the near future (e.g., large-scale urban sensor networks, crowd-aware city business). The emerging ubiquitous services will utilize a bunch of massive data streams continuously generated from a real world. Accordingly, they will be situation-aware; the services proactively detect the occurrences of interested events and trigger proper actions automatically. For this purpose, they commonly rely on real-time monitoring on voluminous data streams. Often, such services are remarkably large-scale, spanning a number of people and devices over a wide geographic area. To address these challenges, we propose a scalable monitoring framework consisting of $\textsl{BMQ-Processor}$ and $\textsl{Tran-Seq}$, which effectively achieves semantics and processing requirements newly arising in the large-scale, senor-rich ubiquitous services. First, we present BMQ-Processor, a high-performance Border-Crossing Event (BCE) detection framework. A BCE is intuitively represented as a data stream crossing the borders of a user-specified interest range. Many scenarios show that BCE is an important and frequently-used primitive event. To facilitate the specification of massive BCEs, we first characterize a new query semantics, namely, Border Monitoring Query (BMQ). More importantly, we develop a high-performance processing mechanism which operates over a novel $\textsl{stateful query index}$. BMQ-Processor efficiently evaluates a large number of BMQs over voluminous data streams in a shared and incremental manner. In particular, we design BMQ-Processor that supports one- and multi-dimensional BMQs. Lastly, we demonstrate excellent processing performance and low storage cost of BMQ-Processor through extensive experiments and analysis. Second, we present Tran-Seq, a high-performance BCE composition fram...