Monitoring Continuous Phenomena : Background, Methods and Solutions

Monitoring continuous phenomena by stationary and mobile sensors has become a common due to the improvement in hardware and communication infrastructure and decrease in it’s cost. Sensor data is now available in near real time via web interfaces and in machine-readable form, facilitated by paradigms like the Internet of Things (IoT). There are still some obstacles in the usability of the data since the positions (in space and time) of observation and the positions of interest usually do not coincide. Interpolation is the technique to fill such gaps and there are manifold methods to perform it. To actually operate a monitoring system, there are problems like unambiguous identification of interpolation method and associated parameters, appropriate interface to store observations and retrieve interpolated data, continuous update of the interpolation model for real time monitoring, compression and progressive retrieval of observational data and critical states definition and notification by using aggregation of values. This book proposes a general system architecture that addresses these problems. It is not confined to details about particular interpolation methods but rather takes a holistic view on the problem of monitoring. State-of-the-art technologies like geostatistics, sensor web enablement and field data types are introduced and applied in order to provide a viable toolset for the problem domain. The focus is on the overall organization of the monitoring and the architectural design of the software system and the associated simulation framework that is used to systematically evaluate different monitoring approaches. The whole cycle of a monitoring entailing observation, interpolation, discretization, storage, retrieval and notification is covered. Concrete solutions for several common problems in this context are provided. The monitoring of continuous phenomena like temperature, soil moisture or air pollutants is becoming more common due to the availability of Cover 1 Title Page 2 Copyright Page 3 Dedication 4 Foreword 5 Preface 9 Acknowledgements 11 Table of Contents 13 List of Figures 17 List of Tables 19 1. Introduction 22 1.1 Motivation and Challenges 23 1.2 Main Contributions 23 1.3 Observing and Interpolating Continuous Phenomena 30 1.4 Deterministic Approaches 32 1.5 Geostatistical Approaches 34 1.6 Mixed Approaches 36 1.7 Simulation 37 1.8 Summary 40 2. Monitoring Continuous Phenomena 42 2.1 Overview 43 2.2 Requirements 45 2.2.1 (Near) Real-Time Monitoring 45 2.2.2 Persistent Storage and Archiving 46 2.2.3 Retrieval 47 2.3 Resources and Limitations 48 2.3.1 Sensor Accuracy 50 2.3.2 Sampling 50 2.3.3 Computational Power 52 2.3.4 Time (Processing and Transmission) 52 2.3.5 Energy (Processing and Transmission) 53 2.4 Summary 54 3. Spatio-Temporal Interpolation: Kriging 58 3.1 Method Overview 59 3.2 The Experimental Variogram 60 3.3 The Theoretical Variogram and the Covariance Function 61 3.4 Variants and Parameters 66 3.5 Kriging Variance 69 3.6 Summary 71 4. Representation of Continuous Phenomena: Vector and Raster Data 72 4.1 Overview 73 4.2 Vector Data Properties 76 4.3 Raster Data Properties 77 4.4 Raster-Vector Interoperability 78 4.5 Summary 81 5. A Generic System Architecture for Monitoring Continuous Phenomena 82 5.1 Overview 84 5.2 Workflow Abstraction Concept 85 5.2.1 Datasets (Input/Source and Output/Sink) 87 5.2.2 Process/Transmission 88 5.3 Monitoring Process Chain 89 5.3.1 Random Field Generation by Variogram Filter 91 5.3.2 Sampling and Sampling Density 94 5.3.3 Experimental Variogram Generation 100 5.3.4 Experimental Variogram Aggregation 101 5.3.5 Variogram Fitting 106 5.3.6 Kriging 109 5.3.7 Error Assessment 109 5.4 Performance Improvements for Data Stream Management 110 5.4.1 Problem Context 111 5.4.2 Sequential Model Merging Approach 112 5.4.2.1 Overview 112 5.4.2.2 Related Work 113 5.4.2.3 Requirements 113 5.4.2.4 Principle 114 5.4.2.5 Partitioning Large Models: Performance Considerations 116 5.4.3 Compression and Progressive Retrieval 119 5.4.3.1 Overview 119 5.4.3.2 Related Work 120 5.4.3.3 Requirements 120 5.4.3.4 Principle 121 5.4.3.5 Binary Interval Subdivision 121 5.4.3.6 Supported Data Types 122 5.4.3.7 Compression Features 124 5.5 Generic Toolset for Variation and Evaluation of System Configurations 126 5.5.1 Context and Abstraction 127 5.5.2 Computational Workload 130 5.5.3 Systematic Variation of Methods, Parameters and Configurations 134 5.5.4 Overall Evaluation Concept 136 5.6 Summary 139 6. A General Concept for Higher Level Queries about Continuous Phenomena 140 6.1 Introduction 141 6.2 Interpolation 142 6.3 Intersection 145 6.4 Aggregation 146 6.5 Conclusions 148 7. Experimental Evaluation 150 7.1 Minimum Sampling Density Estimator 152 7.1.1 Experimental Setup 152 7.1.2 Results 152 7.1.3 Conclusions 156 7.2 Variogram Fitting 156 7.2.1 Experimental Setup 157 7.2.2 Results 160 7.2.3 Conclusions 162 7.3 Sequential Merging 162 7.3.1 Experimental Setup 163 7.3.2 Results 163 7.3.3 Conclusions 165 7.4 Compression 166 7.4.1 Experimental Setup 166 7.4.2 Results 169 7.4.3 Conclusions 171 7.5 Prediction of Computational Effort 172 7.5.1 Experimental Setup 172 7.5.2 Results 173 7.5.3 Conclusions 173 7.6 Higher Level Queries 174 7.6.1 Experimental Setup 174 7.6.2 Results 178 7.6.3 Conclusions 180 7.7 Case Study: Satellite Temperature Data 183 7.7.1 Experimental Setup 184 7.7.2 Results 186 7.7.3 Conclusions 188 8. Conclusions 190 8.1 Subsuming System Overview 191 8.2 Perspective 196 References 198 Index 210 Interpolation;,System,architecture;,Field,data,type;,Geostatistical,methods;,Remote,sensing Interpolation,System architecture,Field data type,Geostatistical methods,Remote sensing "The monitoring of continuous phenomena like temperature, soil moisture or air pollutants is becoming more common due to the availability of cheaper and better sensor infrastructure. Better methods for processing, storing and provisioning the data produced by such observations are still evolving. The fundamental features among them are interpolation, real-time provisioning, critical states definitions (e.g. exceeded threshold of the daily mean) and field data type management. The book gives an overview of monitoring and proposes several solutions for problems that occur in this context"-- Provided by publisher