Advancing Big Data Benchmarks: Proceedings of the 2013 Workshop Series on Big Data Benchmarking, WBDB. cn, Xi'an, China, July16-17, 2013 and WBDB. us, San José, CA, USA, October 9-10, 2013, Revised Selected Papers

This book constitutes the thoroughly refereed joint proceedings of the Third and Fourth Workshop on Big Data Benchmarking. The third WBDB was held in Xi'an, China, in July 2013 and the Fourth WBDB was held in San José, CA, USA, in October, 2013. The 15 papers presented in this book were carefully reviewed and selected from 33 presentations. They focus on big data benchmarks; applications and scenarios; tools, systems and surveys. Preface 6 WBDB 2012 Organization 7 WBDB 2012 Sponsors 8 Contents 9 Big Data Benchmarks 11 A BigBench Implementation in the Hadoop Ecosystem 12 1 Introduction 12 2 BigBench Overview 13 3 Technologies for BigBench on Hadoop 14 3.1 Hadoop 14 3.2 Hive 15 3.3 Mahout and NLTK 15 4 Query Implementation 16 4.1 Loading Data 16 4.2 Hive Queries 17 4.3 Hive and MapReduce 18 4.4 Hive and Natural Language Processing 19 4.5 Mahout 20 5 Evaluation 24 6 Related Work 25 7 Conclusion 26 References 26 A Mid-Flight Synopsis of the BG Social Networking Benchmark 28 1 Introduction 28 2 Scalability 31 3 Closed Versus Open 34 4 Rating Mechanism 34 5 Validation 36 6 Additional Actions 37 7 Conclusions and Future Research 38 References 40 A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks 41 1 Introduction 41 2 Related Work 43 3 Design Considerations 44 4 Benchmarks for Hadoop RPC 44 5 Performance Evaluation 46 5.1 Experimental Setup 46 5.2 Micro-benchmark Results 46 6 A Case Study: Enhance Hadoop RPC Design Over Native InfiniBand 48 7 Conclusion and Future Work 49 References 50 Experience from Hadoop Benchmarking with HiBench: From Micro-Benchmarks Toward End-to-End Pipelines 52 Abstract 52 1 Introduction 52 2 Experience with HiBench 53 2.1 Overview of HiBench 53 2.2 Tradeoffs Between HiBench and Synthetic Trace-Based Approaches 54 2.3 ETL-Recommendation Pipeline 55 3 Future Work 57 References 57 WGB: Towards a Universal Graph Benchmark 67 1 Introduction 67 2 Existing Graph Benchmarks 68 2.1 HPC Scalable Graph Analysis Benchmark 68 2.2 Graph Traversal Benchmark 69 2.3 Graph500 69 2.4 BigBench 69 2.5 BigDataBench 69 2.6 RDF Benchmarks 70 3 Workload Characterization 70 3.1 Online Queries 71 3.2 Updates 73 3.3 Iterative Queries 73 4 Data Generation 74 5 WGB in Action 76 6 Graph Processing and Analysis Systems 77 6.1 MapReduce-Based Systems 77 6.2 Vertex-Centric Systems 79 7 Conclusion and Future Work 79 References 80 HcBench: Methodology, Development, and Full-System Characterization of a Customer Usage Representative Big Data/Hadoop Benchmark 82 Abstract 82 1 Introduction 82 2 Related Work 84 3 Methodology for Benchmark Development 85 3.1 Job Diversity 86 3.2 Number of Jobs 86 3.3 Input Data Size Mix 87 3.4 Compute, Storage, and Network Intensive Jobs 87 3.5 Inter-job Arrival Pattern 87 3.6 Putting It All Together 88 4 HcBench Application Level Performance 89 4.1 Hadoop Cluster HW and SW Components 89 4.2 Benchmark Performance Summary 89 4.3 Benchmark Performance Summary 90 4.4 Data Throughput 90 4.5 Concurrency During Map and Reduce Phases 90 4.6 Response Times and Map and Reduce Execution Times 91 4.7 Benchmark Repeatability 92 4.8 Response Time Job SLA vs Throughput 93 5 HcBench OS Level Performance Characterization 93 5.1 CPU Utilization 94 5.2 HDFS Disk Bandwidth and Request Size 94 5.3 RX and TX Network Bandwidth 95 5.4 Performance Summary 98 6 Micro-architectural Level Performance Characterization 98 6.1 Micro-architectural CPU Performance Metrics 98 6.2 Benchmark Full-Run: DataNode 0 98 6.3 Benchmark Run in Steady-State: DataNodes 0 to 7 100 7 Conclusions and Future Work 101 Acknowledgements 102 References 102 Applications and Scenarios 103 Big Data Workloads Drawn from Real-Time Analytics Scenarios Across Three Deployed Solutions 104 Abstract 104 1 Introduction 104 2 Motivation and Related Work 105 3 Smart City &hx2013; Solution Flow, Workload, and Metrics 106 4 Content Management Under High Volume &hx2013; Solution Architecture, Workload, Metrics 108 5 Electronic Fraud Detection and Prevention: A Simple Real-Time Workload 109 6 Summary 110 References 111 Large-Scale Chinese Cross-Document Entity Disambiguation and Information Fusion 112 1 Introduction 112 2 System Overview 113 3 Document-Level Information Extraction 115 4 Corpus Level Information Extraction 119 5 Conclusion and Further Work 124 References 125 Tools, Systems, and Surveys 127 Big Data Operations: Basis for Benchmarking a Data Grid 128 Abstract 128 1 Introduction 128 2 Data-Oriented Systems 130 2.1 iRODS 132 3 Operations in a Data Grid 133 3.1 Search and Discovery 135 3.2 Data Movement 135 3.3 Access Control 136 4 Benchmarking 137 4.1 Benchmarking Approach 138 5 Conclusion 141 Acknowledgement 141 References 141 BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking 143 1 Introduction 143 2 Background and Requirements 145 3 Related Work 146 4 The Methodology of BDGS 148 4.1 Selection of Data Sets 148 4.2 Generation of Synthetic Data Sets 149 5 Real-World Data Sets 149 6 Synthetic Data Generators 150 6.1 Text Generator 151 6.2 Graph Generator 152 6.3 Table Generator 153 7 Experimental Evaluation 154 7.1 Metric 154 7.2 Experiment Configurations 155 7.3 Evaluation of BDGS 155 8 Conclusions 157 References 157 A Multidimensional OLAP Engine Implementation in Key-Value Database Systems 160 Abstract 160 1 Introduction 160 2 Related Work 161 2.1 Business Intelligence on Big Data 161 2.2 Cube Algebra with TPC-DS Queries 163 3 System Design 164 3.1 Demand Driven Cube Modeling 165 3.2 Cube Logical Structure Generation 165 3.3 Cube Physical Storage Implementation 166 3.4 Cube Data Shard and Distributed Aggregation Computation 167 3.5 Cube Data Initialization 167 3.6 OLAP Query Execution 168 3.7 Resilient Distributed Datasets (RDDs) 169 4 Experiments for Big Data Benchmark 170 4.1 Cube Data Population Testing 172 4.2 Cube Querying 173 5 Conclusion 174 Acknowledgements 174 References 174 Big Data Cloud-Based Advisory System 176 Abstract 176 1 Introduction 176 2 Approach 177 3 Architecture 177 3.1 Overview 177 3.2 Controllers 179 3.3 Service Side 180 4 Testing 181 5 Advisory Service 181 6 Prototype 182 7 Conclusion 182 References 182 MPP SQL Engines: Architectural Choices and Their Implications on Benchmarking 184 Abstract 184 1 Introduction 184 2 Hardware Infrastructure 185 2.1 Server 185 2.2 Network 185 2.3 Storage 185 3 MPP SQL Engines 187 4 Implications for Benchmarking 189 4.1 Hardware Infrastructure 189 4.2 SQL Engine 190 5 Benchmark 191 6 Benchmark Results 192 6.1 Hardware Platforms 192 6.2 SQL Engines 195 7 Conclusions 196 References 197 Towards an Industry Standard for Benchmarking Big Data Systems 198 Abstract 198 1 Introduction 198 2 Why TPC? 200 3 What does it Take? 203 4 Update from the TPC 204 5 Conclusion 204 Acknowledgements 205 References 205 Author Index 207 Front Matter....Pages I-XII Front Matter....Pages 1-1 A BigBench Implementation in the Hadoop Ecosystem....Pages 3-18 A Mid-Flight Synopsis of the BG Social Networking Benchmark....Pages 19-31 A Micro-benchmark Suite for Evaluating Hadoop RPC on High-Performance Networks....Pages 32-42 Experience from Hadoop Benchmarking with HiBench: From Micro-Benchmarks Toward End-to-End Pipelines....Pages 43-48 Big Data Benchmark - Big DS....Pages 49-57 WGB: Towards a Universal Graph Benchmark....Pages 58-72 HcBench: Methodology, Development, and Full-System Characterization of a Customer Usage Representative Big Data/Hadoop Benchmark....Pages 73-93 Front Matter....Pages 95-95 Big Data Workloads Drawn from Real-Time Analytics Scenarios Across Three Deployed Solutions....Pages 97-104 Large-Scale Chinese Cross-Document Entity Disambiguation and Information Fusion....Pages 105-119 Front Matter....Pages 121-121 Big Data Operations: Basis for Benchmarking a Data Grid....Pages 123-137 BDGS: A Scalable Big Data Generator Suite in Big Data Benchmarking....Pages 138-154 A Multidimensional OLAP Engine Implementation in Key-Value Database Systems....Pages 155-170 Big Data Cloud-Based Advisory System....Pages 171-178 MPP SQL Engines: Architectural Choices and Their Implications on Benchmarking....Pages 179-192 Towards an Industry Standard for Benchmarking Big Data Systems....Pages 193-201 Back Matter....Pages 203-203