Big Data SMACK : A Guide to Apache Spark, Mesos, Akka, Cassandra, and Kafka

This book is about how to integrate full-stack open source big data architecture and how to choose the correct technology—Scala/Spark, Mesos, Akka, Cassandra, and Kafka—in every layer. Big data architecture is becoming a requirement for many different enterprises. So far, however, the focus has largely been on collecting, aggregating, and crunching large datasets in a timely manner. In many cases now, organizations need more than one paradigm to perform efficient analyses. __Big Data SMACK__ explains each of the full-stack technologies and, more importantly, how to best integrate them. It provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples in every situation. The book focuses on the problems and scenarios solved by the architecture, as well as the solutions provided by every technology. It covers the six main concepts of big data architecture and how integrate, replace, and reinforce every layer: * The language: Scala * The engine: Spark (SQL, MLib, Streaming, GraphX) * The container: Mesos, Docker * The view: Akka * The storage: Cassandra * The message broker: Kafka **What you’ll learn** * How to make big data architecture without using complex Greek letter architectures. * How to build a cheap but effective cluster infrastructure. * How to make queries, reports, and graphs that business demands. * How to manage and exploit unstructured and No-SQL data sources. * How use tools to monitor the performance of your architecture. * How to integrate all technologies and decide which replace and which reinforce. **Who This Book Is For** This book is for developers, data architects, and data scientists looking for how to integrate the most successful big data open stack architecture and how to choose the correct technology in every layer. Learn how to integrate full-stack open source big data architecture and to choose the correct technology—Scala/Spark, Mesos, Akka, Cassandra, and Kafka—in every layer. Big data architecture is becoming a requirement for many different enterprises. So far, however, the focus has largely been on collecting, aggregating, and crunching large data sets in a timely manner. In many cases now, organizations need more than one paradigm to perform efficient analyses. Big Data SMACK explains each of the full-stack technologies and, more importantly, how to best integrate them. It provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples in every situation. This book focuses on the problems and scenarios solved by the architecture, as well as the solutions provided by every technology. It covers the six main concepts of big data architecture and how integrate, replace, and reinforce every layer: The language: Scala The engine: Spark (SQL, MLib, Streaming, GraphX) The container: Mesos, Docker The view: Akka The storage: Cassandra The message broker: Kafka What You Will Learn: Make big data architecture without using complex Greek letter architectures Build a cheap but effective cluster infrastructure Make queries, reports, and graphs that business demands Manage and exploit unstructured and No-SQL data sources Use tools to monitor the performance of your architecture Integrate all technologies and decide which ones replace and which ones reinforce Who This Book Is For: Developers, data architects, and data scientists looking to integrate the most successful big data open stack architecture and to choose the correct technology in every layer Integrate full-stack open-source fast data pipeline architecture and choose the correct technology--Spark, Mesos, Akka, Cassandra, and Kafka (SMACK)--in every layer. Fast data is becoming a requirement for many enterprises. So far, however, the focus has largely been on collecting, aggregating, and crunching large data sets in a timely manner. In many cases organizations need more than one paradigm to perform efficient analyses. Big Data SMACK explains each technology and, more importantly, how to integrate them. It provides detailed coverage of the practical benefits of these technologies and incorporates real-world examples. The book focuses on the problems and scenarios solved by the architecture, as well as the solutions provided by each technology. This book covers the five main concepts of data pipeline architecture and how to integrate, replace, and reinforce every layer: The engine: Apache Spark The container: Apache Mesos The model: Akka< The storage: Apache Cassandra The broker: Apache Kafka Front Matter....Pages i-xxv Front Matter....Pages 1-1 Big Data, Big Challenges....Pages 3-7 Big Data, Big Solutions....Pages 9-16 Front Matter....Pages 17-17 The Language: Scala....Pages 19-40 The Model: Akka....Pages 41-66 Storage: Apache Cassandra....Pages 67-95 The Engine: Apache Spark....Pages 97-130 The Manager: Apache Mesos....Pages 131-163 The Broker: Apache Kafka....Pages 165-203 Front Matter....Pages 205-205 Fast Data Patterns....Pages 207-224 Data Pipelines....Pages 225-250 Glossary....Pages 251-258 Back Matter....Pages 259-264 Part 1. Introduction -- Chapter 1. Big Data, Big Problems -- Chapter 2. Big Data, Big Solutions -- Part 2. Playing SMACK -- Chapter 3. The Language: Scala -- Chapter 4. The Model: Akka -- Chapter 5. Storage. Apache Cassandra -- Chapter 6. The View -- Chapter 7. The Manager: Apache Mesos -- Chapter 8. The Broker: Apache Kafka -- Part 3. Improving SMACK -- Chapter 9. Fast Data Patterns -- Chapter 10. Big Data Pipelines -- Chapter 11. Glossary