Next Generation Databases: NoSQL, NewSQL, and Big Data: What every professional needs to know about the future of databases in a world of NoSQL and Big Data

Contents at a Glance 5 Contents 6 About the Author 15 About the Technical Reviewer 16 Acknowledgements 17 Part I: Next Generation Databases 18 Chapter 1: Three Database Revolutions 19 Early Database Systems 20 The First Database Revolution 22 The Second Database Revolution 23 Relational theory 24 Transaction Models 25 The First Relational Databases 26 Database Wars! 26 Client-server Computing 27 Object-oriented Programming and the OODBMS 27 The Relational Plateau 29 The Third Database Revolution 29 Google and Hadoop 30 The Rest of the Web 30 Cloud Computing 31 Document Databases 31 The “ NewSQL ” 32 The Nonrelational Explosion 32 Conclusion: One Size Doesn’t Fit All 33 Notes 34 Chapter 2: Google, Big Data, and Hadoop 36 The Big Data Revolution 36 Cloud, Mobile, Social, and Big Data 37 Google: Pioneer of Big Data 38 Google Hardware 38 The Google Software Stack 40 More about MapReduce 41 Hadoop: Open-Source Google Stack 42 Hadoop’s Origins 43 The Power of Hadoop 43 Hadoop’s Architecture 44 HBase 47 Hive 49 Pig 51 The Hadoop Ecosystem 52 Conclusion 52 Notes 52 Chapter 3: Sharding, Amazon, and the Birth of NoSQL 53 Scaling Web 2.0 53 How Web 2.0 was Won 54 The Open-source Solution 54 Sharding 55 Death by a Thousand Shards 57 CAP Theorem 57 Eventual Consistency 58 Amazon’s Dynamo 59 Consistent Hashing 61 Tunable Consistency 63 Dynamo and the Key-value Store Family 65 Conclusion 65 Note 65 Chapter 4: Document Databases 66 XML and XML Databases 67 XML Tools and Standards 67 XML Databases 68 XML Support in Relational Systems 70 JSON Document Databases 70 JSON and AJAX 70 JSON Databases 71 Data Models in Document Databases 73 Early JSON Databases 74 MemBase and CouchBase 74 MongoDB 74 JSON, JSON, Everywhere 76 Conclusion 76 Chapter 5: Tables are Not Your Friends: Graph Databases 77 What is a Graph? 77 RDBMS Patterns for Graphs 79 RDF and SPARQL 80 Property Graphs and Neo4j 81 Gremlin 83 Graph Database Internals 85 Graph Compute Engines 85 Conclusion 86 Chapter 6: Column Databases 87 Data Warehousing Schemas 87 The Columnar Alternative 89 Columnar Compression 91 Columnar Write Penalty 91 Sybase IQ, C-Store, and Vertica 93 Column Database Architectures 93 Projections 94 Columnar Technology in Other Databases 96 Conclusion 97 Note 97 Chapter 7: The End of Disk? SSD and In-Memory Databases 98 The End of Disk? 98 Solid State Disk 99 The Economics of Disk 100 SSD- Enabled Databases 101 In-Memory Databases 102 TimesTen 103 Redis 104 SAP HANA 106 VoltDB 108 Oracle 12c “in-Memory Database” 109 Berkeley Analytics Data Stack and Spark 110 Spark Architecture 112 Conclusion 113 Note 113 Part II: The Gory Details 114 Chapter 8: Distributed Database Patterns 115 Distributed Relational Databases 115 Replication 117 Shared Nothing and Shared Disk 117 Nonrelational Distributed Databases 120 MongoDB Sharding and Replication 120 Sharding 120 Sharding Mechanisms 121 Cluster Balancing 123 Replication 123 Write Concern and Read Preference 125 HBase 125 Tables, Regions, and RegionServers 126 Caching and Data Locality 127 Rowkey Ordering 128 RegionServer Splits, Balancing, and Failure 129 Region Replicas 129 Cassandra 129 Gossip 129 Consistent Hashing 130 Order-Preserving Partitioning 134 Replicas 134 Snitches 136 Summary 136 Chapter 9: Consistency Models 137 Types of Consistency 137 ACID and MVCC 138 Global Transaction Sequence Numbers 140 Two-phase Commit 140 Other Levels of Consistency 140 Consistency in MongoDB 141 MongoDB Locking 141 Replica Sets and Eventual Consistency 142 HBase Consistency 142 Eventually Consistent Region Replicas 142 Cassandra C onsistency 144 Replication Factor 144 Write Consistency 144 Read Consistency 145 Interaction between Consistency Levels 145 Hinted Handoff and Read Repair 146 Timestamps and Granularity 147 Vector Clocks 148 Lightweight Transactions 150 Conclusion 153 Chapter 10: Data Models and Storage 155 Data Models 155 Review of the Relational Model of Data 156 Key-value Stores 158 Convergent Replicated Data Types 158 Data Models in BigTable and HBase 161 Column Family Structure 161 Versions 162 Cassandra 163 Cassandra Collections 165 JSON Data Models 166 Binary JSON (BSON) 167 Storage 167 Typical Relational Storage Model 168 Log-structured Merge Trees 170 SSTables and Bloom Filters 171 Updates and Tombstones 172 Compaction 172 Secondary Indexing 173 DIY Secondary Indexing 173 Global and Local Indexes 175 Secondary Indexing Implementations in NoSQL Databases 176 Conclusion 176 Chapter 11: Languages and Programming Interfaces 177 SQL 177 NoSQL APIs 179 Riak 179 Hbase 181 MongoDB 183 Cassandra Query Language ( CQL) 185 MapReduce 187 Pig 189 Directed Acyclic Graphs 191 Cascading 191 Spark 191 The Return of SQL 192 Hive 193 Impala 194 Spark SQL 195 Couchbase N1QL 195 Apache Drill 198 Other SQL on NoSQL 200 Conclusion 200 Note 200 Chapter 12: Databases of the Future 201 The Revolution Revisited 201 Counterrevolutionaries 202 Have We Come Full Circle? 203 An Embarrassment of Choice 204 Can We have it All? 205 Consistency Models 205 Schema 206 Database Languages 208 Storage 209 A Vision for a Converged Database 210 Meanwhile, Back at Oracle HQ ... 211 Oracle JSON Support 212 Accessing JSON via Oracle REST 214 REST Access to Oracle Tables 216 Oracle Graph 217 Oracle Sharding 218 Oracle as a Hybrid Database 220 Other Convergent Databases 220 Disruptive Database Technologies 221 Storage Technologies 221 Blockchain 222 Quantum Computing 223 Conclusion 224 Notes 225 Appendix A: Database Survey 226 Aerospike 226 Cassandra 227 CouchBase 228 DynamoDB 228 HBase 229 MarkLogic 230 MongoDB 230 Neo4J 231 NuoDB 232 Oracle RDBMS 232 Redis 233 Riak 234 SAP HANA 234 TimesTen 235 Vertica 236 VoltDB 236 Index 238 www.it-ebooks.info "It's not easy to find such a generous book on big data and databases. Fortunately, this book is the one." Feng Yu. Computing Reviews. June 28, 2016. This is a book for enterprise architects, database administrators, and developers who need to understand the latest developments in database technologies. It is the book to help you choose the correct database technology at a time when concepts such as Big Data, NoSQL and NewSQL are making what used to be an easy choice into a complex decision with significant implications. The relational database (RDBMS) model completely dominated database technology for over 20 years. Today this "one size fits all" stability has been disrupted by a relatively recent explosion of new database technologies. These paradigm-busting technologies are powering the "Big Data" and "NoSQL" revolutions, as well as forcing fundamental changes in databases across the board. Deciding to use a relational database was once truly a no-brainer, and the various commercial relational databases competed on price, performance, reliability, and ease of use rather than on fundamental architectures. Today we are faced with choices between radically different database technologies. Choosing the right database today is a complex undertaking, with serious economic and technological consequences. Next Generation Databases demystifies today's new database technologies. The book describes what each technology was designed to solve. It shows how each technology can be used to solve real word application and business problems. Most importantly, this book highlights the architectural differences between technologies that are the critical factors to consider when choosing a database platform for new and upcoming projects. Introduces the new technologies that have revolutionized the database landscape Describes how each technology can be used to solve specific application or business challenges Reviews the most popular new wave databases and how they use these new database technologies