PySpark Recipes : A Problem-Solution Approach with PySpark2

Quickly find solutions to common programming problems encountered while processing big data. Content is presented in the popular problem-solution format. Look up the programming problem that you want to solve. Read the solution. Apply the solution directly in your own code. Problem solved! __PySpark Recipes__ covers Hadoop and its shortcomings. The architecture of Spark, PySpark, and RDD are presented. You will learn to apply RDD to solve day-to-day big data problems. Python and NumPy are included and make it easy for new learners of PySpark to understand and adopt the model. **What You Will Learn** * Understand the advanced features of PySpark2 and SparkSQL * Optimize your code * Program SparkSQL with Python * Use Spark Streaming and Spark MLlib with Python * Perform graph analysis with GraphFrames **Who This Book Is For** Data analysts, Python programmers, big data enthusiasts Quickly find solutions to common programming problems宣ountered while processing big data. Content is presented in the popular problem-solution format. Look up the programming problem that you want to solve. Read the solution. Apply the solution directly in your own code. Problem solved! PySpark Recipes㯶ers Hadoop and its shortcomings. The architecture of Spark, PySpark, and RDD are presented. You will learn to apply RDD to solve day-to-day big data problems. Python and NumPy are included and make it easy for new learners of PySpark to understand and adopt the model. What You Will Learn: ծderstand the advanced features of PySpark and SparkSQL Optimize your code Program SparkSQL with Python Use Spark Streaming and Spark MLlib with Python Perform graph analysis with GraphFrames Front Matter ....Pages i-xxiii The Era of Big Data, Hadoop, and Other Big Data Processing Frameworks (Raju Kumar Mishra)....Pages 1-14 Installation (Raju Kumar Mishra)....Pages 15-44 Introduction to Python and NumPy (Raju Kumar Mishra)....Pages 45-83 Spark Architecture and the Resilient Distributed Dataset (Raju Kumar Mishra)....Pages 85-114 The Power of Pairs: Paired RDDs (Raju Kumar Mishra)....Pages 115-136 I/O in PySpark (Raju Kumar Mishra)....Pages 137-161 Optimizing PySpark and PySpark Streaming (Raju Kumar Mishra)....Pages 163-185 PySparkSQL (Raju Kumar Mishra)....Pages 187-233 PySpark MLlib and Linear Regression (Raju Kumar Mishra)....Pages 235-259 Back Matter ....Pages 261-265