Before Machine Learning, Volume 2 - Calculus

Does the complexity of calculus in machine learning leave you feeling lost in a thicket of equations? Are you eager to find a guide that maps out this territory with clarity and ease? Enter a unique exploration where the world of calculus is demystified through the fascinating biology of bees, offering a perspective on mathematics that is as enlightening as it is unexpected. In this book, I take you on a journey through the mathematical landscapes of derivatives, gradients, and algorithms, illuminated by the natural wisdom of bees. Drawing parallels between the meticulous behaviors of these remarkable insects and the principles of calculus, I present a narrative that is rich with insight and alive with humor. This is not a mere textbook—it's a dialogue. It's a story told through the lens of bee biology, where every concept from gradient descent to neural networks is related back to the intuitive understanding of nature's own algorithms. Together we will find the connections between the disciplined dance of bees and the structured world of mathematics. Here's what to expect: A fresh take on calculus, viewing complex concepts through the simplicity and order of bee behavior. An engaging journey from the basics of calculus to its application in machine learning algorithms like RMSprop, Momentum, and ADAM. Key algorithms like linear regression and neural networks. Coding exercises and practical tasks that mirror the principles you'll learn. Prepare to embark on a transformative journey into the heart of calculus. With bees as our guides, we'll unlock the secrets of machine learning and reveal the mathematical patterns that underpin the digital world. Let's take flight on this adventure together, where the buzz of bees brings mathematics to life. Contents Contents 3 1 Why Calculus? 8 2 Pointing Fingers and Crossing Lines: The last breath of just linearity. 10 2.1 Swipe Right for co-domains: Functions. . . . . . . . 18 3 Changing Times and Tangent Lines: The Derivative. 26 3.1 The Tale of the Eight with Wobbly Knees: Infinity. 44 3.1.1 Forever Close, Never Met: The Untouchables Asymptotes. . . . . . . . . . . . . . . . . . . 52 3.1.2 Better Sharpen Our Pencils: Continuity, The No-Lift Zone! . . . . . . . . . . . . . . . . . . 58 3.1.2.1 Boomerang Bonanza: The Inverse Function’s Return Trip. . . . . . . 58 3.1.2.2 The Mood Swings of the Graph World: Piecewise Functions. . . . . . . . 61 3.2 Claustrophobia in the Calculus Corridor: The Limit. 63 3.3 Deciphering the Undefined - L’Hopital’s Rule. . . . 77 3.3.1 After You, Please: The Cordial Conduct of Function Composition. . . . . . . . . . . . 86 3.3.1.1 The Key to the Calculus Handcuffs: Chain Rule. . . . . . . . . . . . . . 86 3.4 Get Me Higher and I’ll Give You Better Curves: Taylor Series. . . . . . . . . . . . . . . . . . . . . . . . 94 3.5 Valleys and Hills: A Critical Situation for Points and Their Roots. . . . . . . . . . . . . . . . . . . . 105 3.5.1 Zeroing in: Newton-Raphson Method. . . 114 3.5.2 The Original GPS: Gradient Descent. . . . 134 3.5.3 Dare You Cross Me, Line? Convexity. . . . . 139 3.5.4 Really?! Dishing Out More Dough? The Cost Function. . . . . . . . . . . . . . . . . 143 4 Cleaning Up The Derivatives Debris: The Integral. 157 4.1 That Riemann Guy Is a Bit Square, Isn’t He? . . . 162 4.2 This seems important: The Fundamental Theorem of Calculus. . . . . . . . . . . . . . . . . . . . 167 4.3 Hold On, Logs Aren’t Just for Fireplaces? Scaling Data. . . . . . . . . . . . . . . . . . . . . . . . . . . 194 5 A Free Upgrade: More Dimensions. 200 5.1 Numerous Entries But Single Exit: Scalar Functions.201 5.1.1 Grab Your 3D Glasses for these Multidimensional Integrals. . . . . . . . . . . . . . . . . 202 5.1.2 I Am Quite Partial to These Derivatives. . 205 5.1.3 Just Follow My Lead: The Gradient. . . . . 208 5.1.4 GPS 2.0: Multidimensional Gradient Descent. . . . . . . . . . . . . . . . . . . . . . . 211 5.1.5 Bending Realities, Unveiling Curvatures: The Hessian. . . . . . . . . . . . . . . . . . . . . 212 5.1.6 I Thought We Were Past Lines: Linear Regression. . . . . . . . . . . . . . . . . . . . . 219 5.1.7 Newton, Raphson, and Hessian Walk Into a Bar: No Joke, They Found a Root! . . . . . 226 5.1.8 A Mathematical Marriage: Scalar Functions and Neural Networks. . . . . . . . . . . . . 232 5.1.8.1 Wake Up Call for Neurons: Curve Alert with Activation Functions. . 243 5.1.8.2 Wait, Which Way? The Min-Max Scaling. . . . . . . . . . . . . . . . . 249 5.1.8.3 The Price of Disorder: Cross Entropy as a Cost Function. . . . . . 257 5.1.8.4 A Detour Through Randomness: Stochastic Gradient Descent. . . . . . . . 260 5.1.8.5 Upgrading Our Locksmith Skills: The Multivariate Chain Rule. . . . . 262 3 Contents 5.1.8.6 One Step Back to Give Two Steps Forward: Backpropagation. . . . . 269 5.1.8.7 Customized Routes to Minima: RMSProp. . . . . . . . . . . . . . . . . 278 5.2 A Crowd Comes In, A Parade Goes Out: Vector Functions. . . . . . . . . . . . . . . . . . . . . . . . 286 5.2.1 A Clandestine Mathematical Liaison: Vector Functions and Neural Networks. . . . . . 294 5.2.1.1 Gathering Steam: The Momentum Algorithm. . . . . . . . . . . . . . . 303 5.2.2 Smooth Sailing with a Customized Map: Embarking on the Adaptive Moment Estimation. . . . . . . . . . . . . . . . . . . . . . . . 310